Evaluation of Social Interaction (ESI) - About Calibration

Rater calibration requires that potential raters view and score videotaped ESI observations during the course and then complete 10 live observations after the course. Rater calibration allows AMPS Project International to determine each rater’s severity and whether or not he or she is scoring the ESI in a valid and reliable manner.

To become an ESI Calibrated Rater, an occupational therapist must complete the following steps:

  • Attend a three-day training course
  • Test 10 clients who perform 2 or 3 ESI tasks (at least 5 must be healthy, well persons 2 years of age or older)
  • Independently interview and score live clients (the use of video tapes is not acceptable)
  • Enter the data into the computer using the ESI computer-scoring program
  • Submit exported data using the online Rater Calibration Center within three months of taking the course

For detailed instructions, use the Rater Calibration Requirements pages of the course handouts.

The rater will receive a confirmation code when data has been successfully submited. If a rater does not receive a confirmation code, AMPS Project International did not receive the data. It is the responsibility of the occupational therapist to review data entry for accuracy. Errors may result in having to retake the course, test additional clients, or pay additional fees. Data must be exported from the ESI computer-scoring program. Backup copies of data are not accepted.

Once data is received, AMPS Project International will complete a detailed analysis of rater severity and therapist reliability as an ESI rater. When finished analyzing rater data, AMPS Project International will provide a detailed letter regarding the results of the analysis. The letter detailing the results of your data analysis will be made available for download from the Rater Calibration Center within 2-3 weeks as indicated by the Rater Calibration Center when data is received. If the analysis reveals that the rater does not demonstrate valid and reliable scoring, he or she will be asked to test an additional ten clients or, in the rare case of extreme (and invalid) rater severity, to take the course again.

Submission of data for an additional ten clients provides the potential ESI user with an additional opportunity to learn to administer and score the ESI in a valid and reliable manner. Approximately 95% for those who resubmit calibrate successfully. Those who do not calibrate as valid and reliable raters with the second set of data must retake the course if they want to use the ESI in clinical practice or research.

For technical assistance with software, please contact AMPS Project International by e-mail: info@AMPSIntl.com

This page was last modified on 11 January 2010