Reliability and Generalizability of an Occupational Therapy Fieldwork Assessment Tool in Japan

Document Type (Journals)

Original Research

Abstract

Occupational therapists require high-quality clinical education to develop the skills needed for safe and effective practice. In Japan, the duration of occupational therapy education is shorter than in many other countries and the rapid expansion of training schools has created new challenges for assuring the quality of fieldwork. Assessment tools used in long-term clinical training must therefore provide sufficient reliability to support fair pass–fail decisions and meaningful feedback. This study examined the reliability of two conventional learner assessment tools used in one Japanese occupational therapy program: a 17-item supervisor rating form and a 10-item faculty rating form completed after debriefing sessions. Retrospective assessment data from long-term placements between 2018 and 2021 were analyzed using generalizability theory. Variance components were estimated for students, items, assessors, and their interactions, and phi coefficients for absolute decisions were calculated. For supervisor ratings, the phi coefficient was 0.54, with large proportions of variance attributable to assessors nested within students and assessor-by-item interactions. For faculty ratings, the phi coefficient was 0.31, with most variance arising from assessor-by-item interactions and relatively little variance due to students. Both tools therefore fell below commonly recommended reliability thresholds for high-stakes summative assessment. These findings indicate that the current fieldwork assessment tools do not provide a sufficiently dependable basis for pass–fail decisions and underscore the need to clarify assessment criteria, strengthen supervisor and faculty training, and move toward a more formative, programmatic workplace-based assessment model in Japanese occupational therapy clinical education.

Biography

Keiko Takeuchi, OTR, PhD is an assistant professor in the Department of Occupational Therapy at Seijoh University, Japan. She received her PhD from the Graduate School of Medicine, The University of Tokyo. Her research focuses on occupational therapy clinical education, workplace-based assessment, and competency-based training models.

Hirotaka Onishi, MD, MHPE, PhD is a lecturer in the Graduate School of Medicine, The University of Tokyo, Japan. His research focuses on medical education, assessment, programme evaluation, clinical education, and clinical reasoning.

Declaration of Interest

The authors report no declarations of interest.

Recommended Citation

Takeuchi, K., & Onishi, H. (2026). Reliability and Generalizability of an Occupational Therapy Fieldwork Assessment Tool in Japan. Journal of Occupational Therapy Education, 10 (2). Retrieved from https://encompass.eku.edu/jote/vol10/iss2/17