In this article, a case study on the design, development, and evaluationof a multimedia-based performance assessment (MBPA) for measuring confinedspace guards’ skills is presented. A confined space guard (CSG) supervises oper‐ations that are carried out in a confined space (e.g. a tank or silo). Currently,individuals who want to become a certified CSG in The Netherlands have toparticipate in a one day training program and have to pass both a knowledge-based MC test and a practice-based performance-based assessment (PBA). Ourgoal is to measure the skills that are currently being assessed through the PBA,with the MBPA. We first discuss the design and development of the MBPA.Secondly, we present an empirical study which was used for assessing the qualityof our measurement instrument. A representative sample of 55 CSG students,who had just completed the one day training program, has subsequentlyperformed in the MC test, and then, depending on the condition they wereassigned, the PBA or the MBPA. We report the psychometric properties of theMBPA. Furthermore, using correlations and regression analysis, we make anempirical comparison between students’ scores on the PBA and the MBPA. Theresults show that students’ scores on the PBA and the MBPA are significantlycorrelated and that students’ MBPA score is a good predictor for their score onthe PBA. In the discussion, we provide implications and directions for futureresearch and practice into the field of MBPA.