Morin, S., Bourdeau, L.-M., & Robert, J.-M.
Using the Gage R&R method to evaluate the reliability and assessment process of the Creative Engineering Design Assessment
Creativity, being a key competency of engineering, must be taught and assessed. Here, we verify the reliability of the Creative Engineering Design Assessment (CEDA), a psychometric test for engineering students, as well as clarify the assessment process. To test the former, the gage repeatability and reproducibility (R&R) method was applied innovatively. Our findings suggest that the use of the gage R&R method is relevant in a psychometric environment and that the measured total variation is satisfactory. In addition, control charts were used to further analyze the assessment strategy’s reliability. We demonstrated that the assessment process for the two qualitative criteria (Originality and Usefulness) of the CEDA was in control, that is, the values’ variation was caused by unpredictable but normal and inevitable events. The results demonstrate the test’s reliability according to two concepts (repeatability and reproducibility) and allow for the refinement of the assessment process by defining the Likert scales with more precision.
Testing, Psychometrics, Methodology in Applied Psychology, 2017, Vol. 24, pp. 499-514, DOI: 10.4473/TPM24.4.2