Review of Mislevy,Behrens, Dicerbo, and Levy's "Design and discovery in educational assessment: Evidence-centered design, psychometrics, and educational data mining."

Mislevy, Behrens, Dicerbo, and Levy’s (2012) article is highly technical and rather difficult to grasp if you are a novice to complex concepts around assessment and analysis such as myself.  So, this review will simply provide a rudimentary summary of overarching message Mislevy et al. (2012) attempt to convey to readers such as myself.  As it states in the title of the article, the focus is on educational assessment.

Mislevy et al. (2012) make it a point to compare evidence centered design (ECD) to assessment in the “standard assessment paradigm” (p. 14).  Mislevy et al. (2012) use the terms, “highly scripted” and “constrained” to describe assessments in the standard assessment paradigm (p. 14).  Comparatively, Mislevy et al. (2012) describe ECD as a flexible framework for describing a wide variety of methods and objective associated with educational assessments.  This is important because technology is diversifying the way content is being taught and therefor prompting a need to diversify how student proficiency is being assessed.  With the use of technology, there is an opportunity to generate more meaningful assessment data that is captured continuously throughout the learning process versus inconsistently and intermittently.

Psychometrics has been used in educational testing, however, Mislevy et al. (2012) take a stance that it has “focused on data produced in the standard assessment paradigm” so there is opportunity to leverage psychometrics differently within the ECD framework.  The same goes for educational data mining (EDM) methods.  Mislevy et al. (2012) emphasize the importance of eliminating the current limitations of how EDM is used to inform the design of assessments. Such limitations involve focusing only on specific outputs and inputs as it relates to scoring processes.   Rather, it is proposed that EDM is used to broaden how data is being analyzed.

Image Rights: https://creativecommons.org/licenses/by-nc/2.0/,
Provided by: Kevin Whytock (https://www.flickr.com/photos/7815007@N07/)

In sum, this article encourages a limitless and creative approach to assessing student proficiency through the use of more flexible psychometrics and EDM methods.  Mislevy et al. (2012) offer a framework that fosters innovative approaches to using the tools and strategies that are already integrated in the assessment process making it seem like a doable task.

My dangerously simplified translation…use technology as a tool to obtain student proficiency measures throughout the learning process, integrate psychometric and EDM methods to organize the assessment data, analyze the data, use it to guide the design your assessment methods, embed those methods into your content, then repeat.  The importance of this process is to use technology to assess more accurately and maintain rigor and relevance.

Reference:

Mislevy, R. J., Behrens, J. T., Dicerbo, K. E., & Levy, R.  (2012).  Design and discovery in educational assessment: Evidence-centered design, psychometrics, and educational data mining.  Journal of Educational Data Mining (4)1 11-48.