Mislevy,
Behrens, Dicerbo, and Levy’s (2012) article is highly technical and rather
difficult to grasp if you are a novice to complex concepts around assessment
and analysis such as myself. So, this review will simply
provide a rudimentary summary of overarching message Mislevy et al. (2012)
attempt to convey to readers such as myself.
As it states in the title of the
article, the focus is on educational assessment.
Mislevy et
al. (2012) make it a point to compare evidence centered design (ECD) to
assessment in the “standard assessment paradigm” (p. 14). Mislevy et al. (2012) use the terms, “highly
scripted” and “constrained” to describe assessments in the standard assessment
paradigm (p. 14). Comparatively, Mislevy
et al. (2012) describe ECD as a flexible framework for describing a wide variety
of methods and objective associated with educational assessments. This is important because technology is
diversifying the way content is being taught and therefor prompting a need to
diversify how student proficiency is being assessed. With the use of technology, there is an
opportunity to generate more meaningful assessment data that is captured
continuously throughout the learning process versus inconsistently and intermittently.
Psychometrics
has been used in educational testing, however, Mislevy et al. (2012) take a
stance that it has “focused on data produced in the standard assessment
paradigm” so there is opportunity to leverage psychometrics differently within
the ECD framework. The same goes for educational
data mining (EDM) methods. Mislevy et
al. (2012) emphasize the importance of eliminating the current limitations of
how EDM is used to inform the design of assessments. Such limitations involve
focusing only on specific outputs and inputs as it relates to scoring
processes. Rather, it is proposed that EDM is used to
broaden how data is being analyzed.
 |
Image Rights: https://creativecommons.org/licenses/by-nc/2.0/,
Provided by: Kevin Whytock (https://www.flickr.com/photos/7815007@N07/) |
In sum, this
article encourages a limitless and creative approach to assessing student
proficiency through the use of more flexible psychometrics and EDM methods. Mislevy et al. (2012) offer a framework that
fosters innovative approaches to using the tools and strategies that are
already integrated in the assessment process making it seem like a doable task.
My dangerously
simplified translation…use technology as a tool to obtain student proficiency
measures throughout the learning process, integrate psychometric and EDM
methods to organize the assessment data, analyze the data, use it to guide the design
your assessment methods, embed those methods into your content, then repeat. The importance of this process is to use
technology to assess more accurately and maintain rigor and relevance.
Reference:
Mislevy, R. J., Behrens, J. T., Dicerbo, K. E., & Levy, R. (2012). Design and discovery in educational assessment: Evidence-centered design, psychometrics, and educational data mining. Journal of Educational Data Mining (4)1 11-48.
