Assessment Insights

How Do You Measure What Someone Knows?

[fa icon="calendar"] May, 2016 / by Press Room

Press Room

If you work in assessment, you know that the answer to this question is far from simple. Early this month our director of Psychometrics, Dr. Jennifer Dunn, gave a talk at TEDxPiscataquaRiver in Portsmouth, NH. Titled “Using Data to Motivate Change,” Dr. Dunn described her own introduction to psychometrics, and how the challenge of measuring the intangible—what a person knows—captured her interest. Here are some of the main points of her talk.

The three “pillars” of good testing

  • Reliability and validity underpin the quality of the measure
  • Define the story and gather evidence to support the design
  • Use the results to articulate a call to action

The complexity of measuring a construct well

For example, if a student answers 2 + 2 = ? correctly, does this tell us that

the student can add? No.

the student can add single-digit numbers? No.

the student can add to 4? No.

So what can we confidently conclude? That the student can add 2 + 2!

Understanding the “right” story behind the data

It’s tempting to use a single example—or anecdote—to sway a decision. But we need to make sure to ask: Is this anecdote a compelling exception or the rule? We need to make sure that the anecdotes we use to bring data to a human scale illustrate what the results really tell us. If we understand the stories the data tell us, we can use those stories to influence positive changes.

Watch the full talk!

Topics: Assessment Literacy, Accountability

Press Room

Written by Press Room

Our Press Room chooses topics of interest from activities around the company, to provide a closer look at ways to improve learning and instruction, and to help demystify assessment issues.