Assessment Principles & Concepts
Adapted from the Assessment Workshop led by the Dodge Foundation President and CEO David Grant. Click on each for a full explanation.
- The primary purpose of assessment is to improve performance, not audit it.
- Good assessment requires being clear about mission and goals, the standards to which you aspire, and the criteria by which you would measure success.
- Therefore, it is about MEASURING WHAT MATTERS. (If you assess what you value, others will value what you assess.)
- And, necessarily, it becomes about PLANNING BACKWARDS.
- Assessment that improves performance involves FEEDBACK.
- One tool for getting useful feedback on what matters most is the RUBRIC.
- Good assessment requires a variety of measures, data, and feedback.
- Good assessment is ongoing. It is about continuous improvement. And unless we designate and protect the TIME to do this work, it will not happen.
- Done collectively, assessment builds community.
The primary purpose of assessment is to improve performance, not audit it.
This principle is the one on which all of the following are built. We ought to embrace it quickly, but many don’t – or can’t – and I believe it is because our long experience makes us automatically assume otherwise. Our experience tells us that assessment is about praise or blame, about success or failure, about who’s in or who’s out.
With all due respect to teachers, this experience is deeply rooted in schools. When I ask groups to think back to their high school teachers and imagine asking them for a synonym for “assessment,” most people quickly say the answer would be “test.” We are used to being “assessed” at the end of things, at the end of chapters, units, and courses.
As students, too seldom were we allowed to incorporate feedback on a specific performance into our next one. Sometimes, we got the information on “how we did” days or weeks after we did it and could barely make the connection between a letter or number grade and the performance we wish we could improve.
(The most dramatic example of this is standardized tests, which purpose is to sort students, to identify a spectrum of performance rather than improve it.)
When I used to run workshops for high school teachers, there were two groups who seemed to automatically adopt a student-centered, performance-based approach to assessment. They were athletic coaches and teachers of performing arts. For these two groups, of course, there is never any question but that the whole point is the student’s performance, not the subject being “covered,” so feedback comes frequently, with no “grades” attached, and is aimed at a well-understood and ambitious performance that is coming up soon. Which leads us to the second principle:
(Back to Top)
Good assessment requires being clear about mission and goals, the standards to which you aspire, and the criteria by which you would measure success.
To get better at something, you have to know what it is you are trying to do. This might mean something quite specific in form, like the angle of the body entering the water during a competitive dive, or quite specific in intent and impact, like inspiring a hushed silence during a theatrical performance. So getting good at assessment requires being comfortable with – more than that, expecting and demanding – questions like,
What are we trying to do here?
What would it look like if we succeeded?
What standards are we aspiring to?
By what criteria would we judge the success of our work?
These questions are at the heart of the next two principles:
(Back to Top)
Therefore, (good assessment) is about MEASURING WHAT MATTERS. (If you assess what you value, others will value what you assess.)
This is another good question to add to the ones above: “Are we measuring what matters?” Not only does this lead to a discussion about what really matters, which is a good thing to do, but it also raises the question of how you go about measuring something that resists quantification – most things that really matter do resist quantification.
The most common response from someone resisting an emphasis on assessment is “You can’t measure that.”
To which I would respond, “If you can describe it, you can measure it.”
The key here is that if we haven’t stopped long enough to ask what really matters, we end up measuring what is easiest to measure. In evaluating the success of a workshop, for example, a social service agency may say “Over a hundred people showed up.” This is an important fact, but it tells us nothing about their experience.
Or, to take it a step further, the agency may say, “In an evaluation filled out at the door, over 70% of the participants found the workshop either ‘helpful’ or ‘very helpful.’” Fair enough, but is that what matters most? No, when pressed, the group will say what matters most is whether or not the participants ever used the information of the workshop and how well they used it. This data is much harder to get; you have to have identified it as important early on, then designed a way to get it.
This has always been one of the core concepts propounded by Grant Wiggins, who helped inspire this whole effort – that good assessment is about design. To be good assessors we have to become good designers, and we have to embrace the next major principle/concept:
(Back to Top)
And, necessarily, (good assessment) becomes about PLANNING BACKWARDS.
Many of us in the non-profit world resist this notion. It seems too abstract, as opposed to the very concrete needs that should be addressed right away. We are interested in moving forward, not backwards, and as soon as possible. But let me continue the example mentioned above.
Let’s say the social service agency has asked, “What would it look like if our workshops met our highest hopes and goals?” and then answered, “People would be using the information with their clients – in fact, they would say the workshop significantly changed for the better their professional practice.”
Then, planning backwards, they would say, “How are we going to get that information?” They might come up with an incentive for participants to be part of a follow-up study; they might identify those participants before the workshop itself and bring them in on the design of pre- and post-workshop activity.
Another benefit of planning backwards in a project is that it forces you to do the work mentioned in #2 above. Being clear about mission and goals, standards and criteria then help keep you on track. Things come up all the time that demand our time and energy, and we need to have some way of saying, “This is what we have decided is important. This is what we have decided to do, and this is what we have decided not to do, however worthy that activity might be.”
I see these first four principles/concepts as a unit. The first is about purpose, and the next three work together to form a framework for understanding how to proceed.
With these four as touchstones, we proceed as assessment designers to the next two, more specific items:
(Back to Top)
Assessment that improves performance involves FEEDBACK.
Feedback is a central concept of a good assessment system, but we tend to think about it in ways that limit the usefulness of the idea.
First, we use the word too broadly. If someone says, “Good job,” we say, “Thanks for the feedback.” But for our purposes here, let’s not call that feedback, because, however good it makes us feel, it is not useful in helping us get better at whatever it was we were doing.
Second, we tend to think that we are already getting it. True – usually from our best friends and our staunchest critics. It is worth asking: “Who aren’t we hearing from, because we haven’t set up a way to hear from them?”
Also, we tend to picture feedback as coming verbally, from people. Another terrific source of feedback is data, which requires us to ask, “Are we getting the data that would help us know if we are being effective?”
It is a useful exercise to ask ourselves how we would finish this sentence: “To be exemplary feedback, it must be ________________.” Some people quickly think of words like “honest” or “constructive” or “tactful,” but I would argue these words come in response to a sense that feedback is too often not these things. I think the adjectives that help us think about feedback most productively are:
- Timely. You need to get information when there is still time to use it, not when it is too late.
- Descriptive. We need to defuse the idea that feedback is either praise or blame. But for someone to describe what he sees, it helps to know what the person getting the feedback is trying to do … therefore, I would add,
- Contextual. The best feedback systems are set up after doing the work in the second principle presented above – the identification of goals, standards, and criteria for success. Within that context, we can ask for feedback that describes where our current efforts seem to be in relation to our vision of where we want to be.
One tool for getting useful feedback on what matters most is the RUBRIC
If describing what good work “looks like” would be helpful, we should have a tool, or a format in which to do so. I am using the word “rubric” here to mean just such a tool. In the Dodge assessment workshops, we spend time learning how to design and build rubrics, a process I will not try to replicate in this brief commentary.
However, we have included examples of rubrics in the “Models” section of this web site, and they will demonstrate how various organizations have used rubrics to: get more specific about what they are trying to do; and measure what matters.
Rubric-writing takes time, so it is a practical necessity to decide which ones are worth the time and effort. From an assessor’s point of view, they should be written on those aspects of an organization’s (or individual’s) performance where fostering improvement would make the most difference in achieving mission, goals, and objectives. A danger with “new” rubric writers is that they want to write them on everything from breakfast cereals to late night television.
Rubrics are great, but they are only one of the tools available to us. They are probably most useful in the planning stages of a project, or for feedback within an organization, where everyone understands the rubric. It bears remembering that…
(Back to Top)
Good assessment requires a variety of measures, data, and feedback.
There is an axiom in the assessment world that no single assessment instrument can tell you everything you want to know. So it helps to be familiar with a variety of tools:
- questionnaires,
- surveys,
- interviews,
- focus groups,
- follow-up studies,
- use of outside evaluators, etc.
As we build the Models section of this web-site, we hope to provide examples of many of these, along with commentary on their use.
There are two other principles/concepts that I believe are very important in a productive approach to assessment. They are:
(Back to Top)
Good assessment is ongoing. It is about continuous improvement. And unless we designate and protect the TIME to do this work, it will not happen.
This principle is a return to the first one. It reminds us that when the purpose of assessment is to improve performance, we stop seeing it as a one-shot, high-stakes judgment.
Rather than seeing assessment as something we do “at the end,” we see it as something that is built into the work – it informs and improves the work. And, in the process, it can have a result that people were not expecting, which is…
(Back to Top)
Done collectively, assessment builds community.
When assessment is approached in the spirit of these principles and concepts above, it becomes about institutional and personal development.
That is why I worry when organizations have an individual “in charge” of assessment. Unless that person is a great teacher, with the job of having everyone in the organization “get it,” the temptation for others is to say, “Oh, she handles evaluation. I don’t need to think about it.”
That’s a wasted opportunity. The theme of Dodge’s initiative here is that organizations are better off if they become assessment cultures. It can only happen gradually, as ways of thinking and behaving slowly become habits. The best thing about it, though, is that the process makes its own case.
As we apply the concepts above to our external work, the work gets better. As we apply them to our internal workings, we shift the emphasis away from personalities and onto a collective vision and ever-increasingly effectiveness in reaching it. It puts at the center of our relationships with each other our mutual striving towards worthy and ambitious goals.
What is more unifying, more edifying, and more fun than asking what our mission really is, what matters most to us, and how we can help each other achieve a vision of good work?
In short, when these principles and practices become habits in an organization, people apply them to both their internal and external worlds. They create an assessment culture.