The Vagaries of Assessment

Assessment is a hot button topic for many in the higher ed world, and the discussions about assessment can be very heated and passionate. The purpose of this post is to give a broad overview of assessment as it is often thought of in a very narrow way.

Each of us makes many assessments each and every day of our lives whether we are in teaching or not. People assess each and every day. Avoiding assessment is improbable. Engaging in the dialog about assessment can shape what form it will take in higher ed.

What Assessment Is and Is Not

As one of its definitions of assessment, Merriam-Webster has “the act of making a judgment about something.” In its US English section, Oxford Dictionaries online has as a definition for assessment “the evaluation or estimation of the nature, quality, or ability of someone or something.” Note that a “judgement” or “evaluation” is being made of “someone” or “something” else. This is important because many of the discussions I have been part of or read about seem to focus on what a particular type of assessment looks like (e.g., standardized tests).

Assessment may be quantitative and/or qualitative. Assessment may be frequent and/or infrequent. Assessment may be standardized and/or not. Assessment can be limiting and/or freeing. Assessment is just what people do. People make it good or bad.

Standardized testing is currently a widely used assessment method in the USA in the K-12 sector, and many colleges and universities use the ACT or SAT as part of their admissions process. Graduate and professional students often encounter the GRE, LSAT or MCAT. All of these standardized tests are assessments, but not all assessments are standardized tests.

Another way to think of this is an apple is a fruit, but not every fruit is an apple. This may seem trivial, but many have fixed in their minds that assessments are always standardized, summative tests.

In addition, assessment is being done to them by someone else. They often forget (or are not aware) that they make assessments each and every day about others as we all do.

Formative and Summative

Formative assessments are done along the way to help foster growth. These may be comments on an iterative writing assignment, reviews made during the pilot of a process and the feedback an employee may receive from a supervisor. Formative assessments are meant to be developmental.

Summative assessments are done at intervals and are used as “reports” of some kind. Examples include students’ recorded scores on tests (quantitative or qualitative), quarterly profit of a company and the percentage of college students graduating in 6 years or less. Summative assessments are meant to be measurements that tell something.

The same type of assessment may be used in a formative or summative manner. A quiz (quantitative or qualitative) may be used to help a student understand how he or she is doing with the material (formative). Another assessor may use the same quiz to determine and record how the students measure up (summative). Up against what may vary, and is a separate topic for discussion.

Learning, Process and Program

Assessments may be used to assess learning, a process or a program. Much debate abounds about the ability to assess learning. This depends upon how learning is defined. Sometimes, learning is about memorizing facts. An example I keep in my mind is a conversation I had with a faculty member about the need for individual assessment of facts. The faculty member pointed out that in a first responder course, each student did need to know some memorized facts, and this needed to be assessed.

Imagine a first responder who shows up to the scene of an accident, pulls out a smartphone, is fortunate to have a data connection and googles “what do I do with a bony looking thing sticking out of a leg?” Is this what is desired of a first responder? Just for fun, be sure to try this search.

Some learning cannot be assessed because the learning may not occur for some time after the formal class, workshop or interaction ends. This is where assessing the process can be very useful. Was the environment as conducive to setting the stage for this future learning? I think of planting a garden in the spring. The results will come later (or not at all), but I can make sure I have prepared the garden and care for it as best as possible by assessing what I am doing. If I am haphazard in my garden preparation and care, the results will likely be haphazard.

Program assessment looks at what a collection of courses (e.g., a college major or degree), initiatives (e.g., domestic abuse interventions, road repair) or units (e.g., municipal services, corporate profit centers) achieve. Program assessment is a higher level look to make sure all the parts that make up the program are resulting in the desired outcome(s).

Making the Implicit Explicit

The pattern that develops out of learning, process and program assessment is one of assessing if what is intended is being achieved. In order to do this, the implicit needs to be made explicit. Making the implicit explicit is often a difficult, messy experience. This requires looking deep within to reveal what is truly wanted, and often can be very difficult to articulate.

Assessment can occur without the implicit being made explicit as this happens all the time, but getting to what is intended is haphazard until the assessor has wrestled with what he or she truly wants as a result.

Intended results may be very specific or as simple as an experience. The achievement of either can be assessed.

Grades, Although Assessments, Are Meaningless

Grades are meaningless. This statement is difficult for many of us because we want to believe we know what a particular grade means, and that its meaning is universal.

An “A” has got to mean the learner excelled! Right?

Did the learner excel? What was the assessment(s) used to arrive at the “A”? Does an “A” in section 1 of a course mean the same as an “A” in section 2 of the same course? Does an “A” in one department mean the same as an “A” in another department? Should they even mean the same thing?

What if letter grades were discarded, and a Pass/Fail system were used? What does “Pass” mean? Joe passed Accounting! What kind of accounting can Joe now do?

What do grades tell us about the learner’s:

  • ability to interact with others?
  • creativity?
  • determination?

Even if a college has a cohesive meaning for what its letter grades represent, is this meaning communicated to those who are looking at the grades?

But Grades May Be Useful

Recently, 2 colleagues stated how they use grades in a useful fashion. Their basic premise is grades are not going away any time soon. Given this, grades may be used with learners to:

  • serve as a guide post on a journey of learning.
  • provide motivation to act on formative feedback.

In both cases, the final grade may be meaningless in terms of what it represents, but it was useful in helping the learner progress. What they taught me was that because letter grade assessment was not going away, it should be used as best as it can be used. Good point. Let me state this more broadly:

Assessment is happening (people assess each and every day), so use it to our best advantage.

Goals and Objectives

Goals and objectives help us frame what we are going to assess. Goals are the broad desires that people create for the learning, process or program. Objectives are measurable items derived from the goals. If your background uses non-measurable goals and measurable goals, this is fine. The idea is one is not measurable while the other is.

Remembering that assessment can be used for learning, process or program, establishing objectives is not as threatening as some make it out to be. If a person does not want to assess learning because the goal is an open learning environment, objectives may be created to assess if the process supports this goal.

Assessment Need Not Be Limiting

When goals and objectives are created, some argue this limits their ability to go beyond what has been stated. To a degree, this is true. The idea behind stating goals and objectives is so everyone involved understands what is supposed to be achieved. Nothing in assessment states this is all that may be achieved.

For example, a course on General Biology will likely have common goals and objectives across all sections. As long as the sections address the common goals and objectives, each section is free to go beyond these. Goals and objectives become the minimum that is done in any learning, process or program. Going beyond the minimum is always an option. Additionally, how a course fulfills these objectives may vary from section to section. Assessment need not be prescriptive although some have made it so.

Participate in the Dialog or Have It Forced by Others

Assessment is something each of us does many times a day, every day. To think that groups outside of higher ed will not assess higher ed is naive. They have already made an informal assessment of higher ed which is motivating them to make a formal assessment.

What is disconcerting to many in higher ed is the standardized nature of the assessments being suggested. Moving away from a standardized assessment will require those in higher ed to engage in a dialog on what form of assessment will be valuable to all involved.

Should we be assessing the process? When is it appropriate to assess learning, and how can this be done in a meaningful fashion? How should we assess our programs?

What is it we all value? We value something. Let us make it explicit so we and others know what we are doing.