Ensure Accurate Self-Assessment in Peer, Mentor, and Coach Work

Feedback_KeyAssessment has become a billion dollar industry and is strongly fueled by the conventional wisdom that an individual’s ability to assess him or her self is typically less reliable, less accurate, or less valid than an external or expert source of assessment. Most of us have between 12 to 18 years of experience with this viewpoint as our teachers graded our work from kindergarten to graduate school. While there might have been the occasional unhappiness about a particular grade as not being fair, for the most part we learned to accept this conventional view that “teacher knows best.”

This same perspective is perpetuated in the workplace. While even the most conservative organizations will solicit employee views about their own progress, growth or development, they rely mostly on reports from supervisors, managers, co-workers, or assessment inventories to obtain “real” data. Self-assessment data is often described as “subjective,” “self-serving,” or “inaccurate.” It would be ridiculous, most managers believe, to rely exclusively on an employee’s self-evaluation of performance when the outcome of that evaluation may determine promotion, advancement or pay increases.

The vision of most mentors, peer trainers, and coaches includes a commitment to their partners’, trainees’, or clients’ ability to have accurate views of their own work, development, or skills. But what happens when a mentor’s assessment clashes with the self-assessment of the person he or she is mentoring? Or when a peer trainer’s assessment of a training participant doesn’t match the self-assessment of that person in training? Or when a coach’s view of a client and the client’s view of him or herself do not line up? In other words, even though both parties want to improve the accuracy of self-assessment, what happens that prevents this outcome?

During the 1970’s I experimented with a number of self-assessment procedures in my university courses. I was especially interested in the relationship between external assessments of learning and how much learners perceived they had gained. I wanted to know what factors influenced a difference between a learner’s own assessment and an instructor’s assessment of that learner. Since both learners and instructors were dealing with the same content, the same procedures, and the same evaluation data, why were they so often at odds with each other when it came to the assessment of progress, results, or outcomes.

Part of my motivation for this experimentation was a dissatisfaction with the traditional grading system that minimized the student role and contributed to the atrophy of student ability to make accurate self-assessments. Another element in my intention was my experience during my own graduate student days on the campus of the University of California at Berkeley during the Free Speech Movement, where “student power” became our everyday mantra. I was determined to find ways to integrate student power into the classroom and show how it actually improved academic achievement.

My first experiments were almost completely based on my Berkeley experience. “At the end of the term,” I told my first group of undergraduates, “give yourself a grade and provide a rationale.” My courses became oversubscribed by eager students willing to participate in this experiment. While many enrolled because they wanted to learn about the topic, seize the opportunity to re-vitalize their intrinsic ability to make assessments, or become associated with such a radical procedure, others enrolled for the perceived easy grade and opportunity to improve their grade point average.

Although all course outlines and grading procedures had to be submitted to department chairs, the sheer weight of such pages from all the members of the department likely prohibited my chair from anything but just filing them in a “current course outlines” folder. Thus, I had not received any feedback about my intended procedure.

When the semester came to an end, I had several hundred self-assessments to read. And as you’ve probably guessed by now, almost every student gave him or herself an “A” grade. The accompanying rationales ranged from a single sentence to several pages. I’ll leave it to your imagination as to the variety of reasons students provided for such a mark.

I should note here that students did participate in both essay exams and quick quizzes, and they also wrote a paper on a related topic of their choosing. Students received scores on their exams and feedback about their papers. Some of the most paradoxical rationale for the end-of-term self-assignment of the “A” grade came from students who scored at the low end on the exams and wrote questionable papers. Going into those details is beyond the scope of this article, but mostly the extent of their rationale was “I learned a lot.”

When I submitted my grade sheets listing the extensive number of “A” grades and then went on holidays, I was still very dissatisfied with the outcomes and thought I really hadn’t come much closer to understanding the relationship between self-assessment and actual learning, let alone how to make this happen in a practical and concrete way in the classroom.

After returning from my vacation, I was greeted by a stack of mail and a copy of the university newspaper that had as a headline: “University bans self-assessment as a grading option.” My mail included mostly requests from senior administrators to attend meetings to explain why my grade sheet was so out-of-line with all other grade sheets; followed by requests to attend the academic senate hearings on why my grade sheet had so many high grades; and followed by a request from the university president to meet and explain my rationale.

In the relatively short time I had been employed by the university, I was delighted to have such an interest in my work. Of course, it didn’t bode well for my progress towards tenure to have so many administrators and colleagues “annoyed.” (I use that term because the terms that most of my colleagues used would not be suitable for this newsletter.)

To make this long story short (oops, probably too late), I went back to all the rationale statements provided by the students and identified a number of themes. These themes led to designing a different way to integrate self-assessment into the classroom and increase the likelihood that such assessments would more accurately reflect or match actual learning. The resulting system was implemented successfully in virtually all of my courses and was continued until retirement from university teaching (with tenure).

The elements of that system are described in Carr, R. (1977). The effects of specific guidelines on the accuracy of student self-evaluation. Canadian Journal of Education, 2 (4), 65-79. This study used videotape feedback and three degrees of specific, objective instructions to impact self-assessments. Theme analysis of the original course data and continuing data from subsequent courses confirmed that the more explicit the criteria used to make the self-assessment and the more involvement in the creation of those criteria by the person making the self-assessment, the more likely that the self-assessment matched other “objective” measures of learning.

In our work as mentors, coaches, and peer supporters, if we want self-assessment to play a significant and accepted role, and we want to maximize the accuracy of self-assessment (that is, be reasonably in line with other accurate data sources), then the procedures and structure we use to gain such self-assessment data are crucial.

First, the person who will be the focus of the self-assessment must be actively involved in determining the criteria to be used to make the assessment. The criteria can be created through discussions with supervisors, managers or co-workers, but ultimately they must be criteria that both the self-assessor and the person that the self-assessor will submit the results to agree upon as being important. This often means that pre-determined instruments or previously used sets of criteria will have limited utility unless they contain items valued by both parties. “Re-inventing” the wheel each time is more likely to yield accurate self-assessments.

Second, the criteria developed must be specific, concrete or explicit enough so that all parties will understand not just “what” the criteria are, but also the way in which the criteria are relevant. Most people object to this part. They are more likely to design simple criteria and then spend more time analyzing the response to the criteria. Ambiguous criteria often receive ambiguous answers. When I was asked by a previous employer (a theme park, motion picture, and television production organization that I will not name) to assess “How much have you learned about operating ‘Ride X’?” I responded with the informative phrase, “A lot.”

While there can still be debate as to whether an individual’s own self-assessment or another person’s assessment of that individual can be considered “more accurate,” without the knowledge of the factors that contribute to better self-assessment, we have little chance to prevent individuals from missing the mark, being too critical, or ignoring their own learning.


“Life asks us to make measurable progress in reasonable time. That’s why they make those fourth-grade chairs so small – so that you won’t fit in them again at age twenty-five.”

~ Jim Rohn ~

Advertisements

2 thoughts on “Ensure Accurate Self-Assessment in Peer, Mentor, and Coach Work

  1. We designed and have used successfully for 15 years, an instrument for comparing self-analysis and a mentor’s analysis for the development of self-esteem in a variety of cohorts. The differences are displayed in a series of 6 graphs. The graphs represent the self-analysis of 24 indicators made available online that will highlight strengths and uncover areas limiting performance. They guide the person and their mentor, (coach or manager) with questions, explanations and strategies. Together, they measure and advise strategies to strengthen self-esteem, grow confidence and build resilience.

    Our company, Frontline Solutions Ltd., is happy to assist anyone looking for self-analysis tools that can be supported by a mentor’s analysis in this area of personal performance and change

    See http://www.mentoring.net.nz/index.cfm

    I’m happy to supply passwords to organisations who wish to trial this.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s