Back on May 8, 2010 I
blogged about Rubrics and figuring out
where you are. The fourth
Merriam-Webster dictionary
definition for a rubric is:
“a guide listing specific criteria for grading or scoring
academic papers, projects or tests”
An evaluation rubric includes several questions about both
content and delivery, which are scored on some sort of a scale. How many levels
should be on the scale? At least two, but would either three, four, or five be
better, as is shown above. What names should be given to those levels?
A week ago on LinkedIn at The Official Toastmasters
International Members Group ChenKeat Fan posted on Evaluation sheet for
evaluator in Pathways: what’s your opinion of its usefulness? So far there have
been over twenty comments (including one from me). He complained that having to
quantify on a 1 to 5 scale is cumbersome as compared to the previous scale with
three levels. (But Toastmasters actually used a five-level scale before in
their course on The Art of Effective Evaluation).
I got curious and looked up some history about speech
evaluations. Back in 1981 there was a 47-page booklet by Douglas G. Bock and E.
Hope-Bock on
Evaluating Classroom Speaking. You can
download it as Document ED
214 213 on the
Education Resources Information Center (ERIC) web site. That
booklet ends with a section showing 13 sample evaluation forms. The preceding
section on constructing an evaluation instrument discusses the topic of
Controlling Rating Errors on page 22. It says that:
“The error of central tendency can be controlled by the
number of scale values used on the continuum. For example, if only three
numbers are use, most raters are going to use the middle category. It has been
found that a five-step scale usually results in three steps being used. A
seven-step scale uses about four. A ten-step scale usually produces five. One
way to get raters to use more of the scale is to have more steps.”
A second question is which way the scale should run. As
shown above, graphs displayed with Cartesian coordinates typically have an
x-axis using the right-hand rule, but the left-hand rule also could be used. A
scale with the right-hand rule would have the worst category at the left, and
the best at the right. Since we read English from left to right, there might be
a primacy effect where we would overuse that left category. Those using
languages read from right to left (Arabic, Hebrew, Persian, or Urdu) might have
a different bias than English speakers.
A third question is where the scale should start, as shown
above. Should there be a zero? Some might object that it should begin at 1,
since we all are heroes - not zeroes. One way to organize a speech evaluation
form with five levels is to alternately list a question and the scale (so the
score for it can be circled). Other pages of the form can show details of what
those five levels mean.
One well-known rubric is
The NCA Competent Speaker Speech
Evaluation Form (2nd edition, 2007) from the U. S. National Communication
Association, as shown above. It is described in a 49-page Acrobat .pdf file
you
can download free from their web site. This rubric lists eight competencies – four
for Content and four for Delivery (in four columns) rated on a right-hand rule
axis with three levels: Unsatisfactory, Satisfactory, or Excellent. I have
numbered them 0, 1, 2 – although the form omits that detail but nevertheless
has a bottom row for General Comments and a Summative Score. The form also has
a three-page explanation for those competencies. There is another NCA Competent
Speaker Holistic Speech Evaluation Form that combines each set of four into just
two categories labeled Preparation and Content, and Presentation and Delivery.
In 2012
Communication Education magazine had
an article by Lisa
M. Schreiber, Gregory D. Paul, and Lisa R. Shibley titled
The Development and
Test of the Public Speaking Competence Rubric (PSCR) which you can download. As
shown above, this rubric has eleven categories rated on a left-hand rule axis numbered
from 4 to 0 and titled 4 = Advanced, 3 = Proficient, 2 =Basic, 1 = Minimal, or
0 = Deficient. You also can download a
single-page Table with a detailed explanation
for each item. I
blogged about the PSCR in a July 9, 2012 post titled
A new
scale (rubric) for evaluating speeches.
In their
Success Communication series,
Toastmasters
International has a two-hour course titled
The Art of Effective Evaluation
(
Item 251). It has an
Individual Speech Evaluation Form (
Item 251D), which you
can find at the end of
a handout for it from the Park City club. As shown
above, the form has 12 explicit categories (and room for two optional ones). The
right-hand rule axis has five levels which from left to right are labeled 1 = Needs
Considerable Improvement, 2 = Needs Some Improvement, 3 = Acceptable, 4 = Very
Good, 5 = Excellent. The form is organized into three columns. Each row has a
Category, followed by a Rating (1 to 5) and Recommendations for Improvement.
In the new Pathways educational program from
Toastmasters
International, speeches are evaluated using a three-page form. The first page
has a Purpose Statement, Notes for the Evaluator, and General Comments (with
three categories – You excelled at, You may want to work on, and To challenge
yourself). As shown above for the
Icebreaker Speech, the second page in the
form has seven categories on a left-hand rule axis with five levels labeled from
left to right as 5 = Exemplary, 4 = Excels, 3 = Accomplished, 2 = Emerging, and
1 = Developing. Although those labels are
explained in detail on the third page of that
form, I think they are way more obscure than those used in the PSCR.
As is shown above, perhaps a more honest revised set of
labels would be 5 = Outstanding, 4 = Excellent, 3 = Good, 2 = Fair, and 1 = Poor
(or Poop). Other evaluation guides from Pathways are discussed on
a web page at
UmErYouKnow with a link to
a list of them.
-->
