2. The Narrative Evaluation System (NES):

Evaluation of Academic Performance


UCSC opened in 1965 during a time of unprecedented university growth, student protest, and introspection. The founding faculty, dissatisfied with traditional forms of grading, opted for a system that was intended to provide a better understanding of what a student had achieved in a course, while downplaying the competitive aspects of learning. Faculty-authored narrative evaluations were adopted in lieu of letter grades. UCSC instructors would write a personalized narrative evaluation of each student's academic performance in all courses in which the student earned credit.

The proposal to implement narrative evaluations was approved by the systemwide Academic Senate in 1966 as a variance from the standard University of California grading system. A condition of the variance was the requirement that to receive the bachelor's degree a student must either write a senior thesis or pass a comprehensive examination.

Originally, narrative evaluations supplemented course notations of Pass or Fail. Later, boards of studies were given the option of electing that their upper-division courses be graded A-B-C-D-F for all majors in the discipline. In 1972 the D and F grades were replaced by the course notation, "No Record." A-B-C grades were available at the undergraduate student's option for most upper-division and a limited number of lower-division courses. Graduate student grading was limited to Pass (P), In Progress (IP), Incomplete (I), or Fail (F). All grades and course notations for both undergraduates and graduates were supplemented with narrative evaluations.

Instructors are currently provided with a twenty-page booklet of information designed to guide them in drafting narratives, but no further training is systematically made available. A well-written narrative typically

Evaluations are submitted to the Registrar's Office and added to the student database in standardized format with minimal, non-substantive proofreading. Completed evaluations are sent to students and their advisers. Submissions which are incomplete, ambiguous, or illegible may be returned to the author for correction or clarification, or to the faculty Committee on Narrative Evaluations which oversees the system and advises the Academic Senate.

Once made part of the permanent electronic record, the narratives are maintained indefinitely and sent, on request of the student, to potential employers, graduate and professional schools, and other agencies as part of the student's official transcript. Copies of the narratives are also made available to advisers, academic preceptors, and department offices, as well as to the faculty authors.

The narrative evaluation system is not unique to UCSC. A recent study indicates that at least seventeen colleges and universities in the United States and Canada use some form of narrative evaluation. Of those, Goddard College and Sarah Lawrence have used narratives for over fifty years, and UCSC, Hampshire College, and New College of the University of South Florida have used them for well over twenty years each. However, the narrative system at UCSC is much broader in scope, resulting in up to 30,000 narrative evaluations per term compared to fewer than 5,000 per term elsewhere.

Virtues and Liabilities of the Narrative System

Opinions differ about the true value of the narrative evaluation system. Research published by the Office of Institutional Research, as well as a recent survey conducted by the UCSC Alumni Association, indicates substantial and continuing support from UCSC graduates. A recent senior thesis project involving over 500 undergraduates and 150 alumni also indicated high levels of commitment to the narrative system. Of course, the respondents in such surveys might be viewed as self-selected by virtue of their decision to attend UCSC.

Faculty opinion seems to vary, although no scientific study of faculty opinion has been attempted. The following perceived virtues and liabilities are among the most frequently cited in discussions:

Perceived Virtues

Perceived Liabilities

CURRENT PRACTICE

Although narratives remain the primary form of academic evaluation at UCSC, changes have occurred over the years. Initially, instructors teaching classes with more than forty students were exempted, upon petition, from writing narratives. Later, the exemption disappeared as a matter of general practice and instructors submitted narratives in all courses taught for credit. A procedure was also introduced to allow students to appeal narrative content if the student felt "that an instructor gave the course notation or evaluation based on the student's race, politics, religion, age, sex, sexual orientation, national origin, personal attributes, or anything other than academic performance."

With the growth of the campus, the size of many classes has increased. Several introductory courses now have over 400 students enrolled. To compensate for the difficulty of writing narratives for large numbers of students, a computerized reporting system has been developed. It allows instructors to use menus and grids to prepare narratives electronically. Although the course instructor has final responsibility for the narratives, writing evaluations in larger classes has been possible only because of the involvement of teaching assistants who have often prepared the initial drafts of narratives for undergraduates. The role of the TAs in NES is not recognized in official descriptions of the system and has, at times, become a focus of controversy.

Administration

The preparation, collection, and storage of narrative data require extraordinary commitments of instructor and teaching assistant time and energy and of the institution's scarce financial resources. There is no limit on the length of narratives, although most are no more than two or three paragraphs. Instructors are asked to submit their narratives within fifteen working days after the end of each quarter. About sixty days into the next quarter, a "Report of Outstanding Evaluations" by course is circulated to deans, unit heads, and the Executive Vice Chancellor. At the same time, a cumulative summary listing faculty with 100 or more outstanding evaluations is forwarded to relevant unit heads. Faculty who habitually fail to submit narratives in a timely way are at risk of having their merit or promotion files delayed or turned back; non-Senate faculty who are habitual offenders are at risk of not being rehired. Table A.3 shows the number and percent of narratives received for the six most recent quarters available in September 1993.

Letter Grades

During the first three weeks of the quarter, undergraduates may opt for letter grades (A, B, C, or NP) in all upper-division and a limited number of lower-division courses. Instructor permission is required for letter grades in upper-division individual studies courses.

Graduate students receive only narratives and course notations of P or F. The proportion of undergraduates choosing the letter grade option is approximately ten percent of all course enrollments. When tallied in terms of enrollments in courses approved for letter grades, quarterly averages range from about 18 to 22 percent (see Table A.4).

Comprehensive Requirement

Unlike most other institutions (including other University of California campuses), UCSC undergraduates are required to demonstrate their competence by completing a senior thesis or comprehensive examination in their major field of study. Although the proportions vary widely by discipline, slightly over 34 percent of those graduating in June 1992 wrote theses; 57.5 percent took comprehensive examinations (including the GRE); and 8.4 percent met the requirement through some other means. The current budget crisis has stimulated review and discussion of the comprehensive requirement as boards of studies seek to handle increasing student loads and as faculty are asked to teach more classes.

ASSESSMENT OF THE NARRATIVE EVALUATION SYSTEM

Comparative Grade Distribution

The narrative system presumes that without traditional grades and the related phenomenon of "grade grubbing," students should be free to concentrate on learning. Undergraduates also know that if they do not pass a course, the failure will not be entered on their permanent record (although the record will indicate lack of progress). In granting UCSC a variance, the systemwide Academic Senate approved these efforts to reduce grade pressure, but also sought to ensure high academic standards by requiring that a student's performance be "clearly passing." The UCSC Academic Senate has directed that students should perform at a level equivalent to at least a C to pass a course and receive credit, a higher standard than that in effect elsewhere in the University of California.

Comparisons of the UCSC grading system with other UC campuses which use traditional grading are difficult to effect and can only be general in nature. A comparison of the distribution of grades at several UC campuses has been used to assess the academic performance of UCSC undergraduates. As shown in Table A.5, in all but one instance (UC Santa Barbara in spring 1992) the percentage of UCSC students who earn marks equivalent to C and above is equal to or slightly higher than similar figures for three comparative UC campuses. Conversely, in all but one instance, the percentage of UCSC students with marks below C is lower than at the other three campuses. Such figures are open to several interpretations. One is that UCSC undergraduates perform equally well or slightly better academically than do their counterparts on the three other UC campuses. Another is that the minimum standard of C that is prescribed for a grade of Pass at UCSC is not always strictly adhered to.

Longitudinal data suggest that the proportion of passing grades has been on the increase. Interestingly, many American universities which use traditional grading systems have reported a disturbing pattern of grade inflation over the past two decades. Although a number of interpretations are possible, the trend data presented in Figure A.1 suggest that the phenomenon of grade inflation may occur even in a system where traditional letter grades are the exception.

The proportion of passing evaluations has climbed over the past six years from 89.6 percent overall to nearly 94 percent in 1991-92. All interpretations of such statistics founder, however, on the fact that evaluations, whether numerical or narrative, are a function both of variation in student performance and of shifting criteria of assessment; we have no direct evidence that permits us to estimate the relative contributions of these two factors.

Graduate and Professional School Acceptance

Admissions personnel at other universities were initially discomforted by our narratives, and several UCSC officers devoted much of their time to explaining the narrative philosophy and helping others understand and utilize evaluations when making admissions decisions. This educational process has apparently met with considerable success, since recent studies indicate that UCSC graduates are accepted by graduate and professional schools at rates equivalent to those of graduates from institutions with more traditional grades. For example, a 1991 report issued by the UCSC Office of Planning and Budget indicated that 56 percent of applicants from UCSC were admitted to medical schools for the fall of 1990. This acceptance rate was 4 percent higher than the national average, though not as high as the rates at several other University of California campuses which have pre-professional programs and professional schools. Another study prepared by Planning and Budget reported that of a sample of 147 students who applied for advanced degree programs, 92 percent (135) were accepted at one or more schools.

CONCLUDING REMARKS

The narrative concept is a continuing focus of heated, if sporadic, controversy among faculty, perhaps exacerbated by current budget difficulties and a shift in the faculty reward structure over the years. During the spring quarter of 1993, the UCSC Academic Senate conducted a debate over the merits of NES. On June 9, 1993, a discussion on the floor of the Academic Senate affirmed by a vote of 49 to 12 the Narrative Evaluation System in its existing form. While neither the division of faculty opinion nor the sometimes impassioned debates over proposed changes can be expected to disappear completely, no major changes in campus practice are anticipated for the immediate future.