Strategies for Direct and Indirect Assessment of Student Learning
Mary J. Allen, firstname.lastname@example.org SACS-COC Summer Institute July 28, 2008
See http://assessment.aas.duke.edu/documents/DirectandIndirectAssessmentMethods.pdf for the entire 12 page document
Two Basic Ways to Assess Student Learning:
1. Direct – The assessment is based on an analysis of student behaviors or products in which they demonstrate how well they have mastered learning outcomes.
2. Indirect – The assessment is based on an analysis of reported perceptions about student mastery of learning outcomes.
Properties of Good Assessment Techniques
• Valid—directly reflects the learning outcome being assessed
• Reliable—especially inter-rater reliability when subjective judgments are made
• Actionable—results help faculty identify what students are learning well and what requires more attention
• Efficient and cost-effective in time and money
• Engaging to students and other respondents—so they’ll demonstrate the extent of their learning
• Interesting to faculty and other stakeholders—they care about results and are willing to act on them
• Triangulation—multiple lines of evidence point to the same conclusion
Strategies for Direct Assessment of Student Learning
1. Published Tests
2. Locally-Developed Tests
3. Embedded Assignments and Course Activities
- · Community-service learning and other fieldwork activities
- · Culminating projects, such as papers in capstone courses
- · Exams or parts of exams
- · Group projects
- · Homework assignments
- · In-class presentations
- · Student recitals and exhibitions
- · Comprehensive exams, theses, dissertations, and defense interviews.
- · Showcase vs. Developmental Portfolios: best work vs. evidence of growth
- · Workload and storage demands for large programs can be overwhelming!
5. Collective Portfolios
Strategies for Indirect Assessment of Student Learning
- 1. Surveys
- 2. Interviews
- 3. Focus Groups
Qualitative vs. Quantitative Measures
(from Cal State Stanislaus Assessment Workbook)
Not all assessment data must be quantitative; in fact, a mixture of quantitative and qualitative measures may offer the most effective means of measuring student learning outcomes and program goals. Of course, the assessment measure chosen should be fashioned to the unique needs of the program.
QUALITATIVE ASSESSMENT – “Assessment findings that are verbal descriptions of what was discovered, rather than numerical findings.” - Allen (2004) p.171
Some Examples Include:
Open-ended questions on surveys and interviews
QUANTITATIVE ASSESSMENT – “Assessment findings are summarized with a number that indicates the extent of learning.” – Allen (2004) p.171
Some Examples Include:
Written and Oral Exams
From http://www.jmu.edu/assessment/pass/assmntresources/what_is_assessment.htm with a little bit of editing.
|Why can't we use course grades for assessment?|
|Course grades are assigned to individual students to indicate the extent to which a student has met the instructor's expectations for a given set of course requirements. Assessment results are intended to reflect the extent to which all students achieve the outcomes of a program. Clearly, grades and assessment differ in that one deals with individuals and courses and the other deals with groups and programs. Why does this difference matter?
How grades are assigned varies across courses and course sections. This means we can't compare grades across courses to make inferences about the program as a whole.
Often, criteria unrelated to program outcomes are considered when assigning grades such as attendance, participation, or getting work turned in on time. Some graded assignments may help students prepare for an outcome but not be part of the outcome. This means that grades are not a pure measure of student learning.
Grades are assigned based on one instructor's judgment only. This means that there is a lot of subjectivity in grades. Assessment involves multiple-raters and checks on reliability and validity of scores.
Grades don't tell us what information the student knows and doesn't know. Assessments are intended to provide additional and more descriptive information so that we know more about what each assessment score means about a student's ability-level.