Executive Summary of the Learning Outcomes Assessment Model

The focus of the Learning Outcomes Assessment Model is student learning in the classroom and across the Institution. Drawing from the Community College of Philadelphia’s Mission Statement, the Assessment Task Force derived five core competencies that frame the learning environment encompassing the overall goals of general education and program/discipline curricula, at CCP. These core competencies include:

• Effective Communication
• Quantitative and Scientific reasoning
• Information and Technological Literacy
• Critical Thinking
• Responsible Citizenship
  

As students grow through their learning experiences at CCP, the goal is that they will achieve a high level of proficiency in these competencies through their cumulative learning experiences provided across all courses. Criteria instrumental in the development of this plan are highlighted below.

An effective assessment plan must:

• be aligned with the classroom instruction and learning goals.
• provide constructive feedback regarding learning outcomes to instructors, program and department heads, curriculum coordinators, Deans, and students.
• help students understand the elements of excellent work so that they may begin to develop the skills of self evaluation.
• incorporate assessment of the College’s general education requirements.
• provide standardized measures that accommodate all levels of learning.
• be fair and unbiased.
• satisfy the Middle States Commission on Higher Education’s charge that an institution establish a coherent set of learning goals, that these goals stem from the institutional mission, and that goals at the subordinate levels contribute to the attainment of goals at the higher levels.
• be simple and easy to administer.

Leadership for this ambitious model will be provided by the Director of Academic Assessment. Committees representing the five core competencies will be responsible for articulating appropriate learning outcomes for each competency and developing assessment rubrics that will serve as guides for targeting learning experiences and measuring outcomes.

Ideally, classroom assessments will be varied and frequent, direct and indirect, graded and ungraded with the goal of moving towards electronic portfolios and capstone courses for all students. Institutionally, the core competencies will provide an assessment focus on a rotating cycle beginning with Information and Technological Literacy in year one. Students’ proficiency scores, as measured by the related competency rubrics, will be coordinated with the midsemester grade submission process. This new data can be used to inform students and instructors about learning that has occurred in the classroom, inform departments regarding achievement of program learning outcomes, and provide a base from which to draw a sample to document learning outcomes across the institution.

Assessment is a dynamic and evolving process and this plan should not be considered a static model but rather an optimal beginning. The goal is to create an environment of continual growth and improvement in which students deepen their knowledge base, expand their abilities to think, problem-solve, and evaluate information, as well as foster their capacities for decision-making, communication, and responsible citizenship.

Read the whole plan here: http://www.ccp.edu/vpacaff/Learning.Outcomes.Assessment.Model.pdf

Building on Our Strengths: Classroom Assessments that Work

A Brief Theoretical Overview
Professional Development Week

January 8, 2008

Participants were invited to bring examples of classroom assessment techniques (CATs), defined by Cross and Angelo as anonymous, ungraded student feedback that answers two key questions: What are our students learning, and how do we know?

The session began with a brief theoretical overview of outcomes assessment to build on previous Think Tank discussions from both the January ’08 and September ’07 PD week events. This theoretical presentation resulted in a lively discussion about the nature and types of assessment, and its role in the classroom and in faculty development. Much of the discussion revolved around the differences between assessment and graded evaluations. Assessment is typically more formative and designed for both instructor and student feedback regarding the success of teaching strategies, outcomes articulation and learning, and student engagement, interaction and metacognition. Graded evaluations can be used as summative assessment at the end of a chapter, unit, or semester to also provide feedback on student learning. The major difference, though, is that summative assessments can be used to redirect the scope and sequence of course content, teaching methods and classroom activities. Their strength, in this regard, is that they can significantly change the results of graded evaluations by alerting instructors and students to material or content that has not been mastered before undertaking a graded evaluation.

Classroom assessment is meant to be a cyclical process. Based on institutional core competencies or departmental protocols, which in a best case scenario would be related, faculty define departmental outcomes for each course. The outcomes are articulated in course documents such as syllabi and assignments so that students understand what they are expected to know or do with course content, reading assignments and classroom activities. Throughout each lesson, CATs solicit quick, anonymous student feedback as to key concepts, unclear points, possible applications and so on illustrating for both the student and the instructor whether the outcomes are being met. This information allows instructors to create meaningful transitions between lectures or class meetings to revisit content that students have not yet mastered as well as to redirect lectures and activities when students have achieved mastery earlier than expected. The assessments, therefore, provide feedback that can be used to improve the day to day classroom experience and ultimately the structure of the course. Summative capstone experiences at the unit or semester’s end reinforce this information and provide benchmarks for the mastery levels of course outcomes. In this way, we can use assessment techniques to tell us whether and how well our students are learning.

Group #2 Roundtable: Assessment and Discounted Dreams

What did the film Discounted Dreams say about assessment?

50% of students fail to graduate from Community Colleges (Institutional assessment)

45% pass rate on remedial math (Departmental assessment–course specific)

2/3 of the Community College student population who wishes to transfer does not

What do we do at CCP already with assessment?

Institutional Research is done (i.e., but data appears to be disconnected from course, departments’ and instructors’ daily activities)

Departments and/or Programs build their own efforts of assessment for better student outcomes (i.e., at the student level: entering and exiting student interviews provide useful feedback for course/program changes; at the instructor level: evaluation of Instructor performance–a rather sensitive topic-is a form of assessment when done well–each dept./program uses different approaches for peer evaluation and sensitivity comes into place in terms of reliability of the instrument used as well as biases and differentiation in evaluation when more than one instructor is asked to evaluate classes)

Sharing information on teaching practices among colleagues (i.e., use of assessment techniques in classroom–questions at the end of a class to see if students have learned new material; comprehension checks throughout presentation of new material; class-transitions, especially in the beginning of the class to activate schemata; games at the end of class where students [in groups] test each other on new material, etc.)

Curriculum Facilitation work (doesn’t really reach all Faculty)

What have we learned from these efforts?

Overall, CCP is doing a lot with assessment as stated in #2; however, those efforts appear to be disconnected.

What more do we need to do?

to serve best the students, CCP needs to find out reasons for which students take courses; orient students –in terms of time management and organizational skills as well as student learning styles;

provide students with ease of getting information as to where they need to go for different types of information

(more) assessment needs to be done on a course/class basis
need to work more closely with high schools to target students left behind in material, especially with core courses (English/Math)

How can we accomplish the "more" that we need to do?

Develop a well-thought orientation session (in-session and/or on-line) for students coming into the school where time management skills and student learning styles are addressed.

Provide some form of certification that ensures that students have taken this orientation session prior to taking courses at CCP.

Implement (again?) the old system of “asterisk” next to students’ names on class lists to inform instructors about student activity regarding how often they have attempted to take the course. That will give the Instructor the opportunity to open a dialog with students who may have problems with the respective courses.

More Instructor training is needed on teaching vs. learning styles especially for targeted courses (Math/English)

Group #1 Roundtable: Assessment and Discounted Dreams

Professional Development Day - October 8, 2007
Facilitator: Dawn Sinnott
Recorder: Greg Bovasso

The group, which was composed mostly of art department faculty was not familiar with the assessment plan, and so the facilitator described the plan. The group seemed to agree that assessment of student learning was critical to academic success at various organizational levels (e.g., courses, curricula, etc.).

The group discussed the various methods of teaching and assessment depicted in the film shown prior to the session. The role of assessment in designating remediation for students was particularly notable, and the importance of assessment in serving a ‘gatekeeping” function was noted as having benefits, as well as costs, such as increased attrition. The group discussed the importance of creating a sense of social support in academic programs to reduce attrition. Active learning approaches, such as those depicted in the film and used by discussants in their classes were also seen as critical to student success. In addition, the use of art critiques was discussed as a method of teaching that employs assessment as part of the learning process in the classroom. Such techniques prevent the kind of non-participation and attrition by students shown in the film.

The group discussed assessment as a collaborative process between students and faculty, as well as among faculty, such as when all art department faculty evaluate the portfolio of a single student majoring in art. The group noted that different performance criteria were employed in these assessments, though the goals and process of assessment were uniform. The content of assessment was determined by the skills needed for successful transfer, as defined by the transfer institution. Classroom projects that build these competencies through continuous formative assessments (e.g., ‘critiques’), and portfolio evaluations assess these competencies. Students are given continuous feedback in classes and in mid semester reviews to assure that they meet the program objectives evaluated in portfolio reviews. Students can also compare their work with that of other students in this process.

Some of the competencies required to enter classes include skills learned in English classes given the importance of writing in art history classes and verbal skills in studio class critiques. Faculty described accommodations they make in classroom practices to remediate deficient skills in students. The faculty collaborate with the Architecture department in coordinating competencies and their assessments.

The group agreed that the CCP assessment system resulted in students with a wide variety of basic skills being enrolled in their classes, and that a wide variety of learning outcomes resulted from this assessment process. Much remediation takes place in college-level courses.

The most notable kind of formative assessment at CCP was the designation of students at remedial levels based on testing of their reading, writing and match skills. The completion of the first academic year was also viewed as a kind of formative assessment. Course-level evaluations also were viewed as providing clear feedback to students based on explicit objectives. However, assessment accommodates different student goals by applying a uniform assessment process to all students, but different standards based on students’ differing goals.

The facilitator described the assessment model proposed by the assessment task force as one requiring a feedback loop to improve teaching processes based on assessment. She also described the five competencies at the course level which would be assessed across the curriculum in the model proposed by the assessment task force. This approach was differentiated from the ‘dimensions’ model proposed for the General Education curriculum, and tied to measurable competencies rather than general values. The competency of ‘information literacy’ was clarified, and seen as needing expansion beyond information technology. All courses were seen as developing the five competencies to different degrees but more communication across departments on these competencies was seen as being needed, particularly regarding the implementation of their assessment. Departments may differ widely on their definition of each competency and its assessment to the degree that these competencies would not be truly universal. All agreed that such competencies should not be dictated to a department because this would obstruct the teaching of competencies that a department regards as most valuable, and lead to “teaching to the test” to satisfy organizational requirements at the expense of education. All agreed that such assessments of common competencies cannot be used to evaluate departments or curricula, only individual students. Some believed that exit criteria for evaluating student learning outcomes differ by student based on individual academic goals (e.g., major vs. non-majors), and cannot be generalized across curricula, or even within curricula. A department may have common goals and assessments, but apply different standards of evaluation depending on students’ various academic goals.

The advising process was seen as critical to ensuring that students were directed to meet their personal academic goals rather than institutional goals (e.g., degree requirements) which may not match their goals. The example of the student depicted in the film who repeatedly failed math was discussed as an example of a failed advising process. The importance of advising students to meet true educational goals (e.g., understanding math) rather than just credentialing goals (passing remedial math) was discussed. The system of portfolio evaluation was also seen as a method of providing students with feedback on how well they meet educational goals.

Assessment…. Misconceptions, Fears, and Realities

August 29, 2007, Nicole Duncan-Kinard and Kathy Harter led a discussion on assessment. What follows are their notes recording that discussion.

Misconceptions

• Assessment is equivalent to grading
• Formative and summative assessments are the same
• Assessment of outcomes infringes on faculty rights

Fears

• Assessment is dictated from beyond vs. faculty ownership of the course and the assessment;
• Assessment is viewed as teacher evaluation instead of a means to help the faculty member to progress;
• Assessment of outcomes will lead to a loss of individuality, faculty freedom of choice and responsibility;
• Assessment outcomes will effect tenure decisions.... Class will be perceived as too easy or too hard;
• Admitting that you “don’t know” something can be threatening to self-image/ego of a faculty member;
• Time for assessment of Student Learning Outcome's (SLO’s) is more time-consuming.....grading an essay-type exam is more extensive than T/F or multiple choice.
  

Realities

• Learning is part of a conversation with our students;
• Instructor’s attitude needs to be that the students understand the info vs. I delivered the info.... a SLO model vs. coverage model;
• The need to ensure consistency across sections regarding “protocols/course goals” [use a Teaching Goals Inventory] does not dictate an instructor’s style;
• Learning/classroom culture can be/should be collaborative vs. competitive;
• Tests can be used a punitive measure (low grade) or as a teaching moment...see what students don’t understand and address it;
• SLO’s can be sequential outcomes...as 101 moves into 102;
• For learning, students need to do the work...be engaged.....they summarize/evaluate the class (such as: muddiest area, one minute paper, what did you learn, what do you wish you learned, etc.)....and their input highlights the problematic areas/concepts for the instructor to address the next class;
• There can be student resistance to active engagement. (Afraid to speak out? Must do extra work such as re-writing a paper/lab report for grammatical errors, etc.) How do we overcome this? (Extra credit? Better grade? Designing a classroom culture that is welcoming; one of sharing)
  

Review of Recommendations and Prioritization of the Core Competencies

On August 27, 2007 , the May 2007 report of the AAMP Task Force on Assessment was reviewed. What follows are the notes from the discussion that ensued.

The decision by Dr. Curtis to hire an Assessment Coordinator as recommended in the Task Force report was applauded. Discussion ensued regarding the role of the task force in the hiring process of this person; a strong role was enthusiastically endorsed (committee presence and drafting of the interview questions and writing the job description). This person was visualized as a “go to” person to assist the faculty is a great addition but a collaborative person vs. a dictator is needed. Likewise, there was a strong feeling that the hire should be an internal individual (with release or extended time) to promote the buy-in of faculty....along with IR....and supplemented by a faculty committee.

Prioritization of the core competencies identified in the report was not recommended. Literacy (reading and writing) and numeracy are obviously critical but all the competencies were seen as valid and vital. Rather, it was suggested that after the assessment audit is completed, a matrix of all the competencies vs. courses (and competencies vs. programs) be developed following which all units would indicate where (and how?) the competencies were currently being addressed. Gaps in the matrix would indicate where changes are needed.

We need to acknowledge that the student of today is different from the student of 40 years ago. Students see us more as a supermarket where they get to pick and choose at their whim rather than as an institution of learning. If the reality is that 72% of our entering students need remediation, can we not design courses just for them that they must take along with their 3 developmental courses (vs. the “ludicrous” waiver list).... for instance a 3 credit orientation?

Orienting the culture of the College to assessment is needed but the College’s culture is so nebulous that it is difficult to know and steer. The College is redefining itself; going through a mid-life crisis. Therefore, the administration must establish a long term vision as regards assessment (leadership and commitment)....perhaps Achieving the Dream or the like...and link decisions, energy and dollars to this goal for all constituencies. As regards College-wide assessment we need to stress that assessment asks “What are our students learning?” and “How do we know?” We also need to stress that assessment is

1. SoTL...experimentation and evaluation...continuous inquiry;
2. Is a means to ensure uniformity across multiple course sections by being an objective measurement;
3. Moves us toward/demonstrates that we a Learning College;
4. Has best practices which are available to us...find them, use them! (such as a single, assigned advisor/counselor) ;
5. Is establishing a very simple feedback loop.....define a learning outcome, experiment with the means to accomplish it, assess the outcome and adapt the teaching based on the assessment;
6. Is not limited to course grades at the end of the semester. It can be done at the end of a lesson.....and the next class altered based on that assessment;
7. Is acknowledging that enhancing/changing what the instructor does in the classroom does not guarantee learning but that the way a course is structured most definitely can assist or hurt learning...ex. types of test, group projects, etc;
8. Is a shared responsibility....instructor and student and across the institution. The student is challenged to play a role/to be a learner....active engagement vs. a passive receiver;
9. Is not faculty evaluation. Faculty should not feel threatened by experimenting in the classroom; jobs are secure. Change is good...but for some faculty change/assessment outcomes could be met with fear...an ego issue?