STANDARD III: System of Assessment

 Indicators

Narrative

The Education Department is made up of both the undergraduate and the graduate programs.  There are clearly documented systems of assessment for entrance to and exit from programs at both levels. 

Undergraduate Program

The undergraduate program offers the following majors: elementary and  secondary education licensure, as well as an elementary education major.  The elementary major does lead to recommendation for Vermont teaching license.   Students interested in secondary education may pursue licensure in one of the following content areas: mathematics, biology, chemistry, English, French, Spanish, history, theatre arts (drama), and art (K-12).

All undergraduate teacher licensure majors receive a printed handbook, Saint Michael’s College: Guide to Teacher Education Programs and Licensure Elementary or Secondary Education during their first education course (ED 231: Schools and Society).  These handbooks specify all of the State and College requirements for teacher licensure, including the performance criteria for acceptance to the Education Department and requirements for recommendation for Vermont teaching licensure.  The Guide to Teacher Education Programs (2003) contains the following information:

As stated in the preface, the Guide represents an authoritative source of information about the policies and common practices associated with undergraduate elementary and secondary education programs.  Each year the Guide is updated to incorporate the changes that occur at the State Regulatory level, as well as at the Department or College levels.  For example, as stated on page 26 of the current Guide, Secondary students must attain a 2.7 G.P.A. in their major subject area (class of ’07 must attain a 3.0 G.P.A.).   This is an example of how the Education Department clearly communicates relevant information to students in a timely manner. 

Currently, entrance and exit requirements for undergraduates are the following:

Entrance Criteria

1.         To enter the teacher education program, undergraduate students must have a minimum overall grade point average of 2.5 at the end of the sophomore year. 

2.         Prior to enrolling in many of the 300 level teaching methods courses, students must have a 2.7 G.P.A.

3.         Prior to student-teaching students must have a 3.0 in their Liberal Arts major and a 3.0 in their Education major.

4.         Prior to student teaching, students must pass both Praxis I and Praxis II.

Exit Requirements

5.         Students must complete all program requirements for their Liberal Arts major. 

6.         Students must fulfill all College requirements for the Liberal Studies Requirements (LSRs).

7.         Students must pass the 16 week student teaching experience with a grade of a “B” or better.

8.         Students must “meet standard” on the State Licensure Portfolio rubric.

Faculty and students track each student’s progress as part of the advising process.  Every student and advisor uses the Elementary or Secondary Education tracking form to provide explicit reminders with regard to entrance and exit criteria.  Saint Michael’s uses a computerized course registration system at the undergraduate level that requires that the faculty advisor “flags” each student prior to the student’s ability to register.  The significance of this within the Education Department is that it ensures each student has the opportunity to receive the face-to-face advising session where the discussion about individual student progress in relationship to the entrance and exit criteria can occur. 

In the past, we have not kept track of the number of students who leave the Education program due to academic standings or changes in interests.

Graduate Program

The graduate program is a licensure program designed for people who already have a baccalaureate degree in a liberal arts content degree that is considered acceptable by the Vermont Department of Educator Licensing.  The graduate education program offers both M.Ed. and C.A.G.S. programs.  As identified in the graduate course catalogue, the concentration areas are the following:  

As documented in the Graduate Catalogue, requirements for admission into the M.Ed. completion are:

Entrance Criteria:

1.         Writing Assessment.  Graduate students must demonstrate writing proficiency on the Graduate Writing Assessment prior to acceptance for candidacy and student teaching.

2.         Candidacy.  After a graduate student has taken at least six credits, but no more than twelve credits she/he completes the candidacy form with a Saint Michael’s advisor.  This form serves as a statement of concentration within the M.Ed. program listing: courses taken, transfer credits accepted, statement of goals, and proposed courses to be taken.

3.         Grade Point Average.  A cumulative G.P.A. of 2.8, and a G.P.A. of 2.8 within the major. Effective 2004, the G.P.A. requirement will be 3.0

4.         Interview.  Successful completion of an entrance interview with members of the graduate faculty.

5.         Previous Experience. Evidence of successful work with children or youth.

The following table highlights the requirements for individual licensure areas.

Exit Criteria
The exit criteria that are common across all areas are the Praxis exams and the licensure portfolio.

6.         Praxis Exams.  Initial licensure students must pass Praxis I and Praxis II (where applicable as per Vermont Department of Education, Office of Licensing) prior to student teaching.

7.         Student Teaching.  Students must pass the 16 week student teaching experience with a grade of a “B” or better.

8.         Licensure Portfolio.  At the conclusion of student teaching, all graduate students seeking initial teaching licensure must submit a licensure portfolio.  This portfolio must “meet standard” on the Vermont Department of Education portfolio rubric in order to receive a recommendation for licensure.  Examples of completed portfolios and the accompanying assessment rubric can be found in the Exhibit Room.

9.         Additional requirements relevant to the individual area of licensure may vary.

Historically, we have not tracked the number of graduate students who are not accepted into candidacy, although several examples of "rejected" applications are available in the Exhibit Room. 

Narrative

At both the undergraduate and graduate levels, there are three common resources used in designing assessments: the Education Department theme and dispositions, the Vermont 5 Standards and 16 Principles, and the subject matter competencies per licensure/endorsement area.  Curriculum maps indicate which of the Standards and Principles are addressed in each syllabi at the both undergraduate and graduate levels.  Both full-time and adjunct faculty receive information that indicates which of the Vermont Standards and 16 Principles  should be addressed in the course they are teaching.  All of the syllabi from the full-time faculty explicitly cross reference the Vermont Standards and 16 Principles.  We are in the process of supporting adjunct faculty to state explicitly on their syllabi the Standards and Principles that are highlighted in their courses (refer to course syllabi in the Exhibit Room).    

Learning opportunities and assignments are designed with the purpose of supporting students in becoming excellent novice teachers who embody the knowledge, skills and dispositions required.  Assessments occur within individual courses, across courses, during field-based placements, within the standardized requirements of Praxis I and Praxis II, and ultimately during student-teaching.  In addition, each student maintains a developmental portfolio. 

During each phase of development in the teacher education program, there are opportunities for students to receive honest and critical feedback.  Examples of various formative and summative assessments are found in the Exhibit Room where each course has a course folio.  Each course folio includes the course syllabi and examples of student work with the accompanying faculty feedback.

Throughout the candidate’s progression through the Teacher Education program at Saint Michael’s College s/he is kept apprised of his/her performance using both formative and summative assessments.  Following is an outline of the progression toward licensure at the undergraduate level, along with the identification of actions that the student can take if s/he is unsuccessful at any stage.

 Undergraduate Progression Toward Teacher Licensure

 Graduate Level

While the progression toward candidacy is slightly different at the Graduate level, ultimately the evaluation is the same.  

 Graduate Progression Toward Teacher Licensure

It is important to note that every student in the undergraduate and graduate programs has a faculty advisor who is responsible for guiding their course of study, as well as their professional development.  Guidelines, structures and forms (Link to: ESL, elementary, secondary, middle level, theatre arts, visual arts elementary, visual arts secondary,  Frequently Asked Questions and Licensure Candidacy Form Guideline.) are available to support the student in planning out the necessary course of study, as well as prompting them to pay attention to state department licensing regulations. 

 Graduate Special Education

The Special Education Concentration

M.Ed. Degree

Certificate of Advanced Graduate Study

M.Ed. with additional endorsements in Resource

Room Teacher or Consulting Teacher

There are two additional assessments required for graduate students pursuing a special education license.   Both assessments occur as part of the year long practicum experience (GED 661 and GED 662).  The first assessment includes the the additional components required for the special education licensure portfolio.  The second assessment is the special education practicum evaluation form.  As the reader will note, the evaluation form is the list of Vermont State special education knowledge and performance competencies.  This form is completed in the middle of the semester to serve as part of the formative assessment, and again at the end of the semester as a component of the summative assessment.  The evaluation is completed by the graduate student, cooperating teacher and the Saint Michael’s College supervisor.

Student Responsibilities and Artifact Development

Undergraduate and graduate students initiate the development of a professional portfolio during their first education course (i.e., Schools and Society for undergraduates and Decision to Teach for graduates).  The portfolio process is developmental in nature, and, therefore, continues through student-teaching.  During the student-teaching semester, a Licensure Portfolio is created.  The licensure portfolio is based on SMC requirements and Vermont State Department of Education requirements. 

Undergraduate Portfolio Development

Following are the descriptions of each that are given to students and contained within the Undergraduate Education Guide.

 The following table specifies artifacts that are suggested for the licensure portfolio entries are listed for students at the undergraduate level.

In 2002 an additional one credit undergraduate course (ED 415: Portfolio Design) was offered for the first time.  This course specifically targets licensure portfolio preparation for the undergraduate elementary education students.  The primary purpose of this course is to guide students through the final stages of their licensure portfolio.   The undergraduate secondary education majors receive the parallel support ED 417: Reflective Practices.  This course has been required for the past 5 years. 

Portfolio Development: Graduate Students

Graduate students receive a hard copy of the Portfolio Process for Teacher Licensure in their initial graduate course, GED 516:  Decision to Teach.  All of the critical information related to portfolio requirements are stated explicitly in the handbook. 

During GED 689: Student-Teaching Seminar all initial licensure candidates (i.e., elementary education, middle school endorsement, secondary education, TESOL) receive focused direction and support to create their licensure portfolios.  The following tables summarize the required artifacts for their portfolios and they cross reference the artifacts with the graduate course(s) where they would have created the item for the first time.  This approach supports the students in recognizing that they did learn the information in a deliberate and systematic manner across time.  The tables are also included in the handbook given to graduate students seeking initial endorsements.

      Integration of Portfolio Artifacts with Six Entries Required by DOE

In addition, students seeking initial licensure in Special Education must include additional artifacts and make explicit connects with the Five Standards for Vermont Educators.

Guide for SPECIAL EDUCATION Initial License
Integration of Portfolio Artifacts with Six Entries Required by DOE 

Reading Specialist and Principal Candidates  do not have the same portfolio requirements as initial licensure candidates.  These professionals already have an initial teacher license and, therefore, have met the portfolio criteria.  However, each program does require a more focused version of the portfolio where the candidate establishes a connection between elements of her work and the Standards.  Examples are available in the Exhibit Room. 

Narrative:

A variety of formative and summative assessments are used throughout the Saint Michael’s teacher preparation program at both the graduate and undergraduate levels.  Students have a variety of ways to demonstrate mastery across the curriculum.  The Assessment Matrix highlights several categories of assessments.  In addition, syllabi for every course and work samples are available in the Exhibit Room.

In addition, students must pass Praxis I and II prior to student teaching, pass student teaching with a “B” or better and reach “meet standard” on the Vermont State Licensing rubric.

Narrative

Students have ongoing opportunities to examine information about the assessment systems throughout their course of study at Saint Michael’s College.  The information is easily accessible (i.e., located in the undergraduate and graduate offices) as well as online and given to students during advising each session.  The major documents at the undergraduate level include:

Student handbook for Elementary Education

Student handbook for Secondary Education

Guide to Teacher Education Programs and Licensure

Portfolio Rubric

Portfolio Course List

Portfolio "Report Card"

At the Graduate Level the primary documents include:

Graduate Catalogue

Portfolio Process for Teacher Licensure

In addition to the programmatic information, students receive written information about the Praxis 1 and 2 examinations that are required by Saint Michael’s prior to student-teaching, copies of the portfolio assessment rubric and the portfolio report card.  Finally, student teaching provides opportunities for ongoing, in-depth assessment.  All students receive the student evaluation form in advance.

There are three primary sources of evaluation used along the way for teacher education licensure candidates at both the undergraduate and graduate levels.  First the assessments that are embedded within each course include formal criterion referenced tests, as well as informal assessments  and the department  Assessment Matrix (e.g., rubrics, performance-based, interviews, self-assessment, and conferencing).  Refer to the individual course syllabi for specifics.  All of the assessments are linked to both the Vermont Standards and Principles and the individual endorsement competencies.  Professors are responsible to engage in an ongoing separate course renewal which includes examination of course evaluations and student feedback.  The portfolio process offers an indirect glance into each of the courses because while individual student artifacts are not re-graded, the artifact is examined by the reviewer. 

The second formal method of assessment are the developmental and licensure portfolios.  Refer to Indicator 3.03 for an in depth description of the process as well as to the Exhibit Room for examples of portfolios in various stages of completion.

Finally, in accordance with State Licensure Regulations, we use the standardized Praxis levels I and II.  Undergraduate and graduate students must pass both Praxis I and II prior to student-teaching.  Close examination of student performance on the Praxis exams has yielded interesting information about our students' profiles and about the SMC programs. 

Examination of the Praxis II data in Secondary Education History and English reveal that the percentage of students not meeting the Vermont cut scores on the first attempt is unacceptably high.  While our sample sizes are small, members of the Education Department are concerned and have already begun to examine the problem.  In the fall 2003, the following action steps were initiated:

1.         Request additional information from ETS to answer the following questions:

            a.         How many times did students take the Praxis?

            b.         Which students (graduate or undergraduate) have failed the Praxis?

2.         Cross reference the liberal arts courses with content for the exam.

3.         Survey graduates in those specific areas to obtain information about their experiences with the Praxis II exams.

4.         Speak to the Liberal Arts faculty in each department to obtain their input.

a.         Faculty from the mathematics, history and English Departments have agreed to take the Praxis exams to obtain first hand knowledge of the content.

b.         Faculty from the aforementioned departments will engage in discussions regarding their experiences with the Praxis II exams during the 2004-2005 academic year.

5.         Ultimately, we will answer the questions:

a.         Why have too many Saint Michael’s students not passed the Praxis II exams in history and English?

b.         Is there a need to alter the SMC liberal arts coursework?

c.         Do we need to teach to the test?

Ultimately, in order to be recommended for licensure at the undergraduate and graduate levels, Saint Michael’s students must: obtain a G.P.A. of 3.0 in both Education courses and content area courses,  obtain passing scores on the Praxis exams, earn a “meets standard” on the elements of the licensure portfolio, and pass student-teaching with a grade of a “B” or better.  This comprehensive system of assessment allows faculty with the ability to recommend or not recommend any particular student, with a high degree of certainty.

Student Reactions to the Portfolio Assessment

One way to gather information to inform the "quality of the candidate assessment system" is by asking for critical feedback from the students themselves.  Since 2002 two areas of assessment that have received significant attention have been the portfolio assessment process and the Praxis exams.  Following is the description of how we obtained feedback form our students and graduates, as well as how we incorporated that feedback into our system of assessment.

In the spring of 2003 the Education Department conducted a survey of graduates from the past 5 years and their employers (i.e., principals).  The purpose of the study was to determine their perceptions of the quality of their:

(a)       preparation

(b)       satisfaction with their experiences of the SMC Program, and

(c)        experiences developing their licensure portfolio. 

The data are descriptive in nature, using qualitative methods (i.e., thematic coding) for the narrative feedback and quantitative methods (i.e., SAS program) for the forced choice options.  While the raw data are available, specific trends emerged that have had an immediate impact on our courses.  Specifically,  

A.        48 of the 86 respondents indicated that the portfolio process was a rushed experience.

B.        46 of the 86 indicated that they collaborated with their peers.

C.        44 of the 86 indicated that they collaborated with faculty.

D.        Only 39 of 85 respondents indicated that the experience was reflective.

E.        19 of 80 respondents indicated they were involved in the assessment of their portfolios.

F.         39 of 81 indicated that the portfolio was used during their interview process.

As a result of this survey, we have generated the following actions for our department:

A.         Addition of ED 415: Portfolio Development course for undergraduate elementary education majors.

B.         Increased explicit emphasis on the connections between student work, instructional practices and portfolio development.

C.         Increased planning around the role of students in the assessment of their portfolios.

D.         Creation and initial use (piloting) of the new portfolio report card.

Given the aforementioned changes in the Saint Michael’s Education Department Portfolio Guidelines and what appears to be far fewer changes required via the State level, we anticipate continued improvement in the use of portfolio assessment as an integral and ongoing part of our teacher education program.  Samples of student developmental and licensure portfolios are available in the Exhibit Room.

Narrative

Functionally, the assessment of teacher licensure students by liberal arts faculty is primarily focused on their individual programmatic or departmental areas.  Given a 21 credit teaching load across campus, it is unusual for faculty to have any significant amount of time for cross department strategic and curricular planning.  Liberal arts faculty design curriculum expectations in alignment with their content and programmatic focus areas, while education does the same.  In the secondary education program, Liberal Arts faculty advisors are given a copy of their subject area competencies (as delineated by the DOE) and asked to write a letter to verify that their advisee has demonstrated content competence.  In addition, in the Spring of 2003, the Department Chairs for English, history, biology, biochemistry, chemistry, mathematics, Spanish, French, Latin, theatre and art were given the new state endorsement competencies for each of their areas with a request to specify if and in which courses they address each competency.  This process was particularly interesting as it opened doors to understanding between faculty in the different departments.  This will serve as the basis for continued dialogue among departments. 

As described under Indicator 3.06, a working relationship between the Education Department and the mathematics, history and English Departments has been forged to address the issues of pass/fail rates on those specific Praxis II tests.  Faculty members in those liberal arts areas took Praxis II exams and have provided written responses to our Department (refer to indicator 1.01).  We are looking forward to a cross-departmental discussion with faculty from all departments. The focus of the discussion will be on the preparation of novice teachers.  Faculty who have taken the Praxis exams will be members of the panel.  This will occur during the 2004-2005 academic year.

In addition, during the Spring 2004 semester, members of the Education Department met with each of the Liberal Arts departments to address a request made by several undergraduate secondary education majors.  The request: Could additional opportunities be created for secondary education majors to have another practica experience in their content areas?   Several Liberal Arts faculty suggested inviting students to co-teach with them.  A pilot experience was initiated in the History Department in the Spring 2004.  It is likely that this type of elective will continue and expand.    

Narrative

The nondiscriminatory statement in the undergraduate course catalogue reads:

Saint Michael’s College is committed to affirmative action and equal opportunity.  It does not discriminate against students, employees, or applicants for admission or employment, on the basis of race, color, gender, age, national origin, religion, disability, sexual orientation or physical characteristics.  Members of the College are encouraged to communicate in a gender-neutral manner.

The nondiscriminatory statement in the graduate course catalogue reads:

The Graduate Programs seek students who possess qualities of intellect and character that will enable them to benefit from and contribute to a rigorous program of advanced studies.  Each applicant is judged on individual merit without regard to race, color, creed, sex, age, religion or sexual orientations, and national or ethnic origin,  An otherwise qualified individual will not be excluded from admission solely by reason of his/her disability.