|
|
|
|
|
Standard VII: Institutional and Program Renewal
Narrative Since the last ROPA accreditation visit in 1996, the faculty members in the undergraduate and graduate programs evolved into one Education Faculty. At that time (i.e., 1996), the theme of the Saint Michael’s Education Department was: Learning is at the center of everything we do. With learning at the center of our paradigm, we view successful teaching as the result of knowledge blended from content, pedagogy, and learners. Qualities of successful teachers (i.e., respectful, resourceful, reflective, research-based) add a dimension to our program and make it uniquely ours. Since that time a series of retreats supported the evolution of a single department with a new, shared theme.
New meeting structures were necessary to support the merged department. For the past several years, the Education Department has a whole faculty meeting (e.g., undergraduate and graduate) once per month for 3 hours with a focus on department wide issues that effect both undergraduate and graduate students. Issues that are unique to either graduate or undergraduate students, are dealt with during separate meetings that occur once per month. Faculty who teach in both the undergraduate and graduate programs may attend both meetings depending on the issues being discussed. We also continue to have a Department-wide retreats twice a year with the intention of sustaining our shared agenda regarding the preparation of teachers. The collaborative efforts have had positive impact on both faculty and students. The program as a whole is more coherent and we are making better use of the expertise within the Department including:
Narrative The formal process for ongoing self-assessment is embedded within the action plan that was generated at the conclusion of the 1996 ROPA visit. The goals provided a work plan which the Department maintained over time. Examination of the action plan demonstrates progress at two points in time: 1999 and 2003. The forms of on-going assessment used by the Education Department to inform the ongoing self-assessment include:
Satisfaction Surveys Program Completers A survey of program completers, cooperating teachers and principals is conducted on an annual basis. Those data are examined each fall as a source of information to improve aspects of the program. The data for each of the years since the last ROPA visit are available in the Exhibit Room. Annual Surveys – Synthesis of the Surveys from 1996-2003 On an annual basis, we also examine student, cooperating teachers, and employer satisfaction with the Saint Michael’s program via follow-up surveys. Follow-up surveys of graduates indicate high levels of satisfaction with the Saint Michael’s Programs (raw data are available in the online Exhibit Room). Generally students have enjoyed the variety of courses, high number of courses with field-based requirements and the accessibility of their professors. Interestingly enough, they would also like more of each of those components! For example, respondents noted that they wanted to get into the schools earlier in the program, however, as highlighted on the Field-Based Activity Chart participation in the field begins in their very first Education course at both the undergraduate and graduate levels! Although we would agree with the students and we would invite the opportunity to offer additional methods courses, due to the constraints of the requirements related to their second major, it is impossible to add courses and be able to graduate in a timely fashion. Undergraduate, Elementary Education majors in particular, also noted that they wanted more flexibility within their Liberal Arts requirements. This issue of lessening the numbers of Liberal Arts core requirements was on the ROPA-R Action Plan (1996). As noted on the Action Plan update, the broader faculty did not approve this move. Examination of all of the data affords us with the opportunity to engage in continuous self-assessment Cumulative Survey In preparation for the 2004 ROPA-R visit we took the opportunity to conduct a cumulative survey from the past 5 years. Two different surveys were distributed. 1. Both undergraduate and graduate program completers.
a. Questions 1-5 targeted
levels of satisfaction with the Saint
b. Questions 8-12 inquired about their experiences in
2. Principals who have hired our graduates (indicator 7.05). The survey information was primarily qualitative and descriptive in nature. While each survey had a different set of questions, all were constructed around a Likert scale with accompanying opportunities for short descriptive responses. The Likert data were analyzed using a SAS program and the narratives were analyzed via thematic sorting and categorical coding. Praxis Examinations Another source of assessment data that we use to inform our teaching and learning are the results of the Praxis Examinations. All students must pass both Praxis I and Praxis II in their content areas before student-teaching. Students will not be recommended for Vermont teaching license unless they meet the Vermont cut scores on both tests. Refer to Standards 3 and 4 for in-depth analysis of the Praxis information. Upon receiving the data from the ETS company, the Department Chair conducts an initial analysis of the results. This information is brought to the Education Faculty for further discussion. NOTE: This has been discussed at length in Standards 3 and 4. Portfolios Portfolios are another piece of student data that contributes to our programmatic examination. The undergraduate and graduate student portfolios provide us with the opportunity to study the final product generated by students across programs. Two faculty read each portfolio and are guided by the portfolio report card. Remarkable items are often shared at faculty meetings. Specific descriptions of the portfolio process are found under Standard 4 and examples of student portfolios are in the Exhibit Room.
Narrative Perhaps the strongest evidence of this indicator is the updated Action Plan from the previous ROPA accreditation visit. That document was formally updated in 1999 and in 2003. The entire Education Department participated in the updating process. Each member contributed relevant data and contributed to the analysis. Course Evaluations In addition, as a program we examine the outcome data of our students and faculty. At the undergraduate level we survey students at the end of each course regarding their satisfaction with the course content, pedagogy and faculty using the SUMMA system. The SUMMA evaluations consistently demonstrate that members of our Department rank, on average, higher than the College averages and national averages. At the graduate level, an in-house tailor-made survey is used. Similarly, graduate faculty evaluations are consistently high. Individual course evaluations are confidential documents, and, therefore are not included as data for review in ROPA-R process. At the undergraduate level, summaries of the student evaluation data are reviewed by Dr. Ruoff, Department Chair, first. At the graduate level, Dr. Anne Judson, Director of Graduate Studies, reviews the evaluations. The initial reviews offer both leaders with the opportunity to analyze the strengths and weaknesses across the department by examining individual faculty course evaluations. Faculty, both full-time and adjunct faculty, receive their individual course evaluations and, when appropriate, participate in a dialogue regarding course improvement. Class grades are also reviewed at both levels to ensure that students are making adequate progress toward licensure requirements. At both the graduate and undergraduate levels, mid-semester academic warning notices are sent to students if their performance is falling below standard in any individual course. In such cases, students are encouraged to meet with the faculty member to develop a plan of remediation.
Narrative Undergraduate students have several opportunities to offer their programmatic feedback. First, all seniors complete a survey at the end of their senior year, prior to graduation. This survey provides critical information to both the academic and student life sides of the campus. The second opportunity is a department annual survey sent to education department graduate majors only. The information is used to support our department in the continuous cycle of development. In preparation for the ROPA-R visit, we took the opportunity to conduct a large scale survey of our undergraduate and graduate students who were recommended for initial endorsement between 1995-2002. Undergraduate and Graduate Program Completers Surveys were sent to 243 program completers; eighty-one surveys (i.e., 33%) were returned. Sample Description
Females: N = 72 Certification Areas:
Elementary Education: N = 54 Currently Employed as Teachers: N = 76 States Where Teaching:
Illinois Connecticut Match Between Initial Licensure and Current Assignment
No: N = 13 *Note: This question did not take into account whether or not the graduate added on endorsements after completing his/her initial license. Questions and Responses 1. Did the program offer enough time in the classroom? As shown in the following graph, students were very satisfied with the amount of time they spent in the field during their preparation. Three respondents indicated there was not enough time, 9 responded that there was enough time and 69 responded that there was more than enough time.
2. How well did the program prepare you? Again, levels of satisfaction with their preparation was high among both undergraduate and graduate students. Only 1 former student indicated that s/he was not well prepared; 8 responded that their preparation was average; and 71 indicated that their preparation was above average. Several former secondary education undergraduate and graduate students suggested additional courses in subject areas (e.g., teaching of foreign languages, teaching of English).
All of the raw data are available in the Exhibit Room. Qualitatively, the responses were overwhelmingly positive (refer to the following figure for samples). Chi Square Comparisons Because (a) we use the information from both our course evaluations and the Annual Student Surveys to improve the program on an annual basis and (b) State regulations change over time, we decided to run a Chi Square analysis on the data to see if there were different responses between recent graduates (i.e., 2003-2000) versus earlier graduates (i.e., 1996-1999) with regard to the portfolio experience. There were no statistically significant differences. This information leads us to note with some level of confidence, that our cycle of continuous improvement has been effective from the students perspectives. Cooperating Teachers are critical to the preparation of novice teachers. Our Department (i.e., both undergraduate and graduate levels) is fortunate to have many strong teachers to guide our novices through their final stage of preparation. Annual satisfaction surveys provide us with helpful information as to the strengths and needs of the program. As noted on the attached tables, cooperating teachers indicate consistently high levels of satisfaction with the preparation of their student-teacher. Using a 5 point Likert Rating Scale, Fall 2002 data indicate that 0% of the cooperating teachers thought that the student-teacher was “unprepared” on any given dimension. On every item regarding the preparation (with 1 = very prepared) over 70% of responses rated a “1” or “2”. While not statistically significant, there are two areas (The act of teaching and Classroom management) that cooperating teachers rated “3” which is the midway point, which could be considered average. 23% and 18% of the responses ranked those areas a “3”. The surveys from the Fall 2003 indicate similar findings. As a Department we will set these areas as top priorities for improvement. Portfolio An example of the integration of several sources of data occurred in the summer and Fall of 2003. The annual principal and student surveys were sent out and returned (exhibit room for the raw data). As described in Standard 3, several additional questions were added about the portfolio process. Former students offered several good suggestions, including (a) supporting their growing understanding of the requirements earlier in the program and in a more consistent manner and (b) making the 11 page State assessment rubric more manageable. Examination of the root causes for the first issue led us to concur that students were indeed, reacting to the changing requirements that accompanied the portfolio process as it was evolving across the state. As the portfolio requirements evolved and changed at the State level, the effects were felt at the student level. Thus, their concerns were well founded. At this juncture, it seems that the process has stabilized and we anticipate being able to guide our students through the process and requirements of the portfolio process in a routine manner. A tangible change that we have made was the development of a one credit portfolio development course for Elementary Education majors. As part of this course, Education faculty often facilitate portfolio workshops that are targeted for those enrolled in the course. The addition of the portfolio course is only a requirement for the Undergraduate Elementary Education majors because the other majors (i.e., undergraduate Secondary Education and all graduate licensure programs) have been able to incorporate the portfolio into existing courses). In order to address the issue of manageability of the State assessment rubric, members of the department developed a portfolio report card. Beginning in January 2004, this will be used at both the undergraduate and graduate levels as a way of supporting students in managing the assessments related to the licensure portfolio artifacts. It is anticipated to take a few years before it becomes a familiar part of the students routine.
Narrative Principal Satisfaction Surveys Ninety-three principals returned the surveys that had been hand delivered to them via individual graduates who were working in their schools. Because the surveys were hand delivered, we do not have any sense of the total N size. Sample Description
Females: N = 75 Certification Areas:
Elementary Education: N = 62 Teachers from SMC Graduate or Undergraduate Program
Undergraduate N=53 Responses on each of the 12 questions indicate very high levels of principal satisfaction. Given a Likert Scale with 5 indicating the identified behavior is observed almost always and 1 indicating rarely observed, principals indicated above average (i.e., 4 or 5) 90% or better on each question. There were no statistically significant differences between the responses of principals who hired students from our graduate versus undergraduate teacher education programs. Following is a list of the 12 questions posed and the responses (the raw data are available in the Exhibit Room in the box labeled Standard #7). Principal Survey Questions
The narrative responses from the principals were very positive. The sample of positive responses reflect the majority of the principal responses. There were also a few very specific suggestions for improvement (refer to the Table). The suggestions that will have a direct impact on our action plan are: ways to work with schools in supporting early career teachers and differentiated instruction.
Cross referencing this cumulative survey (1997-2003) with the annual surveys yields similar findings. The majority of principals have had very positive experiences with our teacher education graduates whether the teachers are from the graduate teacher licensure program or the traditional undergraduate teacher education program.
Narrative In May of 2004, Saint Michael’s Education Department met to study the ROPA-R data with the intention of outlining our own commendations and recommendations. This process provided us with a better understanding of the program as a whole. From that point, we we able to leverage our strengths in order to develop the next five year action plan. This action plan identifies annual and long term goals, includes an implementation timeline, and specifies the data sources that will be used to assess growth towards meeting these goals.
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
graduate faculty were offered the opportunity to shift their faculty lines
from instructor to tenure track lines (previously tenure was not an option
in the graduate program), 
