Self Study

The Educator Preparation Program (EPP) at the University of Central Missouri (UCM) assesses teacher candidates' content and pedagogical knowledge at multiple points as they progress through their respective programs. The assessments selected to meet the goals of the EPP are aligned with the Missouri Teacher Standards (MTS) (R1.1.1 Missouri Teacher Standards) and the InTASC Standards (R1.1.2 Assessment Grid and Alignment).

The EPP quality assurance system uses multiple data points across the program progression to assess candidates' performance and readiness to teach (R1.1.2 Assessment Grid and Alignment). The EPP administers five unit assessments to all teacher candidates to monitor individual progress, to provide evidence for use in program evaluation, and to monitor progress and inform decisions at the unit level.

The five unit assessments include:

• Missouri Content Assessment (MOCA) (R1.1.3 MOCA Template),
• Missouri Educator Evaluation System assessment (MEES) (R1.1.4 MEES Template),
• Educator Dispositions Assessment (EDA) (R1.1.5 EDA Template),
• Student Teacher Work Sample (STWS) (R1.1.6 STWS Template), and
• Student Teacher Exit Survey (STES) (R1.1.7 ST Exit Survey Template).

The linked documents provide information about these assessments including alignment to standards, administration details, a copy of the instrument if appropriate, unit and disaggregated data tables, and data analysis. Coursework and clinical experiences prepare candidates to understand the critical concepts and principles of their certification areas and to effectively teach diverse P-12 students. An analysis of candidate performance data, aligned to InTASC Standards, is provided in the narrative as relevant for each component of Standard R1. Much of the data provided in these documents is also reflected in the EPP's most recent Missouri Department of Elementary and Secondary Education (MoDESE) Annual Performance Report (APR). In Missouri, all EPPs are reviewed annually by the state on a variety of criteria and an APR report is issued and shared publicly to show each EPP's performance over the past three years (R1.2.3 APR). As this report shows, overall evidence demonstrates that teacher candidates (TC) from the UCM EPP have the content and pedagogical knowledge necessary for effective teaching. 

Evidence of the ability of UCM's TC to apply knowledge of the principles of learner development, learning differences, and the creation of supportive learning environments is provided through their performance on MTS Standards 2, 4, and 5 (InTASC 1, 2, and 3) as assessed by the MEES; sections 1 and 2 of the STWS; disposition 6 as assessed by the EDA; and sections 1, 3, 4, 5, 6, 7, 8, and 9 of the STES (R1.1.8 Data For Component 1.1).

MTS 2 (InTASC 1 & 2) requires that TC understand how students learn, develop, and differ in their approaches to learning and that the TC provide learning opportunities that are adapted to diverse learners and support the intellectual, social, and personal development of all students. Additional MTS standards relevant to R1.1 are MTS 4 critical thinking, requiring the TC to use a variety of instructional strategies and resources to encourage students' critical thinking, problem solving, and performance skills and to promote students' active participation and success in the learning process (InTASC 1); and MTS 5 which requires that TC use an understanding of individual/group motivation and behavior to create a learning environment that encourages active engagement in learning, positive social interaction, and self motivation (InTASC 3). Data for standard R1.1 were collected from completers in Fall 2020, Spring 2021, Fall 2021, and Spring 2022. Four cycles of data are included in the self-study when possible because the COVID-19 pandemic caused partner schools to be on virtual or hybrid learning from Spring 2020 - Spring 2021 and opportunities to place TC in classrooms were limited; therefore, there were gaps in data due to the MoDESE allowing exemptions from certain assessments; additionally TC performance possibly was adversely impacted by the circumstances of the pandemic.

 

Missouri Educator Evaluation System assessment (MEES)

The MEES is a proprietary instrument (R1.1.4 MEES Template) and the EPP use of the MEES for assessment of TC is mandated by the Missouri Department of Elementary and Secondary Education (MoDESE). The same instrument is used to assess practicing classroom teachers across the state at all career levels. On all standards evaluated by the MEES, candidates are assessed on a scale of 0 to 4. The expected performance level for TC completing student teaching is 3 - Skilled Candidate, upon completion of the student teaching semester. The scoring descriptors are shown on the MEES Rubric (R1.1.4 MEES Template). During student teaching, university supervisors (USUP) complete three formative assessments using the MEES to provide growth feedback for TC. It is also used in a summative manner, with each TC completing a self-assessment in addition to USUP and cooperating teachers (CT) completing a MEES at the end of the student teaching semester. Scores from the USUP and CT are combined to submit to the state for teaching certification. For the 299 TC assessed, means on all nine MEES standards across the Fall 2020, Spring 2021, and Fall 2021 semesters were statistically significantly higher than the target level of 3. Independent samples t-tests were conducted to determine if there were differences in MEES ratings based on a variety of factors. On average female TC had slightly higher ratings across each rater. There was no statistically significant difference in total MEES scores for female or male TC nor in mean total scores between white and non-white TC by USUP and CT raters. However, there was a statistically significant difference in MEES mean scores on TC self-assessments between white and non-white candidates. Non-white TC self assessed MEES total score mean (30.45) was higher than white TC self-assessment MEES total score mean (29.70) (R1.1.9 Analysis of Unitwide Data). Finally, there was a statistically significant difference between Fall 2020 and Fall 2021 total MEES mean scores, as Fall 2021 TCs were scored higher than Fall 2020 on the total MEES mean, which may be attributed to shifts to all-online learning and teaching in Spring 2020 due to COVID-19 in both university and public school settings.

The MEES items that align with R1.1 are items 2, 4, and 5. For all 4 sets of completers, MEES scores on 2.4 "Differentiated Lesson Design" from USUP averaged 3.2-3.3; from CT averaged 3.2-3.4; and from TC averaged 3.2-3.4. Similarly, average scores on 4 "Student Engagement in Critical Thinking" were all at or above 3. Scores from all scoring groups on 5 "Classroom Management" exceeded 3 and ranged from 3.0-3.6 which is higher than for the other items that assess Standard R1.1. These scores provide evidence that UCM completers across programs are knowledgeable in understanding and providing a supportive environment for different learners including effectively implementing classroom management strategies.

 

Student Teacher Work Sample (STWS)

The STWS is an EPP created assessment (R1.1.6 STWS Template); the Education Renaissance version of the Teacher Work Sample was adopted by the UCM EPP prior to adoption of the Missouri Pre-service Teacher Assessment (MoPTA) and utilized as a performance assessment during student teaching until 2015 when MoDESE implemented required use of the MoPTA. When the MoPTA was eliminated as a state requirement in 2018, the UCM EPP decided to revise and implement the STWS as a performance assessment to provide more comprehensive TC data and allow for triangulation of data at the student teaching level. Two cycles of data for the STWS are available for the original STWS (Fall 2020 and Spring 2021) and two cycles are provided for the revised STWS (Fall 2021 and Spring 2022). Data from the first two sections of the STWS are utilized to produce evidence for R1.1. Section 1 Design for Instruction includes the Contextual Factors and the Lesson Planning components and Section 2 Analysis of Student Learning requires candidates to analyze student learning. The first two cycles of data (pre-revision) show that TC averaged on all items within 1.1 points of the highest score possible and for the second two cycles (post-revision) TC averaged within less than 1 point of the highest score possible and no average was lower than 93% of the total amount of points for each item in Sections 1 and 2. Overall results on the entire STWS assessment disaggregated by specific program of study, race, and gender reveal no significant differences in average scores on any single component, section, or overall performance. Each cohort of completers boast a 100% pass rate on the STWS which is expected as this assessment is a requirement for graduation and recommendation for certification from the EPP. However, ratings on the STWS indicate very strong performance by all TC across all groups, and all but one subgroup's averages were above 93% average total score. The only subgroup exception (Middle School, Fall 2021; N=5) can be explained by the small subgroup size; one low score in the very small number of TC in this cohort did cause the average to be negatively skewed.

 

Student Teacher Exit Survey (STES)

The STES (R1.1.7 ST Exit Survey Template) consists of 38 closed-questions divided into groups corresponding to each of the nine MTS and is meant to assess TC perceptions regarding preparation at program completion. All but one item are constructed in a 5-point Likert-type scale of agreement ranging from 1- Strongly Disagree, to 5 - Strongly Agree. Self-ratings on items that align with R1.1 reflect TC perceptions of their preparedness to incorporate interdisciplinary instruction, engage students in the content area, make content meaningful to students, design lessons that include differentiated instruction, implement instruction based on a student's IEP, modify instruction for English language learners, modify instruction for gifted learners, and create lesson plans to engage all learners. Data were not disaggregated by nature of program, certification area, gender, or race because no identifying information is required on the exit survey. Average scores for all TC on 5 of the 8 items aligning to standard R1.1 consistently ranged from 4.05 - 4.63 on the 5-point scale for all four data cycles provided. These high average ratings suggest that the majority of TC felt well-prepared to provide engaging lessons in their respective content areas as well as interdisciplinary instruction. Average scores on the remaining 3 items that align with standard R1.1 were somewhat lower, ranging from 3.39 - 4.2. Average scores on these items indicate that completers do not feel as well-prepared to tailor lessons for all different needs or exceptionalities. All averages still indicate that the majority of TC were in moderate agreement that they were prepared in their programs to modify instruction to accommodate different learner needs as a program completer.

Five of the items on the STES (R1.1.7 ST Exit Survey Template), aligned with InTASC 1 and 2 and MTS, provide evidence of TC perception of their preparation to design differentiated lessons, implement instruction based on an IEP, modify instruction for English language learners, modify instruction for gifted learners, and create lessons to engage all learners. Data from these items provide evidence that TC are comfortable and confident in understanding many aspects of diversity and effectively working with some subgroups - but not equally confident across all subgroups of P-12 learners. An Access, Opportunity and Community (AOC) Workgroup was convened to respond to these consistent data patterns and improve the preparation of TC for working with highly diverse P-12 populations. Several actionable recommendations came from this group (R1.1.10 Diversity Workgroup) including a statement on diversity adopted by the Teacher Education Council in Spring 2022. The focus on preparing TC to work effectively with a diverse group of P-12 learners begins early in the preparation program with two required courses specifically focusing on language, cultural, and learning and ability differences (EDFL 2250 Introduction to English Language Learners and Culturally Responsive Pedagogy and EDSP 2100 Education of Exceptional Children); subsequent coursework continues to address individual and group differences (R1.1.10 Diversity Workgroup). Furthermore, candidates are exposed to diverse learning environments in the sequence of field experiences (R1.1.11 Demographics of Districts).

 

Educator Dispositions Assessment (EDA)

The EDA is a proprietary instrument (R1.1.5 EDA Template) chosen by the UCM EPP to assess TC dispositions. The EDA instrument is used formatively and summatively to assess candidate dispositions at the program and unit level beginning with assessment during the first set of professional education courses and continuing through student teaching. The EDA uses a 3 point scale ranging from "0" representing Needs Improvement to"2" representing Meets Expectations. TC must receive a minimum rating of 1 on each dispositional item to be enrolled in student teaching and must receive a minimum rating of 1 on each dispositional item during student teaching to be recommended for certification by the EPP. Data from EDA items relevant to this standard provide evidence that the EPP prepares TC to work with all learners. On disposition 6, Exhibits an Appreciation of and Value for Cultural and Academic Diversity, on both items a and b, average ratings for all TC were 1.8/2 or above; ratings on this disposition disaggregated by specific nature of program, program of study, race, gender, and first generation status reveal no significant differences in average scores on disposition 6a or 6b. These results indicate that each TC recommended for certification demonstrates through professional and personal behaviors and interactions that they "embrace all diversities as evidenced by implementing inclusive activities and behaviors with goals of transcendence" and "create a safe classroom with zero tolerance of negativity to others as evidenced by correcting negative student behaviors".

The data presented for R1.2 provide evidence that TC know central concepts of their content area and are able to apply content to develop equitable and inclusive learning opportunities for diverse P-12 students. Central concepts of each program area are mastered by candidates through required content coursework. Elementary and Early Childhood Education TC complete core coursework and methods courses in all core areas (literacy, math, science, and social studies); Special Education TC complete specific methods and content courses; and Secondary, Middle School, and K-12 education TC must complete extensive content courses. Initial certification candidates take methods courses and learn to apply content knowledge in the classroom through field and practicum experiences. Content knowledge is assessed through multiple instruments including specific items on the MEES, the STWS Lesson Plan, the MOCA, multiple items on the STES and the EDA disposition 6. Analysis of data from assessment items aligned to R1.2 Content Knowledge (R1.2.1 Data for Component R1.2) indicates that UCM EPP TC are prepared to meet this standard.

 

Missouri Educator Evaluation System assessment (MEES)

The MEES items that align with content knowledge are (1) Student Engagement in Subject Matter; (2) Differentiated Lesson Design; (3) Implementation of Curriculum Standards; and (6) Verbal and Non-Verbal Communication (for description and technical information on the MEES see R1.1.4 MEES Template). Mean scores from all rating sources (USUP, CT, and TC) on these items for all TC in all four data cycles are above the target score of 3 for TC. Further evidence of TC preparation to utilize content-specific pedagogy and apply the central concepts of a content area is provided by data from the STWS Lesson Plan (R1.2.2 STWS Lesson Plan Format). The lesson plan was added to the STWS when it was revised and so this component of the assessment was first required in Fall 2021; thus, only 2 cycles of data have been collected using the STWS Lesson Plan. Across all six assessment points on the STWS Lesson Plan, all means are very high, ranging from 92-99% of the total possible points possible on each lesson plan component (R1.1.6 STWS Template). These data suggest that all TC are proficient in developing lesson plans that are appropriately aligned to relevant standards; sequencing, differentiating, modifying, and creating accommodations with knowledge of their learners; and assessing content effectively. A third source of data is the MOCA, a proprietary instrument required for certification by MoDESE (R1.1.3 MOCA Template for description and technical information). The MOCA is part of the Missouri Educator Gateway Assessment (MEGA) program developed by and administered through Pearson Education. Data from four other instruments (the MEES, STWS, STES, and EDA) are triangulated with MOCA licensure exam pass rates, indicating that TC have the content knowledge necessary for effective teaching.

 

Missouri Content Assessment (MOCA)

The EPP MOCA pass rates, overall and by license group, demonstrate that TC have the content knowledge to pass licensure exams before the end of student teaching; thus, the majority of TC are eligible for licensure upon completing their degree program or alternative certification program. In the four cycles of completers from Fall 2020 to Spring 2022, the overall pass rate prior to the completion of student teaching for each respective completer group was 82.4%, 90.1%, 85.3%, and 88.3%. These pass rates have stayed consistently high even for cohorts that had to transition very suddenly to all-online courses and manage the stressors of the pandemic when the university and public schools closed in-person learning and offered 100% virtual learning. Due to the pandemic, MoDESE temporarily allowed a waiver of the MOCA for certification; TC seeking certification in Spring 2020 did not have to pass the exam. Testing centers which closed in spring of 2020 weren't fully operational again until Spring 2021. The lowest pass rate reported was the first semester back (Fall 2020) as educators struggled to return to semi-normalcy and consistency for students at both the P-12 and the university level. However, in addition to the pass rate of 82.4% prior to the completion of student teaching, 6 additional TC passed the MOCA after graduation, bringing the total to 91% pass rate. Average number of attempts was 1.34. For Spring 2021, the pass rate of 90.1% prior to program completion was increased to a pass rate of 97% when 12 TC passed the test after program completion; average number of attempts was 1.25. For Fall 2021, the 85.3% pass rate was increased to 93% when 6 TC passed after program completion; average number of attempts was 1.2. For the final data cycle Spring 2022, the initial pass rate was 88.3% with average number of attempts 1.2. At the time of writing this report, scores for spring 2022 TC taking the MOCA after program completion were not yet available from Pearson. A notable trend is that the average number of attempts has decreased slightly over the past 4 cohorts of completers.

 

Educator Dispositions Assessment (EDA)

The fourth source of data provided as evidence for standard R1.2 is the EDA, a proprietary instrument chosen by the EPP to assess TC dispositions (R1.1.5 EDA Template), specifically for dispositions 6a (Embraces all diversities as evidenced by implementing inclusive activities and behaviors with goals of transcendence) and 6b (Creates a safe classroom with zero tolerance of negativity to others as evidenced by correcting negative student behaviors). Average scores out of a high score of 2, for all data cycles range from 1.93 to 2 and this provides evidence that TC demonstrate the ability to apply content in developing equitable and inclusive learning experiences for diverse learners.

 

Student Teacher Exit Survey (STES)

The fifth and final source of data that provides relevant evidence for R1.2 is the STES (R1.1.7 ST Exit Survey Template). All 18 items across standards 1, 2, 3, 4, and 6 on the STES provide evidence that the UCM EPP completers believe they are prepared to make content relevant, engaging, and developmentally appropriate for a wide range of diverse learners and promote critical thinking and problem solving in their content areas. On 15 of the 18 items, averages were all above 4 on the 5-point scale for all four data cycles. The only ratings that averaged below a 4 were on the three items related to accommodations and modification for learners with an IEP, English language learners, and gifted learners. Averages on these items ranged from 3.38 - 4.2, remaining positive but indicating that completers have less perceived proficiency in making appropriate modifications than in other areas.

Component R1.3 highlights the importance of educators' ability to apply their knowledge of InTASC standards related to instructional practice at the appropriate progression levels. The UCM EPP ensures that TC can assess, plan, and utilize a variety of instructional strategies to provide equitable and inclusive learning experiences for a diverse student population (R1.3.1 Data for Component 1.3). Data analysis of EPP assessment items aligned to InTASC Standard 6: Assessment, indicates that the EPP is preparing TC to meet component R1.3.

 

Assessment Planning

In all four data cycles reported, TC averaged above the target score of 3 on MEES Standard 7 (InTASC 6) Effective Use of Assessment. The STWS requires TC to plan multiple forms of assessment, both formal and informal, to measure student progress. Average scores of completers were 4.64 and 4.7/5 for the most recent two data cycles (revised STWS) with a similar trend of average scores above 90% on the first two data cycles (pre-revision) and the second two data cycles (revised STWS) for the components of data analysis and impact on student learning. STES data further reflects that TC understand and are prepared to use, develop, and analyze data from multiple methods of assessment to engage learners, to monitor learner progress, and to guide decision making. Further, TC report that they are well prepared to assist students in setting goals and to collaborate with colleagues to analyze assessment data.

 

Instruction Planning

All relevant data sources provide evidence that TC maintain high average ratings for planning instruction to support each student (InTASC 7). Ratings from all raters on MEES Standards 2, 3, 4, and 6 provide evidence that TC are strong in planning instruction with all average scores above the target score of 3. The STWS (revised) Lesson Plan is a second source of evidence that demonstrates TC strength in planning with average scores between 92% and 99% on each component of the lesson plan which includes the requirements for differentiation, accommodations and modifications for different learners. Average ratings of 1.91-1.99/2 for all 4 data cycles across all four components of Standard 5 on the EDA support TC preparedness and ability to plan, and the STES items for MTS 2, 3, 4, and 6 provide additional evidence that TC believe they are well-prepared to plan for instruction and accommodate all learners. As previous data analysis of three items on Standard 2 (implementing an IEP, accommodating for English language learners, and accommodating for gifted students) stated, TC feel moderately positive about preparation in these areas but their average scores are lower on these three items.

The progression of expectations in practicum placements (R1.3.2 Sequence of Clinical Experiences) demonstrate that TC are expected to learn about and execute effective instructional practice (InTASC 8) in increasing complexity through content and methods coursework, early and mid-level field experiences, and student teaching. Assessment data from the student teaching semester provide evidence that TC have met these expectations. These include ratings on the MEES (MTS 1, 2, 3, 4 and 6; InTASC 8); all ratings averaged above the target score of 3 from all raters (CT, USUP, and TC). TC also earned ratings that demonstrate competency in instructional practice on the STWS (resources, differentiation, accommodations and modifications), ratings that demonstrate preparedness in teaching and learning (MTS 5) on the EDA, and high ratings from TC on perceived effectiveness of preparedness to vary instruction on the STES (MTS 3-6).

 

Technology Application

The integration of technology is addressed within R1.3 as it relates to instructional practice. During coursework, TC learn to apply the International Society for Technology in Education (ISTE) Standards to engage students and enhance learning. Additionally, all Spring 2022 completers were required to successfully complete three technology modules (R1.3.3 Technology Modules) during their final semester of coursework in Fall 2021. The EPP is adopting this requirement as a transitional assessment that will help to ensure that UCM completers will master technology for the enhancement of P12 learning (R1.3.4 Transition Plans). A data source that provides evidence of TC proficiency with use of technology to enhance instruction is the STWS (revised) (R1.1.6 STWS Template). In the two data cycles since this instrument has been implemented, TC average scores were 4.83 and 4.84/5 on the task requiring them to provide examples of their use of educational technology, a rationale for the selection of the technology, and a reflection on the impact on student learning.

Educator Dispositions Assessment (EDA)

The data presented for R1.4 provide evidence that the UCM EPP TC are able to apply their knowledge of professional responsibility. The EPP ensures that TC engage in professional learning, act ethically, and take responsibility (InTASC 9), and collaborate with P-12 students and families (InTASC 10). TC are introduced to professional ethics in their required EDFL 2100: Introduction to Professional Teaching course. They are introduced to professional dispositional expectations in their first set of education courses where program faculty complete a dispositions assessment (a modified EDA) as a formative assessment. This allows program faculty to meet with TC of concern and provide guidance and coaching regarding professional dispositions (R1.4.1 Dispositions Implementation). Dispositions are formatively assessed at the mid-level field experience as well, and at this level once again faculty members conference with any TC who scored a zero on any disposition, and work with the TC to create a professional development plan or create learning and performance targets. Each TC must receive a recommendation to student teach from their respective program; a TC who has received a zero on any disposition at that point cannot be placed in student teaching. Similarly, a TC who has received a zero on any disposition during student teaching may not earn a grade higher than a C in student teaching and could be subject to suspension/removal. Consistent trends in strong mean scores were found on dispositions 3, 4, 7, 8, and 9 for the summative dispositional assessment during student teaching (R1.1.5 EDA Template).

 

Aligned Items from MEES, STWS, and STES

In addition to the EDA, to allow triangulation of data to provide evidence for the key concepts within R1.4, the EPP has selected aligned items from the MEES, STWS, and STES to analyze TC performance (R1.4.2 Data for 1.4). The aligned items on the MEES were 8.1 Reflection and Self-Assessment and 9.3 Cooperative Partnerships. All raters (USUP, CT, and TC) provided average ratings of higher than the target score of 3 across all four data cycles. On the STWS on the tasks of cooperative partnerships and professionalism, average scores of 4.51-4.9/5 across all four data cycles provided corroborating evidence of strong professional behavior. Data from the EDA provides additional evidence of TC strengths in the area of professional responsibility; scores from items aligned with R1.4 range from 1.92-2 on a 2-point scale. TC are introduced to professional ethics in the required course EDFL 2100 Introduction to Professional Teaching; a SLO from the course states "Students will think, value, communicate and react to legal/ethical issues, educational research, and changes in education organization and practices." The NEA Code of Ethics is introduced, and TC discuss various scenarios involving ethical decision-making.

 

InTASC Standards

Finally, TC perceptions reflected in responses on the STES offer evidence that they feel confident in their ability to apply the InTASC Standards in relation to collaboration and leadership. On standards 8- Professionalism and 9- Professional Collaboration, TC ratings of preparation ranged from 3.77 - 4.58 on a 5-point scale. One rating that averaged below 4 was on the item "I was prepared to collaborate with parents to support student learning." Although still in the moderate range, this rating does indicate challenges for the EPP. TC collaborating with parents is largely dependent upon the comfort of the classroom teacher and/or the building norms. Confidentiality and parent relationships are two of the reasons TC are not allowed to participate. COVID restrictions also led to fewer in-person opportunities for collaboration. However, TC do begin to practice collaboration during student teaching and evidence of this collaboration is provided by Table 4 Cooperative Partnerships in Support of Student Learning, a section of the STWS (R1.1.6 STWS Template) in which candidates document collaboration and communication with parents during student teaching (R1.4.3 Examples of Partnering with Parents).

High-quality clinical partnerships are central to the success of UCM's Educator Preparation Program (EPP). The EPP has a long history of partnerships with a wide variety of area districts. Partnership models have included a Lab School, Professional Development School, and the current Clinical Model. With an extensive range of district partners, the EPP ensures that teacher candidates (TC) participate in clinical experiences that contribute to their effectiveness and have a positive impact on the diverse P-12 students in the region and state.

CoE Advisory Board

The UCM EPP works with partners to co-construct mutually beneficial P-12 school and community arrangements for clinical preparation and share responsibility for continuous improvement of candidate preparation. The EPP uses a variety of strategies to develop, maintain, and promote the evolution of partnerships. One tool used to create and maintain partnerships is the formation of advisory boards, which are formed to serve the EPP as a whole, each program in the EPP, and the Office of Clinical Services and Certification (OCSC). The College of Education (COE) Advisory Board which serves the unit of the EPP was instituted more than 15 years ago. Membership comprises representatives from a range of partner districts, some of whom are UCM alumni. The current composition of the board includes representatives from 21 public school districts in the region; of the 40 members, 36 are administrators (either building administrators or district-level administrators) and four are teachers (R2.1.2 Advisory Board Examples). COVID restrictions and the associated challenges faced by school districts prevented in-person advisory board meetings in Spring 2020, Fall 2020 and Spring 2021. Due to these challenges, the EPP did not hold Advisory Board meetings in 2020. With the easing of the pandemic and changes in COE leadership, the COE Advisory Board was reconvened in Spring 2021 and advisory board members provided input via a survey (R2.1.3 Spring 2021 Survey). In the Fall 2021 and Spring 2022, the COE Advisory Board held virtual meetings via Zoom (R2.1.6 COE Advisory Board Minutes).

Advisory Board meetings are utilized to capture focused feedback on specific initiatives, programs, assessments, clinical placements, and other similar topics. For example, during Spring 2021, the EPP decided to revisit the mission, vision, and goals for the unit and acknowledged the importance of EPP partners providing guidance on any potential revisions. The survey completed by the COE Advisory Board in spring 2021, asked for input that would be used in revising the mission, vision, and goals (R2.1.7 COE Adv Board - Goals, Mission, Vision). The goals adopted by the EPP reflect the input provided by partners. The survey also provided valuable information for the EPP in continuing to address challenges associated with clinical placements during the pandemic. A second specific example of how the COE Advisory Boards contributed to the shared responsibility for continuous improvement of TC preparation is that the Advisory Board also provided feedback and insights regarding the adoption and use of the Educator Disposition Assessment (EDA) (R1.1.5 EDA Template) to assess candidate dispositions. They provided professional opinions in two areas: first, regarding appropriate and desirable characteristics of TC entering the EPP as compared to TC at completion of the preparation program; second, they suggested which dispositions should be assessed at each checkpoint in the program (beginning, middle and end). This information was collected during an Advisory Board meeting following discussion among members and contributed to the decision to modify the tool to focus on specific dispositions at each point as suggested by the data (R2.1.4 Advisory Feedback on EDA).

 

Clinical Advisory Board

A Clinical Advisory Board was created specifically to facilitate the collaboration of the EPP with district partners in reviewing and revising requirements of each level of clinical placement, and collaborating on the assessments used at the various levels (R2.1.5 Clinical Services Advisory Board). This group was convened by the Director of the OCSC. At the initial meeting of the Clinical Advisory Board, members were invited to share responsibility for defining clinical partnerships and data were collected in the form of responses to prompts such as "What do you think makes a strong clinical partnership between the university and schools?" and "How can the UCM College of Education support school districts in terms of teacher education initial preparation?" and "How can we improve the process of student placement?" The Director of the OCSC analyzed responses and provided recommendations based on that analysis. For example, one theme that emerged was support for full year senior placements for Secondary and K-12 TC. With support from the Teacher Education Council (TEC) and program coordinators for the Secondary and K-12 programs, OCSC staff work with district partners to place TC in the same classroom for both the practicum completed the semester prior to student teaching and for student teaching. Of course, there are some instances where this does not work for either the district or the TC, and the OCSC is flexible in such situations. Another theme that emerged was the need for placements to be determined early to allow full preparation and training of the cooperating teachers. The OCSC is working to address this request.

 

Program Advisory Boards

In addition to the COE Advisory Board and the Clinical Advisory Board, most programs have established their own advisory boards that meet annually to provide input to the program. As is true of the COE Advisory Board, the pandemic caused some disruption in program advisory board meetings although some advisory boards did meet via distance during pandemic restrictions. The advisory board for each program primarily includes teachers who serve as cooperating teachers for UCM teacher candidates in that content or program area. For example, the Art Advisory Board brings together 19 practicing and three retired teachers from 10 districts to provide feedback on program changes, assessment data, changes and recommendations and other issues directly relevant to the K-12 Art Education program. The Early Childhood/Elementary Education Advisory Board has a current membership of 39: 19 early childhood and elementary teachers, 19 building administrators and one district administrator representing 18 public districts and one Head Start Grantee (which includes 18 public school district sites). Program advisory boards are essential in providing input on a variety of program-specific issues including their perspectives on clinical experiences.

 

Memorandum of Understanding

While advisory boards provide a vehicle for collaboration between the EPP and wide ranging partner districts, a Memorandum of Understanding (MOU) provides a formal framework for structure of the partnerships (R2.1.1 Examples of MOUs) outlining the purpose, governing structure, legal considerations, and terms of the partnership, as well as the responsibilities of both parties as it pertains to candidate field experiences at different levels. The Missouri Department of Elementary and Secondary Education (MoDESE) developed the MOUs to be used in all partnerships between an Institute of Higher Education (IHE) EPP and a public partner district. The MOU specifies how candidates are placed and how cooperating teachers are assigned, duration of placements and termination, qualifications and compensation of cooperating teachers, and other guidelines that support the clinical aspect of teacher education in the state. As directed by the MOU, districts are responsible for determining placements for practicums and student teaching. For the UCM EPP, the OCSC maintains relationships and MOUs with each district in which any TC will be placed for an initial, mid-level or final field experience. They make formal requests for placements in whatever manner the district requires but ultimately it is the responsibility of the district to identify qualified cooperating teachers while it is the responsibility of the EPP to ensure that candidates are dispositionally appropriate and have sufficient content and pedagogical knowledge.

 

Future Teacher Academy

A recruitment effort called Future Teacher Academy (FTA) (more information about FTA in R3) provides another way to collaborate with partners. FTA takes place in the fall when high school students interested in the teaching profession participate in an academy held on the UCM campus. In Fall 2021, the high school sponsor teachers attending FTA with their students provided information regarding "specific knowledge, skills, and dispositions relevant to serving diverse populations and ensuring equity in opportunity that program completers will need to meet the challenges of their initial professional roles" (R2.1.8 FTA Sponsors - Diversity) and this input was used by the Access, Opportunity and Community Workgroup in making recommendations to promote more awareness and equity in EPP culture, curriculum, and support for students.

 

Reduced Cost Graduate Cohorts

The EPP values the strong partnerships it shares with school districts and seeks opportunities to serve its partners. The EPP has partnered with districts to establish cohorts of district teachers to complete graduate degrees at a reduced cost with some courses taught by district employees. These Master of Science in Education degrees have been offered in this manner to districts: Special Education, Elementary Curriculum and Instruction, Elementary Mathematics Specialist, and English Language Learners.

Cooperating Teachers (CT)

UCM TC are supervised and coached by two clinical educators during the student teaching experience and clinical experiences prior to student teaching; the cooperating teacher (school-based clinical educator) and the university supervisor (university-based clinical educator). Cooperating teachers (CT) are selected by partner districts based on criteria determined by MoDESE (R2.2.1 MOSPE Requirements), required by UCM EPP, and specified in the MOU. CT selection requirements vary by field experience level. CT criteria for hosting student teachers as outlined in the MOU include that the CT be a full-time member of the school district's faculty, have at least three years of successful teaching experience, have been employed by the school district for at least one year, hold an appropriate content area/grade range license for the assigned candidate, have demonstrated exceptional competence in teaching, hold a master's degree, and be recommended by district officials and approved by the OCSC.

 

University Supervisors (USUP)

The EPP relies upon the expertise of highly qualified university supervisors (USUP) to support and evaluate TC during student teaching and clinical experiences prior to student teaching. USUP are expected to have knowledge of teacher education, teaching methodologies, and public schools as well as the developmental level of the majority of the EPP's TC. They must meet the EPP requirements as outlined in the MOU (R2.1.1 Examples of MOUs) which include that they hold a minimum of a master's degree, have a minimum of 3-5 years in public schools and/or teacher education, and undergo all trainings required by the EPP. The USUP are primarily current program faculty members, retired K-12 educators and administrators, and retired EPP faculty. The number of USUP within a program area is determined by the number of candidates enrolled in student teaching for the semester. Programs with lower enrollments often rely on full-time university EPP faculty members to supervise candidates.

 

CT Placement Preparation

Each academic year, the EPP places an average of 720 candidates with qualified CT for beginning and mid-level field experiences and student teaching in an average of 52 districts across Missouri. Most placements are in the central and western region of the state. For the student teaching placement specifically, many CT have previously hosted a TC; over the past four semesters 62% of CT had hosted at least one student teacher previously and an average of 40% of CT had hosted more than one student teacher previously. An average of 17% of the CT over the past four semesters were male and an average of 5.5% were persons of color, and the majority of CT (an average of 82.5%) had more than five years teaching experience (R2.2.3 CT Characteristics). To prepare CT for UCM EPP and state required student teaching assessments, the OCSC traditionally provided in-person training prior to the start of each semester. The pandemic restrictions resulted in these trainings being converted to an all-online format completed individually by each CT. 

Since the restrictions have eased, districts have expressed a desire to retain the virtual modality of training rather than revert back to in-person training due to a shortage of available substitutes. In addition to training for the Missouri Educator Evaluation System (MEES), the Educator Disposition Assessment (EDA), and the Student Teacher Work Sample (STWS) assessments, some programs provide program-specific training for the CT hosting their TC in pre-student teaching clinical experiences. For example, Elementary and Early Childhood Education faculty host a meeting in each building for CTs at the beginning of each semester prior to the first day of field experience. The CT are provided with an overview of the expectations, assignments, and role of the candidate at each clinical level (R2.2.2 Block Info for CT). However, this training for CT in pre-student teaching experiences is not consistent across all programs. The OCSC has been charged with ensuring consistent training going forward (R2.2.7 Data Based Decisions).

 

USUP Placement Preparation

USUP also receive training each semester to prepare for supervising and assessing student teachers and to re-calibrate expectations and ratings for the different required assessments. The OCSC hosts three meetings for USUP each semester: student teaching orientation meeting, midterm student teaching meeting, and the USUP brunch at the end of the semester. Along with providing pertinent information for the semester, required trainings occur during these meetings. One training required by MoDESE is for the MEES which is used as a formative and summative assessment during student teaching. The training provides an overview of the MEES and then participants view classroom observation videos and assess the teacher in the video using the MEES, followed by discussion and shared rationalization among small groups of USUP to promote scoring reliability (R2.2.4 MEES Training). Another training, required by the EPP, is for the EDA. This training requires USUP to review the items on the EDA at the student teaching level and complete an interrater reliability training during the meeting (R2.2.5 EDA Training). For any USUP who is not able to attend the meeting in person, an online training is available.

 

TC and USUP Feedback on CTs

The EPP solicits feedback from USUP and TC to evaluate CT and their impact on TC success. During the student teaching semester, the USUP and TC complete a Cooperating Teacher Feedback Survey to confirm that the CT performed pertinent duties and created a positive learning environment for the student teacher (R2.2.6 Feedback CT, TC, USUP). Survey results are collected by the OCSC and used to guide future TC placements. These results help the EPP ensure quality student teaching placements for TC with the appropriate levels of support and coaching from the CT. To evaluate the CT, USUP complete a nine item feedback form using a rating scale of 1-9 with one indicating the lowest rating and nine indicating the highest rating. Results from the last four semesters of feedback suggested high levels of satisfaction with the CT, with all USUP ratings on all nine items for all four semesters averaging above 8.3 (R2.2.6 Feedback CT, TC, USUP). USUP indicated they felt welcomed into the CT classrooms; believed the CT had realistic expectations about the TC, shared the classroom in an equitable way, relinquished enough control over the classroom to allow the TC to have independent teaching but still feel supported, was open to differences in personality and/or teaching styles, established a good rapport with the TC, assisted the USUP in addressing concerns about the TC, was a competent and effective teacher, and provided specific feedback to help the TC identify strengths and areas for improvement. Teacher candidates fill out a slightly longer (19 item) form with the same rating scale of 1-9. Data are available for three of the past four semesters (R2.2.6 Feedback CT, TC, USUP); data are not available for Fall 2021 due to an error in revision of the Google form used to distribute the survey. For all items and all semesters, average scores for all CT are above 8 except for a single item (item 13, "My cooperating teacher planned with me on a regular basis") for Spring 2022 which had an average rating of 7.97.

 

TC and CT Feedback on USUPs

Forms are similarly completed by CT and TC to provide feedback about the USUP. Feedback forms for USUP that are completed by CT have seven items on them and the majority of average ratings for all items and all semesters are above 8. On two items, the averages are lower than 8 in some semesters. The item "The university supervisor shared observation notes and feedback with you and the teacher candidate" received average ratings below 8 (6.5 - 7.62) each of the four semesters; and the item "The university supervisor effectively communicated with me concerning the teacher candidate's progress" received average scores lower than 8 (6.9 - 7.85) for three of the four past semesters. The lower trend on these two items may suggest that USUP training should emphasize the importance of sharing observations, notes, and progress with the CT and the TC to provide more transparency as the EPP liaison (R2.2.6 Feedback CT, TC, USUP). TC feedback on USUP provided ratings across nine items. No items had consistently low average scores across all four semesters, but a few averages lower than 8, all in the Fall 2020 semester, may suggest a few TC who were dissatisfied with the relationship they had with their USUP and provided low rankings across multiple items (R2.2.6 Feedback CT, TC, USUP). Feedback from the CT and TC is shared with USUP and used to guide revisions in training.

The UCM EPP works closely with faculty, districts, and TC to implement a diverse range of clinical experiences and co-construct practicum expectations for TC and USUP. All programs, as required by MoDESE, have an introductory early clinical experience (at UCM this is embedded with the introductory education courses), a junior mid-level experience, and a one- or two-semester senior/student teaching experience during senior year. Clinical experiences are intentionally designed in scope and sequence to provide early exposure to diverse classroom settings, various opportunities to observe and practice in different classroom settings, and allow for the authentic assessment of TC readiness for the teaching profession (including dispositional readiness); thus expectations grow progressively more complex and demanding as TC advance through the program. The EPP provides a developmental continuum of clinical experience expectations and outcomes for TC, aligned with InTASC Standards (R1.3.2 Sequence of Clinical Experiences).

 

Early Clinical Experience

For the early clinical placement, TC complete 30 hours in a partner district classroom with the primary focus on observation of the teacher and students in that classroom. The OCSC works with district liaisons or school administrators to secure initial clinical placements and these placements are primarily in seven area school districts. EPP faculty complete an initial dispositions assessment on TC during this early clinical placement and feedback is provided to the TC and program coordinators. During 2021-2022, the EPP has placed 302 TC in the early clinical placement required of all TC prior to admission to the Teacher Education Program. In Fall 2020 and Spring 2021, all early clinical experiences within schools were canceled due to COVID-19 and alternative and virtual observations were used. Thus, FLDX 2150 placement data are reported for fall 2021 and spring 2020 (R2.3.2 Summary of Placements).

 

Mid- and Senior- Level Clinical Experiences

MoDESE requires all Missouri TC to complete at least 45 hours in a classroom as a mid-level clinical experience. However, the UCM EPP requires an additional mid-level clinical experience for all programs and this is usually completed the semester prior to student teaching. The structure of the mid-level and additional clinical experience vary across programs. TC in Secondary and K-12 programs have one mid-level junior clinical experience (30 hours, FLDX 3000) and one senior-level clinical experience (50 hours, FLDX 4970) prior to student teaching. Some programs (Early Childhood Education, Elementary Education, Special Education, and Middle School Education) have more extensive mid-level clinical experience requirements prior to student teaching. For example, TC in Elementary and Early Childhood spend 90 hours in partner district classrooms each semester of the junior year. For the mid-level clinical placements, some programs place their own candidates (such as Art, PE, Middle School, Elementary Education, Early Childhood Education and Special Education) and other programs have candidates placed by the OCSC. Common across all mid-level clinical experiences is a requirement to plan and teach lessons and receive feedback from the CT and the USUP. In addition, a dispositions assessment is completed for each TC in mid-level clinical experiences. Of note, aligning clinical experiences with InTASC Standards using the CAEP-developed continuum provided evidence that the two mid-level clinical experiences completed by secondary and K-12 TC (FLDX 3000 and FLDX 4970) were very similar and did not function to move TC far enough towards being profession ready. The Secondary and K-12 Group has begun to address this and work will continue in the 2022-2023 academic year. Besides personal conferencing and the completion of the EDA for each placement, additional assessment and feedback forms are utilized by different programs. Some examples include those linked here (R2.3.4 Examples Field Documents).

 

Student Teaching

The final semester is a full-time student teaching semester. From Fall 2021 to Spring 2022, UCM placed TC in 54 districts across the state. A variety of assessments are used during the student teaching semester to ensure that candidates are prepared for initial employment in their area of specialization upon completion of the semester. Just as with each preceding clinical experience, a disposition assessment is completed by the USUP and reviewed at the end of the semester (R1.1.5 EDA Template). TC who earn a zero on any disposition indicator during the student teaching semester are not eligible for recommendation for certification. The EPP administers four additional assessments to all TC to monitor individual progress, to provide evidence for use in program evaluation, and to monitor progress and inform decisions at the unit level. The four additional assessments include the Missouri Content Assessment (MOCA) (R1.1.3 MOCA Template), the Missouri Educator Evaluation System (MEES) (R1.1.4 MEES Template), the Student Teacher Work Sample (STWS) (R1.1.6 STWS Template) and Student Teacher Exit Survey (STES) (R1.1.7 ST Exit Survey Template).

 

Diversity of Placements

Because the EPP primarily places TC within seven selected districts for early and mid-level clinical experiences, TC have the opportunity to work with racially, economically, and academically diverse groups of students prior to student teaching. TC also gain experiences in Title I and low performing schools. This planned diversity of placement sites exposes TC to a wide range of demographics in students and communities (R2.3.2 Summary of Placements).

The UCM EPP requires a variety of placements for each TC. Partner districts are categorized based on many aspects of diversity and classified as rural, urban or suburban based on size and population demographics in the schools (R1.1.11 Demographics of Districts). Required categories of clinical placements are tracked in the EPP's integrative student information system, Banner, as well as being listed on the Advising Checklist used by Success Advisors (academic advisors) (R2.3.3Diverse Placements Tracking). For the initial clinical experience, a split placement is provided for Special Education, K-12, Secondary and Middle School TC: half of the hours are completed in a rural placement, and half in an urban placement. A few exceptions occur due to individual student issues and transfer students, but the majority complete the split placement.

Although the EPP is located in a midsized city in rural central Missouri, it is less than one hour from districts that serve an urban population and 30 minutes travel from suburban districts. This makes it logistically possible to place TC in a variety of different types of placements. As an example of the diversity in populations that TC may observe and work with, the Sedalia School District, serves a midsized city (population < 22,000) located in the midst of rural and agricultural areas. The racial breakdown of the district enrollment includes 18.3% Hispanic/Latinx children and 8.1% children who identify as multi-race; both populations exceed the state averages for those subgroups. More than half (52.1%) of Sedalia's school population qualifies for free/reduced lunch; 8.46% are English language learners (ELL) which includes a large population of Spanish-speaking as well as Russian-speaking children from Ukraine; 14.16% of students qualify for special education services; and 11% are homeless, which is the largest proportion of homeless children enrolled in any district that the EPP partners with in the mid to western region of the state. This proportion of homeless children greatly exceeds the average proportion in public school populations across the state (2.4%). Although Sedalia is classified as a rural district (total enrollment 4766), the diversity in race, ethnicity, language, homelessness, and socioeconomic status reflect far more heterogeneity than would normally be expected in this small rural town more than 90 minutes from the Kansas City metro area.

A second example of the diversity in partner districts is Raytown Quality Schools (total enrollment 7829), classified as urban and located on the southeastern edge of Kansas City. Raytown offers a large number of placements for UCM candidates at every clinical experience level and also hires a large number of UCM graduates. The population in Raytown Quality Schools is racially diverse: 50.3% Black, 14% Hispanic/Latinx, 9.1% that identify as multi-race, and 24.5% white. Additional demographics include 66.7% of students eligible for free/reduced lunch, 4.76% ELL, 15.83% qualifying for special education services, 2.25% homeless, and 5.17% gifted. These two districts are just two examples of the diverse P-12 student populations where the UCM EPP regularly collaborates with districts to place TC, to expose TC to the different priorities, challenges, and resources in these districts and also to promote the ability (and desire) of partner districts to hire UCM EPP completers.

The extensive range of district partners of the EPP provides diverse placement sites for TC and supervision from CT, and is a key employment pipeline for completers. According to the 2021 placement report provided by MoDESE, over the past three years among the top employment districts for UCM EPP completers were partner districts where many students gained practical experience through their clinical placements, demonstrating the mutually beneficial relationship between the EPP and its district partners. These completers contribute to the fulfillment of workforce needs of districts across the region and state (R2.3.5 Top Hiring Districts).

 

Use of Technology

On the STES, TC indicate that they feel confident in their use of technology in teaching ("I was prepared to use technology to enhance student learning," mean score across four semesters = 4.48/5) (R1.1.7 ST Exit Survey Template). UCM EPP provides opportunities for TC to utilize various instructional technologies, software, online learning platforms, and apps for educators during the teacher education coursework, providing a foundation of confidence to utilize whatever technology is supported in the district in which they have clinical experiences and become employed. The EPP Technology Committee administered a survey regarding specific technologies used in partner districts; some districts assign one-to-one iPads or Chromebooks; utilize Google Classroom, online course platforms such as Blackboard, Canva, or Schoology; and/or adopt quiz or group response apps, among numerous other possible developmentally appropriate integrations of technology into
every level of education. Prior to the pandemic, TC provided face-to-face instruction with embedded classroom technology during internships. The shift to virtual instruction during the COVID-19 pandemic provided the opportunity for TC to apply their knowledge of digital learning while engaging in high-quality clinical experiences in multiple modalities including hybrid, virtual, and in-person instruction during 2020-2021. However, the virtual modality is not currently a priority recognized by the state to include in teacher education requirements. EPP faculty will examine additional ways to incorporate virtual teaching into the teacher education curriculum for initial certification.

The EPP recognizes that it is essential to create a more diverse pool of educators to fill regional and state needs. To achieve this, the EPP sets admission criteria to admit high quality, diverse candidates, and has developed clear processes and protocols to support candidates as they progress through the program.

The target of this component is that the EPP develops and engages in recruitment activities and events to recruit high-quality candidates from diverse populations and backgrounds. A strategic recruitment plan, aligned with the UCM EPP goals, (R3.1.1 Recruitment Plan) was recently approved by the Teacher Education Council (TEC). The Recruitment Plan includes four goals for recruitment along with strategies and action steps for each.

 

First Goal - Aligning Initiatives

The first goal of the Recruitment Plan is to align recruitment initiatives with the demographics and needs of the regions served by the EPP. The EPP seeks to recruit and retain candidates from multiple pathways to address ongoing teacher shortages (R3.1.2 DESE Shortage Areas). Data from 2021 that project shortages in 2022-2023 list the top five areas of shortage as Mild-Moderate Cross-Categorical K-12 Special Education, Elementary Education 1-6, Early Childhood Special Education B-3, Severe Developmentally Disabled B-12, and Early Childhood B-3. The UCM EPP offers degree programs in all five of those areas and offers alternative routes to certification in four of the five. Only one other Institute of Higher Education (IHE) in the state offers programs leading to certification in all five highest areas of shortage. Of the 43 EPPs in Missouri, UCM is one of 25 that offers Mild/Moderate Cross-Categorical K-12 Special Education; one of 12 that offers Early Childhood B-3 Special Education, and one of two that offers Severe Developmental Disabilities B-12.

While the EPP is consistently one of the top four producers of education program completers in the state and has been a high producer of teachers in several shortage areas, the need to do more is recognized by the various stakeholders. The EPP solicited information about critical areas of shortage in partner districts in Spring 2021. The responses received from the 36 school districts participating (R3.1.3 Advisory Board Shortage Areas) identified a different top 5 areas of shortage than those identified by the state. Secondary Science was identified as an area of shortage by more than 47% of the districts represented; Special Education and Secondary Mathematics were both identified as areas of shortage by more than 41% of the districts; Engineering Technology and English Language Learners rounded out the top five each identified by more than 10% of participating districts. Special Education was in the top five of high need areas identified by both the state and regional partners.

The EPP has created two unique and innovative programs designed to address teacher shortages in Early Childhood, Elementary, and Special Education. The award of state and private foundations grants to the EPP will provide support in the coming academic year to participants in these programs (R3.1.5 MSLF and Braitmayer Proposals). One program recruits current paraprofessionals to complete teaching certification requirements. These paraprofessionals have already gained direct experience working with P-12 students and are well-suited for progressing from the paraprofessional role into the teacher role. This program allows paraprofessionals to continue working full-time while taking coursework in a cohort design and completing a majority of field experience within the building where they are employed. The first cohort of 17 paraprofessionals graduated in Spring 2022 and all are now certified in Cross Categorical Special Education, Elementary Education, or Early Childhood Education. The second program, the Early Childhood Certificate program, includes a set of stackable credentials in Early Childhood Education that provides both professional development and a path to teacher certification for early childhood educators. The first cohort of the Early Childhood Certificate program is currently completing the final certificate and several members of the cohort will be awarded early childhood teaching certification later this year. Thus these two programs, designed to remove barriers to teaching certification, are successfully recruiting candidates into some of the areas in which shortages exist.

The EPP offers minors in Mild/Moderate Cross-Categorical Special Education and Early Childhood Special Education B-3, two areas of teacher shortage. These minors help TC successfully test into those areas of need following initial certification in a different area. The College of Education (COE) Advisory Board members identified teachers certified in English language learning (ELL) as another area of regional teacher shortage and the EPP developed a graduate ELL degree program that leads to teaching certification. Additional EPP actions that align with the first goal of the Recruitment Plan include attending the monthly area principal and superintendent groups that meet on campus to promote collaboration between the EPP and district leadership regarding shortages and job openings.

 

Second Goal - Recruiting Events

The second goal of the Recruitment Plan is to increase the number of TC entering and completing the EPP's teacher preparation programs. In the previous four academic years, the total enrollment of the UCM EPP declined from 1236 to 1044. There are multiple reasons for the decline but targeted recruitment events seek to reverse this trend. A grant from the Missouri Department of Elementary and Secondary Education (MoDESE) (R3.1.4 Grow Your Own Grant) will help to provide resources for recruitment efforts. Planned recruitment events focus on recruiting current high school or community college students as well as future educators from nontraditional populations such as current classroom paraprofessionals, individuals without college credit who work in early childhood settings, or individuals who have a non-education college degree. Two large recruiting events are slated to occur annually; one in fall and one in spring. The Future Teacher Academy (FTA) takes place in the fall semester and is co-sponsored with the Regional Professional Development Center (RPDC) on campus. This event, designed to inspire future teachers, brings high school students from the region to campus to experience time on a college campus. The event includes inspirational guest speakers, a faculty panel, and a current student panel. The inaugural FTA was held during the Summer of 2018 and hosted 5 high school students from 3 districts. The most recent FTA was Fall 2021, when 386 students from 29 school districts attended (R3.1.6 Recruiting Events). The spring recruiting event is the more recently conceived Teach It Forward (TIF) Conference also co-sponsored by the RPDC (R3.1.6 Recruiting Events). This on-campus mini-conference creates an opportunity for area high school students and current education majors to attend various conference sessions together. The Spring 2022 TIF attracted 701 registrants from 55 school districts but unfortunately had to be canceled due to inclement weather. Other recruiting strategies targeted toward high school students include beginning communication with admitted students from the time of admission and and identifying UCM COE Student Ambassadors to give presentations in their home high schools.

 

Third Goal - Alternative Pathways

A third recruiting goal is to increase the number of non-traditional students pursuing educator certification through alternative pathways to certification. UCM offers multiple pathways into the teaching profession. In addition to recruiting current classroom paraprofessionals and individuals without college credit who work in early childhood settings, TC are recruited for post-baccalaureate alternative certification. In this program, TC hold a bachelor's degree in a related field of study, may be employed as a teacher or paraprofessional by a district, and are in the process of completing either a set of courses that lead to certification eligibility or a master's degree coupled with courses that lead to certification eligibility. Alternatively certified individuals historically have been a more diverse group than UCM traditional program TC. Thus, continued recruiting for alternative pathways to certification also contributes to the final recruitment goal: increase the number of candidates from diverse backgrounds entering and completing a teacher preparation program. Missouri's teacher workforce is predominantly white and female; the state plan to address teacher diversity includes outreach to include racially diverse and male candidates into all pathways to teacher certification.

 

Attracting Diversity

As a regional comprehensive university, the UCM student population reflects in large part the demographics of the region. As an institution, UCM has struggled to attract and retain faculty and staff of color as well as students of color. In the EPP the majority enrollment trend for white women has been maintained since the university was founded; thus, it is a priority to identify effective strategies for attracting students of color and male students into the various education programs. These strategies include the implementation of the previously-mentioned paraprofessional and Early Childhood Certificate Program that are designed to be accessible to nontraditional students. The FTA and TIF events have successfully attracted participants from a wide range of high schools that serve populations with diverse demographics. To learn more about how to attract, recruit, enroll, support and retain students of color, a focus group was used to collect information about the experiences of students of color in the UCM EPP (R3.1.7 Focus Group). A consultant with expertise in the area of equitable education facilitated the focus group at the end of the Spring 2022 semester. Open-ended questions included inquiries about the students' experiences in the EPP at UCM including classes, professors, administrators, advisors, peers, and student organizations; field placements; and what the students liked about the program. The final question was about what improvements/changes the EPP could make to improve the student learning experience. Themes in the responses will guide faculty, students, and leadership in developing new strategies for equity and inclusion in the EPP that will also contribute to more effective recruitment of traditionally underrepresented groups.

The EPP uses multiple data points to assess TC performance and readiness to teach as they progress through the program. The quality assurance system employs both proprietary tools and EPP-created instruments to measure candidate proficiency on aligned standards.

 

Transition Points from Admission Through Completion

The EPP has designated four gateways for monitoring and supporting candidate progress in initial licensure programs: 1) Application to the Teacher Education Program, 2) Admission to Teacher Education, 3) Recommendation for Student Teaching, and 4) Completion of Program/Eligibility for Certification. Information regarding gateway timing, requirements and evaluation methods is provided to TC in multiple ways. A graphic of the gateway transition points (R3.2.1 Transition Points Graphic) is included on all education syllabi and is posted on hallway and classroom walls in poster form in the COE building. Success Advisors are a primary source of information especially prior to admission into the teacher education program and so it is crucial that advisors are trained and familiar with the four gateways. Faculty mentors also are trained and provided with resources specific to progress through teacher education (R3.2.2 Information for Advisors). Advisors, the transitions points graphic, faculty members and the Office of Clinical Services and Certification (OCSC) website provide information about assessments and evaluation for each gateway including when and how dispositions are assessed (R1.4.1 Dispositions Implementation). The semester prior to student teaching (gateway 3) teacher candidates must attend a student teaching checklist meeting (R3.2.3 ST Checklist Meeting) during which all requirements and transition points are reviewed.

 

Monitoring Progress from Admission Through Completion

The EPP monitors each TC GPA as they progress through the program. As required by the state, initial licensure candidates must have a GPA of at least 3.0 in both content area and professional education coursework for Admission to Teacher Education and must maintain the minimum GPA throughout the program to be eligible for certification. GPAs are captured at several key gateways including admission to Teacher Education, Recommendation for Student Teaching, and Completion of Program/Eligibility for Certification. Program coordinators and the OCSC monitor teacher candidates at each transition point for cohort and individual progress (R3.2.4 Progression Transition Points). Specific requirements must be met as candidates matriculate through the program, so proficiency data related to various competencies is collected and monitored. At each gateway transition point, candidate content knowledge is monitored via GPA in content courses; at gateway 3 (Application to Student Teach) content knowledge is assessed by candidate completion of the Missouri Content Assessment (MOCA) specific to area of certification. Pedagogical content knowledge and skill proficiency is monitored at gateways 2-4 via GPA in professional education courses and at the final gateway (Completion of Program/Eligibility for Certification) by successful completion of the Student Teacher Work Sample (STWS) and target ratings on the Missouri Educator Evaluation System (MEES). Dispositions are monitored at each gateway (R1.4.1 Dispositions Implementation). Effective integration of technology is introduced throughout coursework, and reinforced and assessed through completion of required technology modules (R1.3.3 Technology Modules). TC provide evidence of their ability to integrate technology in the STWS completed during student teaching (R1.1.6 STWS Template) and the Student Teaching Exit Survey (STES) (R1.1.7 ST Exit Survey Template).

 

Advising Support and other Supports in Place

UCM has a two-tiered advising system comprised of professional Success Advisors in the Success Advising Center (SAC), and faculty mentors specific to each academic program in the institution. Advisors within the SAC provide student support, advising services related to coursework, registration, program progression, and completion. Their primary focus for education majors is to assist with planning general education coursework and coursework up to admission into teacher education, and providing information about requirements for Admission to Teacher Education (R3.2.1 Transition Points Graphic). The Central Degree Audit is the UCM degree audit system and it uses the DegreeWorks software from Ellucian. The Central Degree Audit is a valuable academic planning tool for students and Success Advisors and gives a complete picture regarding what degree requirements have been met, what are in progress, and what are remaining. Faculty mentors often work more closely with candidates once they are admitted to Teacher Education, providing information about required major and minor coursework, professional education courses, the requirements at each gateway, the various required assessments, and career advising. Prior to 2018, each college at the university had its own professional advising center; however, a centralized model was adopted in 2018 and remains in place today. Since centralization, there has been a notable attrition rate among Success Advisors within the first year of employment. This turnover has resulted in advisors having decreased familiarity with programs and problems with consistency of support for education students. To help with this, the Director of the OCSC has created training guides for the Success Advisors and meets regularly with them. One of the EPP's goals is to increase systematic support for monitoring student persistence and success and a needed next step is to strengthen the relationship between Success Advisors and EPP programs (R5.4.1 Progress Goals).

 

Candidates Not Meeting Program Expectations

UCM utilizes the Maxient Early Alert system for faculty and staff to submit concerns they have regarding the academic and personal well-being of individual students. TC and other university officials are notified of performance deficits in their coursework and other concerns such as frequent absence or concerns about mental or behavioral health. In education programs, faculty tend to become familiar with their TC due to small class sizes and supervision of field experiences at the junior and senior levels. Generally, faculty reach out when they become aware that a candidate is struggling either academically or personally. Maxient alerts are processed by the Office of Student Engagement and students are contacted to offer resources or a meeting with that office as deemed necessary. When behavioral issues/dispositional concerns are observed in class or in field experience, program faculty request a meeting with the TC to discuss issues and either develop a professional improvement plan with specific targets for the student to meet (R1.4.1 Dispositions Implementation for information about how each program addresses issues), or to refer the TC to other resources.

Program faculty play a key role in connecting TC with support available on campus. EPP program coordinators were surveyed to determine the supports they connect students with at various points in their program (R3.2.5 Program Supports). These supports include Success Advisors, the OCSC, available scholarships, test preparation resources, and professional organizations, in addition to campus resources such as the Tutoring Center, the Writing Center, the Test Prep Center, the Digital Learning Commons, the Counseling Center, and the Office of Accessibility Services. The goal of providing support is to assist TC who are not meeting program expectations and encourage program progression, as well as to help with overall student well-being.

TC who do not meet the requirements for Admission to Teacher Education (R3.2.1 Transition Points Graphic) and who will not be allowed to continue coursework without admission or who need to take a course during the student teaching semester have the route of appeal to the Teacher Education Council (R3.2.6 TEC Appeals). The majority of appeals are due to insufficient GPA at the time of application for Admission into Teacher Education. Occasionally the obstacle for admission does not surface until the mid-level or final field placement and the OCSC may be made aware of a TC issue by the partner school in which a candidate is placed. Policies and procedures related to removal from a field placement, including suspension or removal from student teaching, are outlined in the Policies and Procedures of the EPP (R3.2.7 Policies and Procedures).

Sometimes even with extensive support candidates are not able to meet program expectations or program matriculation is delayed. An option that advisors, program faculty, or the OCSC may discuss with students to retain them at the university and allow them to graduate, is a degree that does not lead to teacher certification, the Bachelor of Science in Educational Studies and Training (BS EST) (R3.2.8 BS EST Degree). This program allows students to apply educational concepts to traditional or alternative educational or training settings. Students must complete 24 credits in a "training area," as well as an internship to apply their educational learning in a training area. Those who may need to move from a teacher preparation program into this degree program are able to utilize their education coursework and apply it within other training settings. As of Spring 2022, seven teacher education majors have changed to the BS EST. Of these seven, the reasons for the change include one student who did not obtain a sufficient MEES score to become certified, three students who could not obtain a sufficient GPA for Admission to Teacher Education; and three students who decided teaching was not a good fit for them.

Education students who choose the BS EST route as an alternative are generally not able to either be admitted to teacher education or not able to be recommended for student teaching, but many students who apply for Admission to Teacher Education drop out of the EPP for various reasons at different points prior to completion. To identify rate of persistence to completion across the EPP, cohorts are identified based on the first semester a candidate is fully enrolled in UCM coursework and tracked from the time they apply for Admission to the Teacher Education Program. The OCSC maintains a spreadsheet with each candidate and this spreadsheet is updated each semester to monitor progress of candidates (R3.2.4 Progression Transition Points) for a summary of recent cohorts as of May 2022). The majority of the candidates in the Fall 2017, Spring 2018, and Fall 2018 cohorts would normally be expected to have completed their degree at the time of writing this report, as each cohort has been enrolled in coursework for 4-5 years. The pandemic was related to a higher-than-normal rate of attrition for education majors (and for college students in general). However, reasons for non-completion include leaving the university, changing major, needing departmental approval for Admission to Teacher Education, insufficient GPA, or not yet being eligible to student teach. The EPP is working to develop a procedure for regularly reviewing cohort members at each of the four gateways and enacting measures to improve retention of cohort members and encouraging persistence to completion. One measure that has been taken is that the EPP has chosen to create education-specific versions of the freshman introductory/orientation course offered by the university; these courses (EDFL 1400 and 1500) allow TC to meet and begin working with faculty much sooner, and provide support and connection early in the college experience with the goal of engaging candidates even prior to their entry into education courses.

 

Tracking and Resolving Candidate Complaints

The EPP utilizes institution-level reporting mechanisms and resolution policies when candidate complaints and grievances arise. Policies and procedures related to grievances against faculty and staff, personal misconduct, free speech, discrimination, sexual harassment, and grade disputes are outlined in the UCM Student Handbook. As previously mentioned, TC who do not meet the requirements for Admission to Teacher Education (R3.2.1 Transition Points Graphic) and who will not be allowed to continue coursework without admission or who need to take a course during the student teaching semester have the route of appeal to the Teacher Education Council (R3.2.6 TEC Appeals).

A series of requirements monitored at four specific gateways during the teacher education program, along with a series of state-required proprietary and EPP developed assessments aligned to the Missouri Teacher Standards (MTS) (R1.1.1 Missouri Teacher Standards) and InTASC Standards (R1.1.2 Assessment Grid and Alignment) allow the EPP to determine TC overall proficiency and readiness to enter the classroom upon completion of the program. Data points include content and professional education GPAs (minimum of 3.0 for each), Educator Dispositions Assessment (EDA) at multiple levels, completion of STWS, satisfactory ratings on the MEES, and Missouri Content Assessment (MOCA) data. The OCSC verifies and approves all initial candidate recommendations for licensure. The UCM EPP also conducts the STES and MoDESE administers one-year follow-up surveys for all education program completers who became certified and employed in Missouri public
schools and also survey their principals.

 

Disaggregated Completion Data

MoDESE requires an annual performance report (APR) to be submitted (R.1.2.3APR) which satisfies the state's requirements for continued state accreditation. Data for the APR include GPA, MEES ratings, MOCA scores, and follow-up survey ratings from first-year teachers and their principals; these data represent many of the key completion data collected by the EPP. Data collected from these key assessments along with the various other required assessments identified by the EPP are used to determine TC readiness to teach effectively upon completion of the program and are also used to inform recruitment and retention efforts, identify needed candidate supports, and suggest curriculum revisions and programmatic changes. Data are reviewed regularly by program faculty, professional education faculty, advisory boards, and the Teacher Education Council (R5.1.2 CI and Unit Data Cycles).

The EPP's use of multiple measures allows for the triangulation of data and evidence of candidate performance. Analysis of data from assessment items from key assessments aligned to each of the components of Revised Standard 1, Content and Pedagogical Knowledge, indicate EPP candidates meet this standard and possess the knowledge, skills, and dispositions required of beginning educators (R1.1.8 Data For Component 1.1, R1.2.1 Data for Component R1.2, R1.3.1 Data for Component 1.3, R1.4.2 Data for 1.4). Data analysis from the required technology modules (R1.3.3 Technology Modules) as well as from the STWS, STES, and state-administered surveys of first year teachers and their principals indicates that UCM EPP candidates are prepared to use instructional technology to enhance student learning.

Missouri Educator Evaluation System (MEES).

Means on all nine MEES standards (N=299) were statistically significantly higher than the target level of 3. An independent samples t-test was conducted to determine if there were differences in MEES ratings based on gender. On average female students had slightly higher ratings across each rater, but here was no statistically significant difference in total MEES scores for female or male teacher candidates nor in mean total scores between white and non-white candidates by USUP and CT raters. However, there was a statistically significant difference in MEES mean scores on candidates' self-assessments between white and non-white candidates. Non-white candidates' self-assessed MEES total score mean (30.45) was higher than white students' self-assessment MEES total score mean (29.70). Finally, when looking at the means across cycles, there is a dip in means at the second cycle (Fall 2020) across all standards but 7, then an increase across all standards. There were no statistically significant differences in ratings between traditional and alternative program candidates. There was a statistically significant difference between Fall 2020 and Fall 2021 total MEES mean scores, as Fall 2021 TC were scored higher than Fall 2020 on the total MEES mean, which may be attributed to shifts to all-online learning and teaching in spring 2020 due to COVID-19 in both university and public school settings (R1.1.9 Analysis of Unitwide Data, R1.1.4 MEES Template).

Missouri Content Assessment (MOCA).

For the four assessment cycles Fall 2020-Spring 2022, pass rates of the MOCA for all completers in the EPP ranged from 82.4-90.1% by the end of student teaching; after completion of student teaching pass rates increased to 88.1 - 97.0%. The mean number of attempts across all programs ranged from 1.20 - 1.34. No statistically significant differences in pass rates nor average scores were discovered when scores were disaggregated by race, gender, or nature of program (R1.1.3 MOCA Template).

Educator Disposition Assessment (EDA).

For the four assessment cycles Fall 2020-Spring 2022, all average ratings exceeded the satisfactory level of 1. No statistically significant differences in average ratings were discovered when disaggregated by race, gender, nature of program, or first-generation status (R1.1.5 EDA Template).

Student Teacher Work Sample (STWS).

For the four assessment cycles Fall 2020-Spring 2022, all average ratings exceeded the required level per component. No statistically significant differences in average ratings on any component were discovered when disaggregated by race, gender, nature of program, or first-generation status (R1.1.6 STWS Template).

For the MEES, MOCA, EDA, and STWS, small n sizes for many specific content-area programs make relevant statistical comparison across programs untenable (R1.1.3 MOCA Template, R1.1.4MEES Template, R1.1.5 EDA Template, R1.1.6 STWS Template).

Student Teaching Exit Survey (STES).

For the four assessment cycles Fall 2020-Spring 2022, the majority of average ratings exceeded 4 on a scale of 5. Some items consistently averaged below a rating of 4. These items were (Standard 2) "I was prepared to modify instruction for English language learners" and "I was prepared to implement instruction based on a student's IEP;" (Standard 5) "I was prepared to manage a variety of discipline issues;" and (Standard 6) "I was prepared to effectively communicate with parents." Data were not disaggregated by nature of program, certification area, gender, or race because no identifying information is required on the STES (R1.1.7 ST Exit Survey Template).

Teachers prepared by the University of Central Missouri's (UCM) Educator Preparation Program (EPP) consistently demonstrate a positive impact on P-12 student learning and development. In Missouri, P-12 student data is not readily accessible to an EPP. Therefore, the EPP uses state-provided completer and employer satisfaction survey data as well as a case study approach to demonstrate that the EPP successfully prepares effective teachers for today's diverse classrooms. Additional evidence of employer satisfaction is provided by continued high rates of job placement inferring employer satisfaction.

Context

From the earliest clinical experience placement, the EPP works to ensure that completers are prepared to effectively contribute to the learning and development of diverse P-12 students. Geographically and demographically diverse placements are intentionally designed and sequenced to provide early exposure to diverse classroom settings, and opportunities to observe and practice in various classroom settings. Practica experiences also allow the opportunity for performance assessment of candidates' readiness for the teaching profession. In many instances, candidates are later employed in districts where they participated in field experiences or in districts that are demographically similar to the field experience placement settings (R2.3.5 Top Hiring Districts). Of the top seven districts that hire the most UCM completers, six are partner districts where the EPP regularly places candidates at one or more levels of clinical experience (R2.3.2 Summary of Placements).

 

Program Impact Case Study

The EPP has implemented a Program Impact Case Study (PICS) to determine the impact of its completers on the learning of students in P-12 settings. Phase 1 of PICS occurred in Spring 2021 with the pilot of PICS (The pilot was scheduled to begin the previous year but COVID restrictions prohibited data collection.) Phase 2 of PICS occurred in Spring 2022. PICS is designed to demonstrate the impact of the UCM EPP by seeking to answer two questions: (1) Are UCM completers/graduates having a positive impact on student learning? and (2) To what extent do completers demonstrate teaching effectiveness that UCM preparation was designed to achieve?

Research Question 1: Are UCM completers/graduates having a positive impact on student learning?

Phase 1 of PICS, completed in Spring 2021, included randomly selected first and second-year teachers from each of the following certification areas: Early Childhood, Elementary Education, Middle School and Secondary Education. The completers were selected from a large suburban school district with whom the UCM EPP has a strong partnership. During Phase 1, each completer was observed and assessed by an EPP faculty member using the Missouri Educator Evaluation System (MEES), a valid and reliable measure developed by the Missouri Department of Elementary and Secondary Education (MoDESE) and used to assess teacher candidates and practicing classroom teachers across the state at all career levels. In addition, the completers provided pre-assessment and post-assessment data on an instructional unit they had taught and also participated in a post-observation interview (R4.1.1 Case Study Handbook).

For Phase 2 of PICS, completed in Spring 2022, the same measures were utilized (observation using the MEES, collection of pre- and post- data, and post-observation interview). Additionally, data were collected to show P-12 students' perception of learning using the Student Perception Survey (SPS), a reliable and valid instrument developed by the Colorado Education Institute to measure elements of student experience demonstrated to correlate most closely to a teacher's ability to positively impact student growth. For the 34 items of the SPS, students are asked to indicate how frequently they experience each item, with a response scale of always, most of the time, some of the time, and never. Items of the SPS load onto four themes. The P-12 students of the EPP completers participating in PICS Phase 2 indicated the EPP completers were having a positive impact on student learning. For the first theme, more than 68% of the students indicated that the EPP completers "effectively used content and pedagogical knowledge to help students learn, understand, and improve" all or most of the time. For the second theme, more than 82% of students indicated that the EPP completers "effectively created an environment that responds to individual students' backgrounds, strengths, and interests" all or most of the time. For the third theme, more than 60% of students indicated the EPP completers "cultivated a classroom learning community where student differences are valued" all or most of the time. For the fourth theme, more than 60% of the students indicated the EPP completers "foster a respectful and predictable learning environment" (R4.1.2 Case Study Report).

The EPP completers participating in Phase 2 of PICS were interviewed by EPP faculty regarding their experiences at UCM and how they translated those experiences into their current classroom teaching. Across the interview questions, completers consistently indicated that the plan of study in their individual programs prepared them for classroom instruction. Classroom management emerged as a topic covered in each program that was important to the success of their classroom. Working with parents and students with diverse needs were identified as gaps in their preparation program (R4.1.2 Case Study Report).

Research Question 2: To what extent do completers demonstrate teaching effectiveness that UCM preparation was designed to achieve?

For both Phase 1 and Phase 2 of PICS, an EPP faculty member observed participating completers and used the MEES to gather data on the teaching effectiveness the completers displayed. As described in R1, the MEES is based on the Missouri Teaching Standards (MTS) (R1.1.1 Missouri Teacher Standards) and includes indicators for each of the nine MTS. For PICS, specific standards and indicators were selected to evaluate completers including:

• Standard 1, Indicator 2: Student engagement in subject matter
• Standard 4, Indicator 1: Instructional strategies leading to student engagement in problem-solving and critical thinking
• Standard 4, Indicator 2: Appropriate use of instructional resources to enhance student learning
• Standard 5, Indicator 1: Classroom management techniques
• Standard 6, Indicator 1: Verbal and non-verbal communication

Data collected during both phases of PICS reveal UCM EPP graduates are ready for classroom teaching, as evidenced by student achievement, observation, and perceptual/anecdotal reflection (R4.1.2 Case Study Report). For each classroom that submitted pre- and post-assessment scores, there was growth in student achievement. EPP completers also reported a sense of efficacy in being ready to teach. This is important as each of the teachers in this particular PICS were confronted with the COVID-19 challenge of teaching virtually and in person. While their school year was far from normal, they each stated their preparation gave them the foundation they needed (R4.1.2 Case Study Report).

In the future, the EPP plans a three-pronged approach to studying the impact of EPP completers on P-12 student learning and development. First, the current PICS will be reviewed and revised prior to implementation during the 2022-2023 academic year. The EPP has experienced some difficulty in recruiting completers to participate in PICS and believe this is likely due to the additional stress resulting from COVID. A careful review of the current process will allow the EPP to make any needed changes that will result in greater participation. Second, an additional case study is currently being piloted to provide a more longitudinal look at the impact of completers who will be followed from their senior practicum through student teaching and their first and second years as teachers. Finally, expanded focus groups will be used to provide additional data from a wider group of completers (R4.1.3 Transition Plan and Pilot).

The EPP demonstrates employers are satisfied with the completers' preparation for their assigned responsibilities in working with diverse P-12 students and their families. Beginning in 2007, DESE in collaboration with the University of Missouri's Office of Social and Economic Data Analysis (OSEDA), began administering an annual survey of first-year teachers (First Year Teacher Survey) in Missouri public schools, along with a companion survey of supervisors of first-year teachers, typically the principal of the school building where the teacher spent the greatest proportion of their time (Principal Survey). The surveys are administered in late spring so that the first-year teachers have nearly completed their first year of employment. MoDESE reviews and modifies the questions periodically and the survey has been updated numerous times since its pilot in 15 years ago. School district data files submitted to MoDESE are used extensively in identification of first-year teachers and their principals (RA4.1.1 Technical Manual, RA4.1.2 Technical Manual Updates). The First-Year Teacher Survey and Principal Survey data are compiled annually and shared with EPPs, and the UCM EPP's data are publicly available on the UCM website.

 

First-Year Teacher Survey and the Principal Survey

The First-Year Teacher Survey and the Principal Survey each consist of 38 survey items, aligned to the MTS. Items on the Principal Survey mirror those on the First-Year Teacher Survey. For example, one item on the First-Year Teacher Survey is "I was prepared to incorporate interdisciplinary instruction" and the corresponding item on the Principal Survey is "The teacher was prepared to incorporate interdisciplinary instruction." Each item is rated on a 5-point Likert scale that ranges from 1 - Strongly Disagree to 5 - Strongly Agree. Three consecutive cycles of data are provided from the First-Year Teacher and Principal Survey (2018-19, 2019-20, 2020-21) (R4.2.1 First Year Teacher-Principal Survey Data) as evidence of completer and employer satisfaction.

Specific data from the Principal Survey for ratings on items aligned to each of the MTS follow (R4.2.1 First Year Teacher-Principal Survey Data).

• Standard 1- Content Knowledge: On the four items for this standard, across the three data cycles, the ratings ranged from 3.84 to 4.32 and only one of the 12 ratings was below 4. These data indicate that principals agreed from a moderate to a strong level that first-year teachers from UCM were prepared in the content area to incorporate interdisciplinary instruction, to engage students in the content area, and to make content meaningful to students.
• Standard 2 - Learning, Growth, and Development: The average ratings for the five items for this standard were lower than for Standard 1 with ratings ranging from 3.48 to 4.12. Therefore, principals moderately agreed that first-year teachers from the EPP were prepared to design lessons that include differentiated instruction, implement instruction based on a student's IEP, modify instruction for English language learners, modify instruction for gifted learners, and create lesson plans to engage all learners.
• Standard 3 - Curriculum Implementation: The average ratings for the two items aligned with Standard 3 ranged from 3.89 to 4.23 across the three data cycles. Principals agreed to a moderately strong level that first-year teachers from UCM were prepared to deliver lessons based on curriculum standards and deliver lessons for diverse learners. One item addressed new teachers' preparation to use technology to enhance learning (not aligned with a specific standard) and ratings ranged from 4.12 to 4.18 indicating moderately strong agreement from principals that UCM EPP completers were well prepared in this area.
• Standard 4-Critical Thinking: Principals concurred that first-year teachers from UCM EPP were prepared to implement a variety of instructional strategies, engage students in critical thinking, and model critical thinking and problem solving as is evidenced by average ratings ranging from 3.97 to 4.10.
• Standard 5- Positive Classroom Environment: Seven items align to this standard with average ratings across the three data cycles ranging from 3.78 to 4.41. Thus, principals agree to a moderately strong level that first-year teachers from UCM are prepared to create a classroom environment that encourages student engagement, use a variety of classroom management strategies, manage a variety of discipline issues, motivate students to learn, keep students on task, foster positive student relationships, and facilitate smooth transitions for students.
• Standard 6 - Effective Communication: Ratings from principals on the six items for this standard ranged from 4.05 to 4.30, with all ratings exceeding 4; some of the most consistently strong ratings were found for this standard which indicates principals feel first-year teachers from the EPP were well-prepared to use effective communication strategies to foster learning, effectively communicate with parents and staff, promote respect for diverse cultures, genders, and intellectual/physical abilities; use technology as a communication tool, and enhance students' skills in using technology as a communication tool.
• Standard 7 - Student Assessment and Data Analysis: Average ratings from principals on the five items for this standard ranged from 3.78 to 4.18 indicating moderately strong agreement that UCM EPP prepares completers to use assessments to evaluate learning, develop assessments to evaluate learning, analyze assessment data to improve instruction, help students set learning goals based on assessment results, and work with colleagues to set learning goals using assessment results.
• Standard 8 - Professionalism: This standard is assessed through ratings on two items and average principal ratings range from 3.92 to 4.22 indicating moderately strong agreement that first-year teachers prepared at UCM EPP were prepared to analyze data to reflect on areas for professional growth and reflect on practices for professional growth.
• Standard 9 - Professional Collaboration: Average scores ranging from 3.97 to 4.31 on the three items for this standard indicate that principals agree to a moderately strong level that these first-year teachers are prepared to collaborate with colleagues and parents to support students' learning, and participate in professional organizations.

One item on the survey asks principals, "Which best reflects your perspective about the overall quality of the professional education program your teacher completed?" using a scale ranging from "1-Very Poor" to "5-Very Good." For each of the three data cycles, the rating averages were 4.16, 4.23, and 4.22, indicating a general consensus among principals that the UCM EPP is "Good" to above the level of "Good." A three-year comparison of ratings by principals of UCM completers to ratings by principals of all Missouri first-year teachers showed that almost 90% of the time, ratings of first-year teachers prepared at UCM were higher than ratings of all first-year teachers across Missouri. In the 2018-2019 academic year, 87% of principal average ratings for UCM-prepared first-year teachers were higher than ratings from principals for all first-year Missouri-prepared teachers in the state and in both 2019-2020 and 2020-2021 more than 92% of principal ratings for UCM-prepared first-year teachers were higher than ratings for all other Missouri-prepared teachers across the state.

Of note, items for which the EPP completer rating was lower than the overall state rating were mostly those measuring Standard 2 - Learning, Growth, and Development; specifically in the areas of designing lessons with differentiated instruction, implementing instruction based on a student's IEP, modifying instruction for English language learners, and modifying instruction for gifted learners. These average ratings were all higher than 3.4 which is satisfactory, but those ratings that are lower than the state average are worth noting as an area for improvement.

The EPP demonstrates program completers perceive their preparation as relevant to the responsibilities they encounter on the job, and their preparation was effective. The UCM EPP regularly examines results of the MoDESE administered First-Year Teacher Survey (described in the previous section) to assess the satisfaction of completers with the preparation program near the end of the first-year of employment. Data from this survey are also part of the Annual Performance Report (APR) providing a weighted score for each standard, and the UCM EPP has consistently met required scores for each program to maintain state accreditation (R1.2.3 APR).

Specific data from the First-year Teacher Survey for ratings on items aligned to each of the MTS follow (R4.2.1 First Year Teacher-Principal Survey Data).

• Standard 1- Content Knowledge: On the four items for this standard, across the three data cycles, the ratings ranged from 4.01 to 4.3 and all ratings were above 4. In addition, on all items for this standard, EPP completers' ratings were higher than the state average. These data indicate that completers feel confident in the content knowledge needed to be effective teachers.
• Standard 2 - Learning, Growth, and Development: As was true for the Principal Survey, completers; ratings for items in Standard 2 were lower than for Standard 1 with ratings ranging from 3.42 to 4.28. EPP completer' ratings were higher than the state average on all but the item, "I was prepared to implement instruction based on a student's IEP."
• Standard 3 - Curriculum Implementation: The EPP completers' ratings were high for Standard 3 with scores ranging from 4.08 to 4.36. All EPP completer ratings were above 4 and all exceed the state average rating.
• Standard 4-Critical Thinking: All EPP ratings for this standard (ranging from 4.21 to 4.42) were above 4 and all were higher than the state average. While all ratings for the stand alone technology item were above 4 and above the state average, the EPP completer rating in this area has declined over the three data cycles.
• Standard 5- Positive Classroom Environment: Ratings on the seven items in this standard ranged from 3.64 to 4.58 and exceeded the state average on all items. The lowest rating was consistently for the item "I was prepared to manage a variety of discipline issues" and the highest ratings were for the item "I was prepared to foster positive student relationships."
• Standard 6 - Effective Communication: Ratings from items on this standard present some concern for the EPP. Ratings ranged from 3.58 to 4.44 with the lowest rated item consistently being "I was prepared to effectively communicate with parents." Of concern, in the last cycle (2020-2021) ratings on five of the six items for this standard were below the state average.
• Standard 7 - Student Assessment and Data Analysis: Average ratings from EPP completers for Standard 7 items ranged from 3.96 to 4.38 with all ratings higher than the state average.
• Standard 8 - Professionalism: Both items for this standard consistently were rated above 4. In one cycle, one item was below the state average.
• Standard 9 - Professional Collaboration: Ratings for the three items in this standard ranged from 3.75 to 4.36. The lowest rated item for both EPP completers and for all Missouri completers was for the item "I was prepared to collaborate with parents to support student learning."

On the First-Year Teacher Survey, completers across Missouri are asked to provide a response to "Which best reflects your perspective about the overall quality of the professional education program you completed" by selecting from a 5-point scale of "1-Very Poor" to "5-Very Good." For all three cycles of data, EPP completers' ratings exceeded the state average and ranged from 4.30 to 4.37.

Because items on the Student Teaching Exit Survey (STES) (R1.1.7 ST Exit Survey Template) mirror those of the First-Year Teacher Survey, the EPP is able to compare completer ratings immediately upon completion of the program with those provided near the end of their first year of employment as teachers (R4.3.1 Comparison Completer and First Year). As expected, ratings from completers immediately upon completion were higher on each item than those of first-year teachers. Until completers are fully responsible for their own classrooms, they may not completely realize the need for preparedness for all situations teachers will face. However, some commonalities of the two data sets are apparent. First, both immediate completers and first-year teachers consistently feel less prepared on some Standard 2 - Learning, Growth, and Development items; specifically, preparation to implement instruction based on a student's IEP, modifying instruction for English language learners, and modifying instruction for gifted learners. One item in Standard 5, "I was prepared to manage a variety of discipline issues" and one item in Standard 6, "I was prepared to effectively communicate with parents" were also commonly rated lower by both immediate completers and first-year teachers. In addition, the principals of the EPP completers rated these same items lower than other items.

An additional pilot effort at a small focus group analysis was also attempted during Spring 2022. However, the number of participants was low and not sufficiently representative. Some of the findings that emerged, although outliers from the EPP's other findings, are concerns that the EPP hopes to more thoroughly examine in a future larger focus group assessment effort (R4.1.3 Transition Plan and Pilot).

 

Looking to the Future

While the EPP believes the data provided through PICS and state survey data provides strong evidence that EPP completers are well-prepared for the classroom, the trends noted previously provide clear guidance on steps for moving forward.

EPP candidates are introduced to differentiation that includes providing instruction based on a student's IEP and modifying instruction for English language learners and gifted students. Collaborating and communicating with parents is also part of coursework. However, the true importance of the need for proficiency in these areas becomes apparent when candidates are in the classroom full-time during student teaching and this is reflected in data from the STES. The EPP is planning to incorporate additional required seminars during the student teaching semester to address differentiation, specifically for special education, ELL, and gifted students. Working with parents will be a topic of another seminar planned for the student teaching semester (R2.2.7 Data Based Decisions).

EPP completers and their employers also consistently mention managing behavior as an area in which they feel less prepared. While all candidates complete a classroom management class, as is true with differentiation, the need to focus on behavior management becomes less abstract during student teaching. The EPP has formed a workgroup to address trauma informed practices, social-emotional learning, and behavior management. This group is currently evaluating the current curriculum to identify any gaps in fully preparing our candidates. In the immediate future, professional development is being included during student teaching to provide support for candidates (R2.2.7 Data Based Decisions).

Finally, the EPP notes the decline in completer confidence in their preparation for using technology to enhance learning. While the pandemic may have impacted this, the need for teachers to be proficient with technology is critical. As described in the EPP transition plan, the technology modules required prior to student teaching are being revised to better reflect the needs identified on the partner technology survey (R1.3.4 Transition Plans, R5.1.7 Partner Tech Survey Results).

As mentioned previously, the EPP plans a three-pronged approach to studying the impact of EPP completers on P-12 student learning and development. First, the current PICS will be reviewed and revised prior to implementation during the 2022-2023 academic year. Second, an additional case study is currently being piloted to provide a more longitudinal look at the impact of completers. Finally, expanded focus groups will be used to provide additional data from a wider group of completers (R4.1.3 Transition Plan and Pilot).

The Quality Assurance System (QAS) developed and utilized by the Education Preparation Program (EPP) at the University of Central Missouri (UCM) provides a sustainable process to review and document operational effectiveness. Because the EPP continues to respond to changes (at the institutional, local, regional, state, and national levels) and to feedback (from completers, stakeholders, and partners), the QAS continues to evolve to address these changes and to ensure continuous improvement. The EPP's QAS includes both initial and advanced programs and this is reflected in the similarities in the Standard 5 narratives for each.

The UCM EPP's QAS is a model for assessment that addresses two specific reasons the EPP conducts assessment. First, the EPP conducts program area level assessment to determine teacher candidate (TC) achievement of the specific learning outcomes for their area of study. Data collected provides evidence that guide changes at the programmatic level. Second, the EPP conducts unit-wide assessment to ensure TC are developing the knowledge, skills, and dispositions required of all educators. Unitwide data collected provide the evidence required to guide changes at the unit level. Both program area and unit assessment provide evidence to meet the needs of accrediting bodies such as CAEP and Higher Learning Commission (HLC).

 

College Level

The QAS is monitored by the Quality Assurance Workgroup (QAW), a group with membership representing both initial and advanced programs. The Dean of the College of Education (COE) is a member of this group as is the chair of the College of Education Assessment Committee (COEAC). The QAW meets regularly to ensure the QAS is functioning to promote continuous improvement (R5.1.1 Selected Minutes QAW) The QAW led the development of the Continuous Improvement Cycle (CIC) that forms the basis of program area and unit level assessment (R5.1.2 CI and Unit Data Cycles). The CIC reflects the importance of using data as a basis for change at the program area and unit levels, and the need for this process to be ongoing, systematic, and continuous. After careful review by the Professional Education Faculty (PEF) and other stakeholders, this CIC was adopted by the EPP's Teacher Education Council (TEC) in fall 2021.

The QAS includes two parallel and closely related data cycles with responsibility for oversight falling to two groups. Unit level assessment and operations are monitored by the QAW. The QAW reviews unit processes related to the QAS, oversees the administration of unit assessments, and oversees data collection, analysis, and documentation at the unit level. In conjunction with the Dean's Office, the QAW plans and conducts two meetings of the PEF each year. Program level assessment is monitored by the COEAC with one member from each of the COE departments. The Coordinator of College of Education Assessment is a member of this committee. This committee reviews and provides feedback on program area assessment and provides professional development to improve assessment plans and processes.

 

Unit Level

At the unit level, the data cycle is an ongoing process overseen by the QAW with multiple levels of review and reporting (R5.1.2 CI and Unit Data Cycles). Data on unit assessments (Missouri Education Evaluation System [MEES], Missouri Content Assessment [MOCA], Educator Disposition Assessment [EDA], Student Teacher Work Sample [STWS] and Student Teacher Exit Survey [STES]) are collected in the fall and spring semesters. The COE Coordinator of Data Management and Technology (CDMT) and the QAW compile unit-wide data for the academic year and share the data with the COE Advisory Board, the PEF, and the Secondary and K-12 Group. The CDMT is a newly created position to provide technical support and expertise in data collection, analysis, and management. The CDMT disaggregates the data by program area and shares program data with department chairs and the appropriate program coordinators. The CDMT provides the QAW with unit-wide data disaggregated by race, gender, first generation status, program type, and certification area. After receiving feedback from the COE Advisory Board (R2.1.6 COE Advisory Board Minutes) and the PEF (R5.1.3 PEF Minutes), the QAW prepares an executive summary (R5.1.4 Executive Summary) of the academic year data and shares this with the COE Advisory Board, PEF, and TEC. The TEC uses this summary to provide data-based recommendations to the Dean of the COE, the head of the EPP unit.

 

Program Level

At the program level, the ongoing process reflected in the data cycle is overseen by the COEAC and includes levels of review and reporting (R5.1.5 Program Data Cycle and Schedule). Data on assessments are collected in the fall and spring semesters. The CDMT shares this data disaggregated by program, gender, race, first generation status, and program type with the program coordinators. Simultaneously, the program coordinators collect program specific assessment data. The program coordinators compile unit assessment and program specific assessment data and share the data with program faculty and advisory board members. Using input from these groups, the program coordinators enter a program report into the UCM assessment management system (Nuventive). These reports are entered annually and the report and data are accessible to program faculty, unit administrators, and university administrators for review. A formal review of program assessment reports is conducted by the COEAC on a three year cycle (R5.1.5 Program Data Cycle and Schedule).The COEAC provides detailed feedback to the program coordinator and this is shared with program faculty.

The data collected from the EPP's unit assessments are an integral part of the QAS (R1.1.3 MOCA Template, R1.1.4 MEES Template, R1.1.5 EDA Template, R1.1.6 STWS Template, R1.1.7 ST Exit Survey Template). These assessments are used unit-wide, are aligned with InTASC and Missouri Teacher Standards (MTS) (R1.1.1 Missouri Teacher Standards, R1.1.2 Assessment Grid and Alignment), and were selected to provide a complete picture of the knowledge, skills, and dispositions of the EPP's TC. Aggregate and disaggregated data from these assessments provide the basis for continual review and improvement of unit and program level processes, and for decision making at the unit and program levels.

EPP program faculty play an important role in the quality assurance process and shared beliefs about the importance of assessment is key to continuous improvement. The COEAC has worked to create a mindset of valuing assessment and using data as part of the process of continual improvement. The committee distributed a Culture of Assessment survey to all PEF members and the results of the survey are being used to plan professional development for the upcoming year (R5.1.6 Culture of Assessment Survey).

UCM's EPP works to ensure all interpretations of data are valid through use of reliable and valid instruments. Multiple assessments are utilized to allow the triangulation of data points to confirm findings. The assessments include both proprietary instruments required by the state and EPP adopted or developed measures. For each of the InTASC Standards and each of the MTS, multiple assessments are used to gather data on the knowledge, skills, and dispositions of candidates (R1.1.2 Assessment Grid and Alignment).

 

Relevant

The InTASC Standards provide a description of the essential knowledge, performances, and critical dispositions required for effective teaching that supports learning and learner development. The measures used as part of the EPP's QAS were selected to assess the knowledge, skills, and dispositions described in the InTASC Standards and are aligned to these standards (R1.1.2 Assessment Grid and Alignment). In addition, the EPP's unit assessments are aligned to the MTS, a set of standards the state of Missouri uses to assess teachers at all levels in their professional careers - from teacher candidates to experienced educators (R1.1.1 Missouri Teacher Standards). Because of the careful alignment to InTASC and MTS, the EPP ensures there is a clear link between what is measured on each assessment and what the EPP intends to measure. The QAS provides for the ongoing review of assessments and the data they produce, and the EPP uses this review to make revisions to assessments. For example, the STWS has been revised to provide needed data regarding the TC proficiency with instructional technology. In addition, TC are required to complete modules on the use of technology and these are also being revised based on partner feedback and TC needs (R1.3.4 Transition Plans, R5.1.7 Partner Tech Survey Results).

 

Verifiable

Valid and reliable measures are essential to ensure quality. The EPP uses a mixture of propriety and EPP created assessments (R1.1.2 Assessment Grid and Alignment). The proprietary measures used by the EPP, the MOCA and the MEES, are required by the state of Missouri and work at the state level provides evidence of validity and reliability. The EDA was adopted by the EPP as a measure of the professional dispositions of TC and is administered at several points in the program. The EDA was originally developed at the University of Tampa and the EDA Technical Manual provides evidence of validity and reliability (R5.2.1 EDA Technical Manual, R1.1.5 EDA Template). The STWS is an EPP created comprehensive unit-wide performance assessment used to evaluate immediately prior to program completion TC ability to design, implement, and assess instruction and to reflect on teaching and learning processes. The STWS has undergone revisions and the current version has been used for two semesters (Fall 2021 and Spring 2022) (R1.1.6 STWS Template). An initial investigation of the reliability and validity of the STWS was conducted in Spring 2022 (R1.1.9 Analysis of Unitwide Data) The reliability of the STWS (initial and revised version) was assessed using a reliability analysis in SPSS to render a Cronbach alpha reliability statistic. Reliability coefficient for the 11 item initial STWS scale used in the Fall 2020 and Spring 2021 (n=213) was .545. The revised version of the revised STWS for Fall 2021, which included 7 additional scale items for a total of 18 yielded a reliability coefficient of .756. The QAW applied the CAEP criteria for EPP-created assessments to the STWS and used the findings to make recommendations for revision and additional reliability analysis (R5.2.2 Sufficiency Criteria STWS STES). The STES is an EPP-created survey designed to assess the perception of completers regarding areas of their preparation for teaching. The items used on the STES mirror those on the state administered survey of first year teachers. The Technical Manual for this state-administered survey (RA4.1.1 Technical Manual) provides a description of the development of the survey as well as information about validity and reliability. The QAW applied the CAEP criteria for EPP Created Surveys to the STES with no recommendations for changes to content being made (R5.2.2 Sufficiency Criteria STWS STES).

The EPP also takes steps to increase inter-rater reliability for those completing unit assessments. For example, training for USUP on the MEES is held each semester. USUP review the MEES rubric, view a video of teaching, provide individual ratings on the MEES, work in small groups to arrive at a group consensus on a rating, and then view and discuss the state assigned rating for the teaching portrayed in the video (R2.2.4 MEES Training). Similarly, annual training for the EDA is held for all those assessing TC dispositions. Raters assess the dispositions displayed in a scenario individually and then as a small group. The group raters are then compared to those assigned by the developers of the EDA (R2.2.5 EDA Training).

 

Representative

The data collected on unit assessments are representative of all TC and programs. These assessments are administered to all TC regardless of program and this ensures analysis of the data encompasses all programs. While the MOCA is specific to a content area, every TC must pass the appropriate MOCA for certification. For all other unit assessments, all TC complete the same version of the assessment.

 

Cumulative

Because unit assessments are administered every semester, the EPP is able to view and analyze data across successive administrations. Unit-wide data is compiled each semester and shared with the PEF at the completion of the academic year. Data from previous semesters is included in the data document so that all PEF members are able to view the data across multiple semesters (R5.2.3 Unit Data for PEF). Because the STWS has been revised recently, two cycles of data are available for the STWS in its original form and two cycles for the revised version.

 

Actionable

The data generated through the administration of unit assessments provide a foundation for EPP decision making (R2.2.7 Data Based Decisions). For example, data gathered from the MEES, STWS, and STES, as well as data from the state administered survey of first year teachers and their principals, indicate TC feel less prepared in behavior management, trauma informed practices, communicating with parents, and working with special education students. An analysis of this data led the EPP to form a task force to plan for addressing these concerns. One result has been the creation of student teaching seminars to address some of these topics. While TC are introduced to these topics early in their programs, the need to be fully informed and confident becomes less abstract and more relevant when TC are engaged in the classroom full-time during student teaching.

Additionally, data gathered from the STES and EDA provide evidence that TC feel fairly well-prepared, comfortable, and confident in understanding many aspects of diversity and effectively working with some subgroups, but not equally confident across all subgroups of P-12 learners. An Access, Opportunity and Community Workgroup was formed to provide guidance to the unit. The group created a statement on diversity and inclusion that was adopted by the TEC in Spring 2022. With program coordinator input, a scope and sequence of concepts related to diversity was created to help the unit identify gaps (R1.1.10 Diversity Workgroup). The group also recommended professional development be offered to faculty and students.

The UCM EPP engages both internal and external stakeholders in program design, evaluation, and continuous improvement processes. This is accomplished internally through the work of groups such as the PEF, the Secondary and K-12 Group, and the TEC. Each of these groups provides input that is vital to the continuous improvement of the EPP. Externally, stakeholders participate in a variety of advisory groups including the College of Education Advisory Board, the Clinical Services Advisory Board, and program advisory boards. The advisory boards include members who hold a variety of roles and responsibilities and, thus, provide varied perspectives to the EPP (R2.1.2 Advisory Board Examples).

Both internal and external stakeholders provided feedback that led to the revision of the EPP's mission, vision, and goals. The College of Education Advisory Board provided input via a survey in Spring 2021 sharing the knowledge, skills, and dispositions needed by new educators. This input was used and is evident in the goals developed and adopted by the EPP (R5.3.1 Mission Vision Goals). The PEF also reviewed and provided feedback on the EPP conceptual framework that includes the mission, vision, and goals at the Spring 2021 meeting (R5.1.3 PEF Minutes). The PEF reviewed the revised mission, vision, and goals in Fall 2021 before they were presented to the TEC for adoption. The internal and external feedback received was vital in creating the mission, vision, and goals that are the foundation of the EPP.

Data on unit assessments are reviewed both internally and externally to identify positive data as well as trends that are of concern. Academic year unit-wide data is shared with the PEF in the early summer. Data for several previous semesters are included so that trends are more clear. The PEF officially examines the data and provides observations during the fall PEF meeting (R5.1.3 PEF Minutes). The COE Advisory Board also provides feedback on unit-wide data at the fall meeting of the group. Most recently, attendees were asked to provide feedback based on three guiding questions: "What patterns do you notice in the data? How does this data compare to what you have observed in the new educators you hire? What suggestions do you have for addressing any concerns noted?" (R2.1.6 COE Advisory Board Minutes). A summary of stakeholder feedback on EPP data was created by the QAW (R5.1.4 Executive Summary). Feedback from the PEF and Advisory Board have contributed to the data-based decisions made by the EPP (R2.2.7 Data Based Decisions). For example, both the Advisory Board and PEF noted the need for more experience with social-emotional learning, behavior management, and communicating and working with parents. This feedback prompted the EPP to create a task force to address these needs. One strategy that has been implemented is the addition of seminars during the student teaching semester.

In addition to reviewing data from unit assessments, stakeholders have also been involved in the creation of assessments. For example, the COE Advisory Board was instrumental in the development of the instruments used to assess TC dispositions prior to the student teaching semester. The EPP adopted the EDA (R1.1.5 EDA Template) to assess candidate dispositions. However, some of the indicators are more difficult to assess prior to student teaching. Advisory Board members were asked to carefully examine the EDA and then, based on their professional experience, identify when a disposition should be assessed - entry to the program, mid-program, or exit from the program (R2.1.4 Advisory Feedback on EDA). This feedback was used to create an adaptation of the EDA to be used at the sophomore level and one appropriate for the junior level. All dispositions may be assessed at each level but only some of the dispositions are required at the sophomore and junior levels.

Feedback on clinical experiences is provided by both internal and external stakeholders. The Clinical Services Advisory Board was formed specifically to engage district partners in the design of clinical experiences (R2.1.5 Clinical Services Advisory Board). One theme that emerged from this group was the benefit in placing TC in the same classroom for both the senior practicum and student teaching. This allows the TC to see what occurs in a classroom from the beginning to the end of the school year. The Office of Clinical Services and Certification (OCSC) now seeks to place TC with the same cooperating teacher for both semesters when possible.

The UCM EPP is committed to developing and maintaining a culture in which all stakeholders, both internal and external, are involved in an ongoing review of the processes of the unit and the performance of the EPP's TC. Because of this commitment to continuous improvement, the EPP has established a structure that supports a set of systems and processes to assess EPP performance towards achieving its goals and meeting relevant standards.

 

Continuous Improvement Process

The EPP has adopted a continuous improvement process (R5.1.2 CI and Unit Data Cycles) that provides the foundation for ongoing review and data-based revision as needed of processes and systems. Within this cycle of continuous improvement, data review processes established by the EPP are in place to gather, input, analyze, interpret, and use information from the QAS effectively (R5.1.2 CI and Unit Data Cycles). Data on unit assessments are collected in the fall and spring semesters. Aggregate unit-wide data and data disaggregated by program, program type, race, gender, and first- generation status are shared with administrators, program coordinators, program faculty, unit and program advisory boards, and the PEF. After review of the data, feedback is gathered and analyzed by the QAW; the QAW then creates an executive summary based on their analysis to be shared with the TEC, PEF, and advisory boards. The TEC as the recommending body of the unit uses the analysis included in the executive summary to provide recommendations to the Dean and the unit as a whole. A parallel process at the program level allows additional review, analysis, and interpretation of program specific data and processes (R5.1.2 CI and Unit Data Cycles).

 

CAEP Aligned Goals

The EPP has adopted a set of goals that fall under five areas aligned with CAEP Standards. These five areas also form the structure for the COE Strategic Plan currently under development. The EPP assesses progress towards these goals, identifies completed steps, and proposes next steps needed to work toward achieving the goals (R5.4.1 Progress Goals).

Goal Area 1 - Professional Knowledge, Skills, and Dispositions (CAEP Standard 1 and 4):

The EPP has identified four specific goals in this area: apply current technology to engage and improve learning for all students; develop and apply the knowledge, skills, and dispositions of effective collaborative practices for educating students with special needs; develop and apply the knowledge, skills, and dispositions for enacting classroom and behavior management practices for all students; and develop and apply trauma informed practices for all students. These specific goals were identified after review of data and feedback from a variety of sources including data from key assessments and stakeholder input. The review revealed the need for enhanced focus on technology, working with students with special needs, behavior management, and trauma informed practices. The EPP has taken data-based steps to address these needs including the creation of a task force to examine existing curriculum for inclusions of trauma informed practices and make recommendations for change (R2.2.7 Data Based Decisions). The task force is currently developing a scope and sequence to assist the unit with identifying gaps and needed changes. Seminar days have been added to the student teaching semester to provide the opportunity for professional development on supporting students with special needs, differentiation, and communicating with parents, as these become much less abstract when TC are in classrooms full-time during student teaching. The EPP Technology Committee is leading an effort to crosswalk student learning outcomes in courses with the ISTE Standards to ensure TC have the necessary knowledge and skills to effectively use technology to enhance learning. In addition, the technology modules required prior to student teaching (R1.3.4 Transition Plans) are being revised to address partner feedback (R5.1.7 Partner Tech Survey Results).

The impact of the EPP's completers on P-12 student learning and development is of primary importance to the EPP. The EPP has faced some challenges as it studies the impact of its completers; the EPP believes these challenges are primarily related to additional stress educators have faced due to the pandemic. The EPP is currently using a case study approach to investigate the impact of completers on P-12 student learning and has struggled to find completers willing to participate in the case study process. A pilot of the use of focus groups was conducted in Spring 2022 as a means to supplement what is learned through the case study (R5.4.2 Pilot of Focus Group). The EPP plans to revise and expand the focus group approach (recruitment, questions, etc.) and move forward with a simultaneous use of case study and focus groups to gather important information about the impact of completers.

Goal Area 2 - Student Recruitment, Retention, and Success (CAEP Standards 2 and 3):

The EPP has identified four specific goals in this area: align recruitment initiatives with the demographics of the regions served by the EPP; increase the number of candidates from diverse backgrounds entering and completing preparation programs; increase systematic support for monitoring student persistence and success in educator preparation programs; and increase the number of non-traditional candidate pursuing educator certification through alternative pathways to certification. Enrollment trends and predictions, retention data, and feedback from partners were used in establishing goals. The EPP is located in a rural area but is close to both urban and suburban school districts; as a result, our partners include a wide variety of school districts. Missouri is currently experiencing a widespread teacher shortage and the shortage is especially acute in rural and urban areas. Thus, recruitment efforts will be primarily focused on rural and urban areas. The EPP will host a Rural Schools Summit in Fall 2022 to learn of specific needs of rural schools and to develop plans for the EPP's role in addressing those needs. Partnerships with urban districts to create alternative pathways to certification are helping to address recruitment needs as well as increasing the diversity of EPP candidates. Retention of TC is a focus of the EPP and data indicates the need for TC to connect early with EPP faculty. The EPP has piloted two freshmen seminar courses (EDFL 1400 and 1500) for teacher education majors to provide support as they transition to the university and an early connection with faculty mentors. These courses were piloted with early childhood and elementary education majors, and will be extended to other teacher education programs (R2.2.7 Data Based Decisions).

Goal Area 3 - Access, Opportunity and Community (CAEP Standards 1, 2, 3):

The EPP has identified two specific goals within this area: embed the central concepts of diversity into existing coursework to increase diverse, equitable, and inclusive pedagogical knowledge across learning environments; and integrate knowledge of contextual factors, resources, and supports to increase differentiated instruction across learning environments. The EPP has partially met this goal with actions taken based on data (R2.2.7 Data Based Decisions). Specifically data from the STES and from the state administered survey of first year teachers and their principals as well as feedback from advisory boards suggest that while TC generally feel prepared to meet the needs of all students, they do not feel equally comfortable meeting the needs of all subgroups. The EPP formed an Access, Opportunity and Community Workgroup and the group submitted a set of recommendations to the unit. The Workgroup also organized a crosswalk of how diversity concepts are introduced, developed, and assessed (R1.1.10 Diversity Workgroup). The Workgroup recommended the development and implementation of a diversity survey to be administered three times to TC to gain an understanding of how confidence with diversity develops (R1.3.4 Transition Plans).The EPP plans to include differentiation as a topic for seminars held during the student teaching semester; while candidates are certainly introduced to differentiation early in their programs, the importance of differentiation becomes more apparent during student teaching.

Goal Area 4 - Collaborative Partnerships (CAEP Standard 2):

The EPP has identified two specific goals in this area: develop and sustain meaningful reciprocal relationships to increase candidate effectiveness and provide mentor training to clinical educators. The easing of pandemic-related restrictions, some relief in the challenges school districts have faced due to the pandemic, and leadership changes for the EPP have brought a renewed focus on partnerships. Advisory boards at the unit and program level are functioning effectively and have provided feedback that has supported continuous improvement efforts. The EPP is committed to continuing to strengthen partnerships and to ensure they are mutually beneficial. A review of existing processes has resulted in the conclusion that while school-based clinical educators at the student teaching level receive consistent training; this is not the case for clinical educators working with pre-student teaching experiences. While some programs do provide training, this is not consistent across all programs. The OCSC is taking the lead in planning for consistent training for all clinical educators (R2.2.7 Data Based Decisions).

Goal Area 5 - Continuous Improvement (CAEP Standards 1, 2, 3, 4 and 5):

The EPP has identified two specific goals in this area: develop and maintain a culture of assessment, defined as evidence-based decision-making to improve and sustain educator candidate learning outcomes; and collaborate with stakeholders to collect and use data to continuously review the impact of educator candidates and completers. These specific goals resulted from a review of existing processes and a shared belief in continuous improvement. The EPP believes progress has been made towards the achievement of this goal. The QAW effectively monitors the functioning of the QAS, the COEAC monitors program review and works to develop a culture of assessment in the EPP, coordinators have been put in place to ensure the tasks associated with implementing continuous improvement processes are completed, and functioning advisory boards meet regularly to provide feedback to the unit and to programs (R5.4.1 Progress Goals). Because continuous improvement must be ongoing and always a priority, the EPP acknowledges the goals in this area will not ever be considered fully met.

The University of Central Missouri (UCM) Educator Preparation Program (EPP) offers a wide variety of graduate programs. In conjunction with CAEP, the Missouri Department of Elementary and Secondary Education (MoDESE) defines an advanced program falling under CAEP review as one leading to a certification for which MoDESE requires an advanced degree (RA1.1.1 DESE Memo). Using this definition, UCM's Educational Leadership, Career Education Director, and Counseling programs are to be reviewed by CAEP. However, the EPP's Counseling program is accredited by the Council for Accreditation of Counseling and Related Educational Programs (CACREP) (RA1.1.2 CACREP Letter, RA1.1.3 Counseling Self-Study). Therefore, this report focuses specifically on the EPP's Educational Leadership and Career Education Director programs.

The Educational Leadership program is offered as part of a Master of Science in Education (MSE) or an Education Specialist (EdS) degree program. Candidates may seek certification as K-12 school leaders (including special education directors) or as superintendents. The Career Education Director is one option in the EPP's Career and Technology Education (CTE) Master of Science (MS) or EdS degree programs. The MS degree (CTE Administration Leadership option) is intended for new CTE administrators pursuing administrator certification and the EdS is intended for current CTE administrators pursuing administrator certification. The EPP prepares school leaders who are reflective practitioners by requiring candidates to think analytically, practically and creatively about the teaching, learning, and decision making processes associated with educational leadership.

Along with formative assessments embedded in coursework, the Educational Leadership and Career Education Director programs use three common summative assessments: the Missouri Content Assessment (MOCA) (RA1.1.4 Superintendent MOCA, RA1.1.5 Building Level Admin MOCA); the Educational Leadership Disposition Assessment (ELDA) (RA1.1.6 ELDA); and the Missouri Performance Assessment for Aspiring Building Administrators (MPEA-PA) (RA1.1.7 MPEA-PA). The formative and summative assessments used by these programs are aligned to the Missouri Leadership Development System (MLDS) Standards which were created from the National Policy Board of Educational Administrators (MPBEA) including the National Educational Leadership Preparation (NELP) Standards/Professional Standards for Education Leaders (PSEL) (RA1.1.8 Standards Crosswalk). The MLDS Standards consist of 32 competencies organized into five domains: Visionary Leadership, Managerial Leadership, Instructional Leadership, Relational Leadership, and Innovative Leadership.

The EPP's candidates in the Educational Leadership and Career Education Director programs demonstrate their ability to apply knowledge and skills to enhance the learning and opportunities for all K-12 students and teachers. Candidates in these programs:

• apply data literacy;
• understand and use research methodologies;
• employ data analysis to develop supportive, diverse, equitable, and inclusive school environments;
• lead and participate in collaborative efforts with all stakeholders;
• support appropriate applications of technology; and
• demonstrate professional dispositions and ethics.

As is shown in the linked curriculum map, these six proficiencies are introduced, reinforced, and assessed in coursework throughout the programs (RA1.1.9 Assessment Plan and Curriculum Map). Program leadership of the Educational Leadership and Career Education Director programs identified three of the six proficiencies for a common and more in-depth analysis: 1) apply data literacy; 2) employ data analysis to develop supportive, diverse, equitable, and inclusive school environments; and 3) lead and participate in collaborative efforts with all stakeholders. In what follows, each of these three proficiencies is described, the method(s) of assessment is detailed, and data are provided. Because the formative assessments are embedded in courses that are not necessarily offered every semester and because the number of candidates completing the assessment at a given time may be small, the data cycles reported may vary and data are reported in the aggregate.

 

Proficiency 1: Apply Data Literacy

Formative Course-Embedded Assessments

The Career Education Director program assesses application of data literacy formatively using the Common Criteria and Quality Indicators (CCQI) Activity completed during CTE 5150: Introduction to Career Administration. This activity requires candidates to evaluate a career education program using the Secondary (or Post Secondary) Career Education Programs Evaluation Tool and then develop a Program Improvement Plan for the program reviewed. The plans are scored by the instructor of the course and feedback is provided to candidates. Over three cycles of data (Fall 2019, Fall 2020, Fall 2021), 38 candidates completed the assignment with a mean score of 99.15/100 (RA1.1.10 CTE Data Literacy).

The Educational Leadership program, K-12 School Leader option, assesses application of data literacy formatively using the Student Learning Data Analysis Project completed in EDAD 5960: Data Analysis for School Leaders. The assessment includes the development of a presentation for a school staff audience reflecting on assessments given, recommendations for the current assessment plan, and ways student achievement may be improved. Over three cycles of data (Spring 2020, Spring 2021, Spring 2022), 128 candidates completed the assignment and 100% met the benchmark by scoring at the "good" or "excellent" level as indicated by the evaluation instrument (RA1.1.11 EDAD Data Literacy 1).

The Educational Leadership program, Superintendent Option, assesses application of data literacy formatively using the Entry Plan Project completed in EDAD 6700: School District Administration. Candidates review the data of a given district and develop an entry plan for their first 180 days as superintendent of that district. The plan must be based on current accessible district data. Because this class is not offered every year, data were collected in the Fall 2017, Fall 2018, and Fall 2020 semesters. Over the three cycles of data, 34 candidates completed the assessment and 100% of the candidates met the benchmark by scoring at the "meets" or "exceeds" level as indicated by the evaluation instrument (RA1.1.12 EDAD Data Literacy 2).

 

Proficiency 1: Apply Data Literacy

Summative Assessment

The Career Education Director and K-12 School Leader programs summatively assess data literacy using the MPEA-PA. Every candidate pursuing Missouri administrator certification must pass the MPEA-PA to be eligible for certification. Passing the assessment indicates that candidates have the appropriate knowledge and skills within the five MLDS Domains to be an effective educational leader. The MPEA-PA is a performance assessment with four steps requiring responses to multiple prompts supported by required evidence. Step 1 of the MPEA-PA is a measure of data literacy as it aligns with MLDS Visionary Leadership Domain, Competency 3, "Understands how multiple sources of data are connected to a mission, vision, and core values." For this step, candidates must use data to identify an instructional problem within the school. Over the three cycles of data (2019, 2020, 2021), 152 candidates from both programs completed Step 1 of the MPEA with a mean score of 3.32 out of 4 points. (RA1.1.7 MPEA-PA).

The Educational Leadership program, Superintendent Option, uses the MOCA to summatively assess application of data literacy. The MOCA is aligned to NELP and NPBEA Standards (RA1.1.4 Superintendent MOCA). As part of the MOCA, superintendent candidates are required to prepare a response in which they analyze student achievement data for a school district and describe how they would address an issue reflected in the data. Over the three-year data cycle (2020, 2021, 2022), 15 candidates completed the MOCA and data were collected and analyzed. The data were disaggregated by gender and race/ethnicity for the same three years. Over the three-year period, the overall pass rate was 67.70% with 2022 as the lowest (50% passing) and 2020 as the highest (100% passing). While the overall pass rate is lower than what the program faculty would like, the EPP recognizes that there are some concerns with the data set, as some candidates may take the test multiple times before passing and the EPP may not be notified of the passing score. The EPP also recognizes that there are some candidates who may have completed their academic program of study up to three years prior to taking the test. This was true for approximately 30% of the test takers included in the data.

Overall, the data for Proficiency 1 indicate that candidates in the EPP's advanced programs under CAEP review are proficient in applying data literacy skills including being able to access, interpret, and communicate data, understanding data sources and constructs, and use appropriate analytical methods.

 

Proficiency 2: Employ Data Analysis to Develop Supportive, Diverse, Equitable, and Inclusive School Environments

Formative Course-Embedded Assessment

The Career Education Director program formatively assesses candidates' ability to employ data analysis to develop supportive, diverse, equitable and inclusive school environments using the CCQI Activity completed during CTE 5150: Introduction to Career Administration. This activity requires candidates to evaluate a career education program using the Secondary (or Post Secondary) Career Education Programs Evaluation Tool. Data analysis is used to create a plan for improvement for the program being evaluated and the school environment is part of the improvement plan as is appropriate. Over three cycles of data (Fall 2019, Fall 2020, Fall 2021), 38 candidates completed the assignment with a mean score of 99.15/100 (RA1.1.10 CTE Data Literacy). An additional formative assessment occurs in CTE 6080: Financing and Funding Career and Technical Education. For this assignment, candidates are required to use data to determine program needs for improvement and qualifications for funding. Over three cycles of data (Spring 2019, Spring 2020, Spring 2021), 44 candidates completed the assignment with a mean score of 100 (RA1.1.13 CTE Data Analysis).

The Educational Leadership program, K-12 School Leader Option formatively assesses candidates' ability to employ data analysis to develop supportive, diverse, equitable and inclusive school environments using the Perception Questionnaire Project completed in EDAD 5960: Data Analysis for School Leaders. Candidates work with their mentor/building administrator to identify a clear purpose for a survey based on perceived problems or challenges within the school. The candidate creates a questionnaire for either parents, staff, or students to gather data around the perceived problem or challenge. The questionnaire is piloted with a small group to gather feedback about the questions and format of the questionnaire. Appropriate modifications are made, and the candidate distributes the revised survey to the appropriate group and collects results. Candidates analyze the results and develop a way to share the results in an easy to understand format with appropriate groups.

Candidates review the results with their mentor/building administrator and develop and implement a plan to improve the perceived environment. Over the three cycles of data (Spring 2020, Spring 2021, Spring 2022), 128 candidates completed the assignment and 100% of the candidates met the benchmark by scoring at the "good" or "excellent" level as indicated by the evaluation instrument (RA1.1.14 EDAD Data Analysis 1). The Educational Leadership program, Superintendent Option formatively assesses candidates' ability to employ data analysis to develop supportive, diverse, equitable and inclusive school environments using the Newsletter and Letter to State Legislator Project completed in EDAD 6760: Politics and School Leadership (EDAD 6760 is offered as needed). Candidates write a letter to their state legislator and an article for the district newsletter around a specific bill that has been introduced or is anticipated to be introduced in the upcoming legislative session. The candidate must provide multiple key points of how the issue would impact their district. Over three cycles of data (Spring 2019, Fall 2019, Fall 2021), 34 candidates completed the assignment and 100% of the candidates met the benchmark by scoring at the "good" or "excellent" level as indicated by the evaluation instrument (RA1.1.15 EDAD Data Analysis 2).

 

Proficiency 2: Employ Data Analysis to Develop Supportive, Diverse, Equitable, and Inclusive School Environments

Summative Assessment

The Career Education Director and K-12 School Leader program summatively assesses candidates' abilities to employ data analysis to develop supportive, diverse, equitable and inclusive school environments using the MPEA-PA. As noted above, every candidate pursuing Missouri administrator certification must pass this assessment. Step 2 of the MPEA-PA is a measure of employing data analysis to create supportive, diverse, equitable, inclusive school environments as it aligns with MLDS Relational Leadership Domain. For Step 2, candidates are required to analyze a data set to create a plan to address the problem identified in Step 1 (previous section), and this will include addressing concerns for the climate and culture of the school environment. Over three cycles of data (2019, 2020, 2021), 152 candidates from both programs completed Step 2 of the MPEA with a mean score of 3.29 out of 4 points (RA1.1.7 MPEA-PA).

The Educational Leadership program, Superintendent Option, uses the MOCA to summatively assess the employment of data analysis to develop supportive, diverse, equitable, and inclusive school environments. As part of the MOCA, superintendent candidates are required to prepare a response in which they analyze student achievement data for a school district and describe how they would address an issue reflected in the data. This analysis will address concerns regarding the school environment. In addition, the multiple-choice portion of the assessment includes questions with a focus on "Culture of Learning" and "Collaboration with Stakeholders." As noted above, over the three-year data cycle (2020, 2021, 2022), 15 candidates completed the MOCA and data were collected and analyzed. Over the three-year period, the overall pass rate was 67.70% with 2022 as the lowest (50% passing) and 2020 as the highest (100% passing) (RA1.1.4 Superintendent MOCA). Because those taking the MOCA are not required to send their scores to the EPP, those who pass on subsequent attempts may not be known to the EPP.

 

Proficiency 3: Lead and Participate in Collaborative Efforts with All Stakeholders

Formative Course-Embedded Assessment

The Career Education Director program formatively assess candidates' ability to lead and participate in collaborative efforts with stakeholders using the Emerging Issues Project completed during CTE 5150: Introduction to Career Administration. For this project, candidates select a topic in CTE and research best practices and/or emerging trends in career and technical education. Utilizing this information, the candidate plans and implements relevant information into CTE programs. The project requires candidates to consider a variety of stakeholder groups in the planning process. Over three cycles of data (Fall 2019, Fall 2020, Fall 2021), 35 candidates completed the assignment with a mean score of 98.62. In Fall 2019, nine candidates completed the assignment with a mean score of 97.78/100; this was the lowest mean score of the three years. In Fall 2020, 12 candidates completed the assessment with a mean score 98.08/100 and in Fall 2021, 14 candidates completed the assessment with a mean score of 100/100. (RA1.1.16 CTE Collaboration).

The Educational Leadership, K-12 School Leader Option, formatively assesses candidates' ability to lead and participate in collaborative efforts with stakeholders using the School Improvement Plan Project completed during EDAD 5770: Instructional Leadership and School Improvement. Candidates work individually or as part of a team to develop a two-year plan to turnaround a struggling school. Candidates must collaborate with staff to develop, articulate, implement, and steward a shared vision of learning for the school. The candidates review the data provided and identify new data resources to investigate in order to identify reasonable school goals, assess organizational effectiveness, and develop a plan to achieve these new goals. Finally, the plan must show how the leader will monitor the success of the plan, make appropriate revisions where necessary, and create a process which will be supported by the stakeholders. Over three cycles of data (Spring 2020, Spring 2021, Spring 2022), 166 candidates completed the assignment and 98.80% of the candidates met the benchmark by scoring at the "good" or "excellent" level as indicated by the evaluation instrument (RA1.1.17 EDAD Collaboration 1).

The Educational Leadership program, Superintendent Option, formatively assesses candidates' ability to lead and participate in collaborative efforts with stakeholders using the Action Plan Development Project completed during EDAD 6160: School and Community Relations. The assignment requires candidates to plan and direct a Community Advisory Committee to complete a school and community relations communications plan. Over three cycles of data (Spring 2020, Spring 2021, Spring 2022), 128 candidates completed the assignment and 100% of the candidates met the benchmark by scoring at the "good" or "excellent" level as indicated by the evaluation instrument (RA1.1.18 EDAD Collaboration 2).

 

Proficiency 3: Lead and Participate in Collaborative Efforts with All Stakeholders

Summative Assessment

The Career Education Director and K-12 School Leader programs summatively assess candidates' ability to lead and participate in collaborative efforts with stakeholders using the MPEA-PA. Step 2 of the MPEA-PA addresses collaboration with stakeholders with the requirement that candidates describe how they worked with their team and other stakeholders to address an identified problem. In addition, a communication plan for use with the team, faculty, and staff is required. Over three cycles of data (2019, 2020, 2021), 152 candidates from both programs completed Step 2 of the MPEA with a mean score of 3.29 out of 4 (RA1.1.7 MPEA-PA).

The Educational Leadership program, Superintendent option, uses the MOCA to summatively assess candidates' ability to lead and participate in collaborative efforts with stakeholders. As part of the MOCA, superintendent candidates are required to prepare responses to two prompts. One of these prompts requires the candidate to create a plan for addressing a district issue impacting student learning. As part of the plan, it is expected the candidate will include how collaboration with other stakeholders will help to address the issue. As noted above, over the three year data cycle (2020, 2021, 2022), 15 candidates completed the MOCA and data were collected and analyzed. Over this period, the overall pass rate was 67.70% with 2022 as the lowest (50% passing) and 2020 as the highest (100% passing) (RA1.1.4 Superintendent MOCA). Because those taking the MOCA are not required to send their scores to the EPP, those who pass on subsequent attempts may not be known to the EPP.

UCM's EPP ensures that candidates in the Educational Leadership and Career Education Director programs have opportunities to learn and apply specialized content and discipline knowledge contained in approved state and/or national discipline-specific standards. The Educational Leadership and Career Education Director programs share three key assessments. Key assessments used in the program are aligned to MLDS Standards, the NELP Standards, and the NPBEA Standards as appropriate. The key assessments are summative in nature and together provide evidence of the EPP's candidates' ability to apply discipline-specific knowledge. Each key assessment is described ahead, and the linked documents provide information about these assessments including alignment to standards, administrative details, a copy of the instrument if appropriate, unit and disaggregated data tables, and data analysis. In addition, a series of planned, sequenced internship experiences provide additional opportunities for candidates to apply their specialized knowledge.

 

Key Assessment: Missouri Content Assessment

The MOCA for Building Level Administrators and the MOCA for Superintendents are proprietary assessments required for certification as a K-12 Building Administrator or Superintendent. Frameworks for each assessment provide an overview of the proficiencies expected for Missouri's educational leaders. The MOCA for Building Level Administrators is aligned to the MLDS standards and the NELP standards (RA1.1.5 Building Level Admin MOCA) and the MOCA for Superintendents is aligned to NELP and NPBEA Standards (RA1.1.4 Superintendent MOCA). Candidates in the Educational Leadership, K-12 School Leader option and the Career Education Director program take the MOCA for Building Level Administrators. Of the 151 candidates who have completed the MOCA, 114 passed on the first attempt (75.5%). Candidates who completed the MOCA in 2019 had the lowest first-attempt pass rate (68.92%) while candidates who completed the MOCA in 2020 had the highest first-attempt pass rate (87,8%). Ninety-one females took the MOCA with a first-attempt pass rate of 73.63% and 55 male candidates took the MOCA with a 76.36% first-attempt pass rate. Additional disaggregated data is available in the assessment overview document (RA1.1.5 Building Level Admin MOCA). Candidates in the Educational Leadership Superintendent option take the MOCA for Superintendents. The first-attempt pass rate for the 15 candidates who have completed the MOCA is 67.70%. Because those taking the MOCA are not required to send their scores to the EPP, those who pass on subsequent attempts may not be known to the EPP. Program leadership in the Educational Leadership and Career Education Director program regularly reviews the MoCA results and uses them to guide curricular updates.

 

Key Assessment: Missouri Performance Assessment for Aspiring Building Administrators

The MPEA-PA is a performance assessment required for all those seeking certification as K-12 principals or career education directors. The assessment was developed by Missouri Professors of Educational Administration and this group is authorized by MoDESE to administer, score, and facilitate quality control of the performance assessment required for certification. The group offers bi-annual professional development conferences to ensure the assessment is consistent and valid. The MPEA-PA has four steps, each aligned to MLDS domains, with multiple writing prompts and requests for artifacts within each step. Each step has a maximum score of 4 and candidates must earn a minimum score of 10 out of 16 to pass. The assessment requires aspiring building administrators to use data to identify a problem that exists, create a plan to address the problem, identify and work with stakeholders, address any legal or ethical issues, and develop a communication plan. The MPEA-PA is extensive and designed to assess all aspects of the work required of a building administrator. Over the three cycles of this report, 151 of the 152 candidates who completed the MPEA-PA achieved a passing score of 10 or higher. The mean score was 13.24 with a standard deviation of 1.65. The data were further analyzed by gender and race/ethnicity for each year. Pass rates were very high for this summative assessment; 152 candidates completed the MPEA-PA with a 99.34%.pass rate. Mean scores for the steps consistently increased from 12.86 in 2019 to 13.52 in 2021. Since the pass rates were high and the population was not diverse, the disaggregated data do not present much new information. The only non-passing score came from a white male in 2019. Females had higher total scores than males in 2019 and 2020 but males' scores were higher in 2021.The race/ethnicity comparisons are difficult to analyze with white candidates making up almost 90% of the participants and no consistent race/ethnicity groups across all three years (RA1.1.7 MPEA-PA).

 

Key Assessment: Educational Leadership Dispositions Assessment

As part of the EPP's 2016 NCATE Legacy visit, reviewers noted the need for consistent assessment of dispositions at the advanced level. In Fall 2020 the EPP adopted the Educational Leadership Dispositions Assessment (ELDA) developed at the University of Tampa. This assessment is aligned with the NELP Standards and measures 15 dispositions using 47 associated indicators. Each indicator is measured using a 3-point scale with "0" representing "Needs Improvement", "1" representing "Developing", and "2" representing "Meets Expectations." In conjunction with their faculty, candidates in the Educational Leadership and Career Education Director programs complete the assessment at entry to the program, mid-program, and at program completion. This provides program faculty with the opportunity to address areas of concern with candidates. Over the three cycles of data (2019, 2020, 2021), 92 candidates have completed the assessment at the entry point and 150 candidates have completed end-of-program assessment. (The difference in the number of candidates completing the assessment at these two points is due to the implementation schedule followed.) Over three data cycles, scores on the entry-to-program administration of the ELDA range from 1.11(embedding vision in all decisions) to 1.79 (accepting constructive feedback) with a mean of 1.47 across all indicators. Scores on the completion-of-program administration of the ELDA range from 1.77 (embedding vision into all decisions) to 1.99 (creating positive relationships) with a mean of 1.91 across all indicators. Analysis of the ELDA data provides the EPP with information to inform program revisions and curricular changes (RA1.1.6 ELDA). Statistically significant increases from beginning of program ratings to end of program ratings indicate that candidates' dispositions improve as they gain experience and feedback throughout their coursework and internships. Female candidates had a larger rating increase overall (0.41) compared to males (0.37).

 

Preparation to Lead in a Variety of School Settings

Missouri's schools are increasingly diverse and the EPP embraces the need for candidates in the Educational Leadership and Career Education Director programs to be well-prepared to lead in a variety of school settings. Items from key assessments provide evidence that the EPP's candidates are prepared and confident in this area. Disposition 6 of the ELDA is titled "Embracing Diversity and Equity" and aligns with NELP 3.1, 3.5, and 5.1. The three indicators for this disposition state that a candidate who meets expectations "6a: Promotes a climate that makes the school welcoming, inclusive, and accepting of all students by treating others fairly and by not showing partiality to any persons or groups," "6b: Demonstrates critical consciousness of culture and race by embracing the diverse needs, interests, and strengths of all students, staff, and the larger community," and "6c: Demonstrates cultural responsiveness and promotes such responsiveness among the school at large." Data on indicators 6a, 6b, and 6c from the end-of-program administration of the ELDA were analyzed. Means on the three indicators were 1.95, 1.91, and 1.91 out of 2 respectively (RA1.1.6 ELDA) indicating a high level of commitment to embracing diversity. Prompts 3 and 4 of Step 1 of the MPEA-PA (RA1.1.7 MPEA-PA) require candidates to consider the community demographic and diversity of all students in planning to address an existing problem as well as the impact of the plan on all students. Over the three cycles of data for this report, 152 candidates completed Step 1 of the MPEA-PA with a mean score of 3.32 and standard deviation of 0.51. Finally, while data on specific competencies is not available, the MOCA Building-Level Administrator test framework consistently references "all students" and specific competencies include the need to hold equity and diversity as a priority and to incorporate diverse perspectives.

As is true in all educational settings, the pandemic has resulted in the need for the EPP to focus on the ability of candidates in the Educational Leadership and Career Education Director programs to fully integrate technology into their work as educational leaders. The programs have reviewed the technology integrated into coursework within their programs (RA1.1.19 Technology Integration). This review will provide a foundation for discussions among program faculty regarding any additional opportunities for candidates to gain experience with educational technology.

The University of Central Missouri (UCM) Educator Preparation Program (EPP) offers a wide variety of graduate programs. In conjunction with CAEP, the Missouri Department of Elementary and Secondary Education (MoDESE) defines an advanced program falling under CAEP review as one leading to a certification for which MoDESE requires an advanced degree (RA1.1.1 DESE Memo). Using this definition, UCM's Educational Leadership, Career Education Director, and Counseling programs are to be reviewed by CAEP. However, the EPP's Counseling program is accredited by the Council for Accreditation of Counseling and Related Educational Programs (CACREP) (RA1.1.2 CACREP Letter, RA1.1.3 Counseling Self-Study). Therefore, this report will focus specifically on the EPP's Educational Leadership and Career Education Director programs.

The University of Central Missouri EPP has established strong and positive relationships and partnerships with many school districts. These partnerships have resulted in opportunities for the EPP's candidates to apply their content knowledge in a variety of school settings. The EPP's partner districts share in the responsibility to develop effective K-12 school leaders, superintendents, and career education directors.

The two options (K-12 School Leader and Superintendent) of the Educational Leadership program and the Career Education Director program require clinical experiences for program completion and certification in Missouri. While the clinical experience format varies across these programs, each program relies on school district partnerships and regularly collaborates with these partners. Partnerships with districts are formalized through collaboratively developed Memorandum of Understanding (MOU) agreements (R2.1.1 Examples of MOUs).

The Educational Leadership and Career Education Director programs have advisory boards that meet regularly and provide valuable guidance to program leadership. The advisory boards' memberships reflect rural, urban, and suburban school districts of varying sizes. Many of those serving on the boards are graduates of the programs and, thus, have extensive knowledge of the programs. Both programs utilize the expertise of their partners to collaboratively design meaningful clinical experiences for candidates. What follows is information specific to each program.

 

Educational Leadership Program

The Educational Leadership program collaborates with an Advisory Board composed of current district principals for guidance and support for the K-12 School Leader program option. Many members of this advisory board are graduates of the Educational Leadership program and, thus, are very familiar with the program. In addition, program leadership has organized an additional advisory board of current superintendents who bring a different perspective from that of the principals. Membership rosters for the two advisory boards reflect diverse representation in district size and nature (RA2.1.1 EDAD Advisory Boards) and the two groups fill important and complementary roles in supporting the Educational Leadership program. Minutes from the meetings of the advisory boards provide evidence of the valuable role partners play in providing feedback to the EPP (RA2.1.2 EDAD Advisory Minutes).

Candidates in the K-12 School Leader and Superintendent programs must complete two semesters of internship. Over the course of their program, they are required to complete a minimum of 300 hours working with administrators in the district in which they are completing the internships (a minimum of 75 hours must be completed in each of the two internship semesters with the remaining hours spread over the two semesters). In conjunction with EPP faculty, the district hosting the internship identifies an on-site mentor to work with the candidate during the internship. For all internships, the clinical experience is designed through collaboration among the candidate, the on-site mentor, and the EPP faculty member who together determine the tasks to be completed, plan for the completion of the tasks, and evaluate task completion. As such, the clinical experience of each candidate in the K-12 School Leader and Superintendent programs is individually constructed to meet the needs of the candidate and the district hosting the candidate for the internship. All internship experiences are aligned with the domains of the Missouri Leadership Development System (MLDS) designed to prepare candidates who are proficient across all aspects of leadership. Those domains are Visionary, Instructional, Managerial, Relational, and Innovative Leadership and the competencies within those domains are aligned with the Professional Standards for Education Leaders (RA1.1.8 Standards Crosswalk).

The Superintendent Advisory Board provides important feedback on potential topics/tasks to be included in internship courses. Recently, for example, the board was presented with a list of the potential topics/tasks (RA2.1.3 Internship Essentials) and were asked to review the list and indicate those that are essential and to add any essential topics/tasks that were missing. Responses were gathered and analyzed (RA2.1.4 Feedback on Internship). The candidate, on-site mentor, and EPP faculty use this prioritized list to plan internship tasks to be completed.

The EPP embraces the need for candidates in the Educational Leadership program to be well prepared to lead in a variety of school settings. As a result, for all internships in the K-12 School leader program, candidates are required to work at least four hours in a school setting that is completely different from their internship placement. For example, those placed in small rural districts may also spend time working in a larger district. Candidates working in districts with a high level of poverty may also spend time working in a more affluent school district. For the Superintendent option, candidates are required to consult with superintendents and assistant superintendents representing a variety of school types and sizes. Specific suggestions for topics/tasks are provided (RA2.1.5 Superintendent Practicum) to guide the candidate and on-site mentor in planning the clinical experience.

The EPP values the strong partnerships it shares with school districts and seeks opportunities to serve its partners. The Educational Leadership program regularly works with partner districts to recruit cohorts of aspiring administrators from that district. Courses are offered within the district and district administrators are employed to serve as adjuncts to teach some of the courses. This provides a convenient and comfortable format for partner districts to grow leadership in their districts. The EPP supports groups of area elementary principals, secondary principals, and superintendents, who each meet monthly on campus, by partnering to provide space and refreshments. These meetings provide an additional opportunity for the EPP to collaborate and gather feedback from stakeholders (RA2.1.6 MASA Agenda).

 

Career Education Director Program

The Career Education Director program is part of the EPP's Career and Technical Education program (CTE). The CTE Graduate Programs Advisory Board provides input and guidance to the CTE faculty. The CTE Advisory Board includes members from area school districts, MoDESE, UCM faculty, and current UCM students (RA2.1.7 CTE Advisory Board). Agendas from this group's meetings provide evidence of the important feedback solicited and received from the group (RA2.1.8 CTE Minutes).

Candidates in the Career Education Director program are required to complete one internship course and then participate in a two-year mentoring program before they are eligible for Missouri certification. Both the internship and mentoring experience are planned and implemented through collaboration among the candidate, the on-site mentor, and EPP faculty. Candidates in the Career Education Director program complete internship hours in area career centers associated with a group of school districts. As part of the internship course, candidates are required to meet with on site mentors to plan a series of career-specific experiences designed to develop the knowledge and skills necessary for career education directors. Agreements are signed by the candidates, on-site mentors, and EPP faculty detailing the responsibilities of each during the internship (RA2.1.9 CTE Internship Agreement). The collaboration involved ensures that each candidate completes an individually designed clinical experience that meets the needs of both the candidate and the host district.

As mentioned previously, Missouri certification as a Career Education director requires candidates to complete a two-year mentoring program. MoDESE has authorized the Missouri Council of Career and Technical Administrators (MCCTA) in collaboration with UCM faculty, to plan, implement, and maintain the quality of the mentoring program. The mentoring program requires candidates to work closely with the assigned on-site mentor to plan the experience following guidelines established by MCCTA. During the first year, candidates and on-site mentors work together to develop the knowledge and practical skills required by career education directors in the area of basic leadership, individual program analysis, finance, and personnel administration. A handbook developed by MCCTA provides clear guidance to the candidate and on-site mentor (RA2.1.10 First Year Mentoring). During the second year, continued collaboration among the candidate, on-site mentor, and EPP faculty results in a plan that assists the candidate in developing knowledge and skills in developing new programs or revising existing ones, curriculum development, student organizations, and program evaluation. A handbook developed by MCCTA for the second year of mentoring provides guidance for the experience (RA2.1.11 Second Year Mentoring). All internship and mentoring experiences are aligned with the MLDS domains.

Because candidates in the Career Education Director program complete their internships in area career centers, they have the opportunity to work with a diverse group of students since several districts send students to the career centers. For example, one career center located in a rural school district works with students from seven rural school districts, two suburban school districts, and one urban school district. This structure provides the opportunity for candidates to work with school districts of varying type and size.

Candidates in the Educational Leadership and Career Education Director programs complete a sequence of clinical experiences that are developmental in nature and designed to provide opportunities for candidates to apply their specialized content knowledge and skills. As part of their clinical experiences, candidates complete the performance assessment for building administrators (MPEA-PA) required for Missouri certification as a building principal or career education director.

The Missouri Standards for the Preparation of Educators (MoSPE) provide specific guidance on the length of clinical experiences and the qualifications of the on-site mentor and the EPP faculty member providing supervision of the internship (R2.2.1 MOSPE Requirements). As specified by MoSPE, all candidates are expected to experience full immersion in a school setting and engage in leadership activities aligned to MLDS domains and competencies. The clinical experience is designed collaboratively by the candidate, on-site mentor, and university faculty supervisor. Each candidate is required to complete 300 hours with the hours divided across the five MLDS domains. On-site mentors must hold at least a master's degree to supervise K-12 School Leader and Career Education Director candidates and at least an Education Specialist degree to supervise Superintendent candidates. All on-site mentors must have a minimum of three years of K-12 supervision experience. The EPP faculty supervisor must hold at least an Education Specialist degree and have a minimum of five years experience in K-12 schools.

 

Educational Leadership Program

Candidates in the Educational Leadership, K-12 School Leader and Superintendent options, complete two semesters of internship courses as a requirement for program completion and certification. Over the course of their program, candidates must complete a minimum of 300 internship hours (a minimum of 75 hours must be completed in each of the two internship semesters with the remaining hours spread over the two semesters). As noted earlier, the specific tasks to be completed during an internship are determined collaboratively by the candidate, on-site mentor, and EPP faculty, and may be drawn from a list of topics/tasks developed with feedback from the program's advisory board (RA2.1.3 Internship Essentials). Because of the collaborative and careful approach to planning the internships, EPP faculty are able to ensure the experiences are developmentally appropriate for candidates. Each candidate's internship plan is unique and reflects the needs of the candidate and the host district. Hours are logged to provide evidence candidates have completed tasks in each of the MLDS domains (RA2.2.1 EDAD Log).

As part of the internships, candidates work with their on-site mentors and EPP faculty on the performance assessment (MPEA-PA) required for certification. This assessment is multi-step and focuses on the aspiring administrators' abilities to serve as instructional leaders. The assessment presents candidates with a problem-based task and allows them to demonstrate the proficiencies they have developed during the program. For each step of the assessment, candidates submit a narrative and related artifacts as evidence. The first step of the MPEA-PA is focused on the visionary leader competencies. In this step, candidates identify a problem or area of concern related to instruction based on a review of data, describe the impact of not addressing the problem, describe the climate and culture of the building where the problem exists, and describe any legal implications of the problem or concern. In the second step, the candidates describe the plan created to address the problem identified in the first step, how the plan was implemented, the collaboration that occurred with the school team and other stakeholders, the communication plan to share information about the plan, the professional learning activities needed, and the method used to determine the impact of the professional learning activities that were implemented. The second step is designed around relational and innovative leadership competencies. Step three focuses on managerial leadership competencies and requires the candidate to describe the routine, procedures and schedules involved in implementing the plan, how the goals and anticipated outcomes were developed, key personnel involved and their specific contributions, methods used to engage faculty and staff, check-points that were established to provide formative feedback, and summative evaluation processes. The final step of the assessment further develops innovative leadership competencies and asks candidates to describe how they prepared themselves to be effective leaders, write a self-reflection on the success or lack of success of the plan, aspects of the plan they would keep in place and those they would change, and the leadership skills and proficiencies gained. Typically steps 1 and 2 are completed by the end of the first internship and steps 3 and 4 are completed by the end of the second internship; however, this may vary depending on the problem or challenge identified and the plan developed (RA1.1.7 MPEA-PA).

As noted above, clinical experiences for both the K-12 School Leader program and the Superintendent program require candidates to work in settings different than those of the district in which they are completing their internships. In addition, the performance assessment completed by K-12 School Leaders candidates during their clinical experience and submitted for certification requires them to address diversity within the plan and to anticipate the impact the plan would have on all students.

 

Career Education Director Program

Candidates working on their CTE Administration certification are required to complete 300 hours of internship and cannot be endorsed for certification until the completion of the internship and the mentoring experience designed collaboratively by the Missouri Council of Career and Technical Administrators (MCCTA), the MoDESE, and UCM as described previously.

During these internship and mentoring experiences, candidates work in one of the 57 Area Career Centers that serve a variety of school districts, from small rural to suburban to larger urban areas (RA2.2.2 Career Center Map). Area Career Centers serve multiple school districts who send their students for particular technical training programs (health science, skilled technical sciences such as automotive technology, graphic arts, welding, law enforcement, etc.). Students from sending high schools intermingle in each career center and program, creating a unique, diverse environment for administrators to lead. Area Career Center directors and assistant directors are also responsible for IEP transitions for students from sending schools. Many Area Career Centers also host adult training programs (i.e., practical nursing) which requires working with adults as well as high school students. As they complete internship and mentoring experiences, candidates are required to document their hours for each competency within the five domains of the Missouri Leadership Development System (MLDS): Visionary, Instructional, Managerial, Relational, and Innovative (RA2.2.3 CTE Log).

Career Education Director candidates are required to complete the same performance assessment (MPEA-PA) as candidates in the Educational Leadership, K-12 School Leader program described previously. The assessment requires candidates to demonstrate their leadership proficiencies through the use of a problem-based task (RA1.1.7 MPEA-PA).

The University of Central Missouri (UCM) Educator Preparation Program (EPP) offers a wide variety of graduate programs. In conjunction with CAEP, the Missouri Department of Elementary and Secondary Education (MoDESE) defines an advanced program falling under CAEP review as one leading to a certification for which MoDESE requires an advanced degree (RA1.1.1 DESE Memo). Using this definition, UCM's Educational Leadership, Career Education Director, and Counseling programs are to be reviewed by CAEP. However, the EPP's Counseling program is accredited by the Council for Accreditation of Counseling and Related Educational Programs (CACREP) (RA1.1.2 CACREP Letter, RA1.1.3 Counseling Self-Study). Therefore, this report will focus specifically on the EPP's Educational Leadership and Career Education Director programs.

The UCM EPP understands the importance of recruiting and supporting candidates who leave the Educational Leadership and Career Education Director programs eligible for certification and well-prepared to perform effectively in leadership roles. From recruitment to completion, the EPP maintains an ongoing and intentional focus on the quality of candidates in advanced programs.

Intentional and carefully planned recruitment of candidates for advanced programs is always of importance and is especially critical now. Missouri, like many other states, is experiencing educator shortages and a great deal of turnover in nearly all certification areas. For example, data from MoDESE indicates Missouri will have 88 new superintendents in the 2022-2023 school year. This is 17 more new superintendents than one year ago and 27 more than five years ago. MoDESE reports a 36.9% five-year turnover rate in school building leadership. In addition, because the EPP hosts monthly meetings of area principal and superintendent groups, the EPP communicates regularly with school districts and, thus, is aware of leadership needs in the area. These data reflect the ongoing need to recruit candidates for the Educational Leadership and Career Education Director programs.

 

Commitment to Diversity

The EPP is committed to recruiting a diverse pool of candidates for advanced programs and acknowledges some challenges that must be addressed. According to a 2020 report from the St. Louis University PRIME center, 96% of the teachers and staff in Missouri schools are white and 78% are females (RA3.1.1 PRIME Report). Because Missouri teachers are one of the largest recruitment pools for advanced programs, it is not surprising that these same trends are seen in the Educational Leadership program's admissions data showing candidates are primarily white and the majority are female (RA3.1.2 Progression Checkpoints). However, candidates for the Career Education Director program are predominantly male. The EPP's commitment to recruiting a more diverse candidate pool is showing positive results: the 2019-2020 cohort for the Educational Leadership program included 5.8% candidates of color at admission while the 2021-2022 cohort includes 18.8% candidates of color (RA3.1.2 Progression Checkpoints). This same growth is found in the data of completers in the Educational Leadership program who are eligible for certification. The 2019-2020 cohort included 6.2% certification eligible candidates of color and the 2020-2021 cohort included 11.8% certification eligible candidates of color. Admissions data for the Career Education Director program does not consistently reflect this same trend; no candidates of color were admitted in 2019-2020 or 2021-2022 and 1 candidate of color (20% of total admissions) was admitted in 2020-2021.

 

Collaborative Recruiting

While the EPP has made progress in diversifying the candidate pool, the overall number of candidates in the Educational Leadership program has declined. The EPP is likely not unique in noticing a decline in the enrollment in advanced programs; because candidates are primarily drawn from classroom teachers who want to move into leadership roles, the greater percent of teachers currently leaving the profession means there is a smaller pool from which to draw future leaders. To address this, the EPP's advanced programs have developed a collaborative recruiting plan (RA3.1.3 Recruitment Plan). The plan includes strategies for increasing overall enrollment, and for specifically recruiting candidates who reflect the diversity of Missouri's schools. The EPP is encouraged by efforts on campus to highlight and grow graduate programs and believes this will support the recruitment efforts of advanced programs. Highlights of the plan include maintaining and expanding online course offerings to make programs accessible to a greater pool of candidates and exploring the creation of a bridge program with another university offering an EdD to attract candidates from other areas of the state.

The EPP has established requirements for admission to, progression through, and completion of advanced programs to ensure candidates are able to successfully complete the programs and are eligible for certification as educational leaders in Missouri. Some requirements are common across all graduate programs at UCM while others are EPP and program-specific.

To monitor academic achievement, the EPP assigns candidates to cohorts based on the academic year of admission to the program. For both the Educational Leadership and Career Education Director programs, candidates are tracked through three transition points: admission to the program, entry to the first internship course, and completion of the program/eligibility for certification.

 

Admissions Requirements

Both the Educational Leadership and Career Education Director programs have requirements for admission to the program. The Educational Leadership requires a minimum undergraduate GPA of 2.75 or a 3.0 GPA in the final 60 semester hours of undergraduate study. Candidates who do not meet either of these minimums may be conditionally admitted and must achieve a 3.5 GPA during their first 12 semester hours of graduate study which must include required degree courses. For conditional admission, the program also requires the submission of a summative teaching evaluation or a letter of recommendation from a current school administrator describing how the program will assist them in meeting their career goals. The mean GPA at admission exceeds the 2.75 minimum and ranges from 3.24 to 3.46. The mean GPA at admission for all subgroups is above 3.0 (RA3.2.1 Academic Achievement).

The Career Education Director program has established similar admission requirements; candidates must have a minimum undergraduate GPA of 2.5 or a 3.0 GPA in the final 60 semester hours of undergraduate study. Like the Educational Leadership program, applicants to the Career Education Director program who do not meet these criteria may be conditionally admitted and must achieve a 3.5 GPA during their first 12 semester hours of graduate study. The mean GPA at admission exceeds the 2.5 minimum (RA3.2.1 Academic Achievement) and ranges from 3.06 to 4.0. The mean GPA at admission for all candidates across all cohorts was 3.83.

 

GPA Requirements

For both the Educational Leadership and Career Education Director programs, no grade below a C may apply to the degree program and no more than six semester hours of credit with a grade of C will be applied toward degree requirements. In addition, the state of Missouri requires a 3.0 GPA for certification. Program faculty monitor the GPA of candidates and provide guidance and alternatives to any candidate struggling to achieve academically. Because of the requirements for admission to the program and the careful monitoring that occurs, candidates complete the programs successfully and are eligible for certification upon completion; the mean GPA at program completion ranges from 3.88 to 3.97 with all subgroups achieving final GPAs above 3.5 (RA3.2.1 Academic Achievement).

The EPP monitors candidate progress and provides support to facilitate candidates' advancement from admissions through completion and certification. Candidates are assigned to cohorts based on the academic year they are admitted to the program and progress is tracked based on the assigned cohort. The time a candidate requires to complete the program varies so not all candidates within a cohort move through the transition points at the same rate.

 

Key Transition Points

The Educational Leadership and Career Director programs have identified three transition points and candidates are monitored based on these. The transition points are admission to the program, entry to the first internship course, and completion of the program/eligibility for certification. The number of candidates in the cohort is monitored as the cohort progresses through these transition points. For the Educational Leadership program, progression from admission to completion is fairly consistent across cohorts. Over the past three cohorts, a total of nine candidates left the program prior to the second transition point (entry to the first internship program). The reason for candidates leaving varies; one candidate left due to GPA concern while others left for other personal or professional reasons. Progression data indicate that if a candidate successfully achieves entry to the first internship, successful completion of the program is likely. For the Career Education Director program, again progression from admission to completion is fairly consistent. Over the past three cohorts, one candidate left the program following the initial internship and returned to the classroom. One candidate is not progressing due to GPA issues but is still attempting to retake classes. Progression data indicate that if a candidate successfully achieves entry to the first internship, successful completion of the program is likely.

 

Certification and Graduation Plans

The EPP communicates the criteria required for each transition point with candidates using varied methods. Certification and graduation plans indicate coursework that is to be completed prior to enrollment in internships. Program faculty are flexible and realize plans may vary depending on the needs of specific candidates. Candidates may not enroll in internships without the consent of program faculty, and this ensures requirements are met before enrollment. Individual conferences with program faculty/advisors provide an ongoing opportunity to communicate requirements with candidates. Finally, the EPP Certification Officer must recommend candidates for Missouri certification and the recommendation involves verifying the candidate has successfully completed all requirements. This ensures no candidates can bypass EPP requirements.

 

Curriculum Map and Assessment Plan

The curriculum map and assessment plan developed by the Educational Leadership and Career Education Director programs provide information and assurance to candidates that the programs are designed to ensure candidates are prepared for leadership roles upon program completion (RA1.1.9 Assessment Plan and Curriculum Map). As the assessment plan indicates, two of the key assessments (Missouri Content Assessment [MOCA] and the Missouri Performance Assessment for Aspiring Building Administrators [MPEA-PA]) are attempted after much of coursework is complete and candidates have experienced the curriculum designed to support them in the completion of the assessments. In addition UCM's degree audit system, Central Degree Audit, is a valuable academic planning tool for candidates and faculty as it provides a complete picture regarding what degree requirements have been met, what requirements are in progress, and what requirements are remaining.

 

Faculty Support for Students

Program faculty in the Educational Leadership and Career Education Director programs provide support in a variety of manners for candidates not meeting program expectations. Regular individual conferences are held with candidates to discuss and plan to address any challenges. Generally, course instructors identify candidates of concern and bring these concerns to the attention of the program coordinators. For example, when concerns are noted on the entry and mid-program administration of the Educational Leadership Dispositions Assessment (ELDA), the faculty, district mentor, and candidate may build experiences into the candidate's internship that are designed to promote growth in the area of concern. Depending on the level of concern, the program coordinator may address the concern or may consult with other program faculty or the department chair to determine a plan of action. Program faculty may also use the university's alert system to indicate academic or behavioral concerns. Submitting an alert can initiate university level support for candidates when appropriate. Finally, program faculty provide information to candidates regarding university level resources available including the Tutoring Center, the Writing Center, the Test Prep Center, the Digital Learning Commons, the Counseling Center, and the Office of Accessibility Services. The goal of providing support is to assist candidates who are not meeting program expectations and encourage program progression, as well as to help with overall student well-being.

 

Student Concerns

The majority of student concerns or appeals the EPP receives are handled within the program and most are readily resolved. For more serious concern, the university has a grievance policy and the EPP procedures follow this policy. Grievances including grade appeals are first addressed with the program coordinator. If the issue is not resolved at this level, the appeal moves to the department chair and then the college Dean or a college grievance committee if necessary. If the candidate feels the decision of the dean or grievance committee is not sufficient, the appeal moves to the Provost for final resolution.

The EPP ensures candidates are prepared to facilitate learning and positively impact the learning and development of P-12 students. The Educational Leadership and Career Education Director programs are designed to lead to candidate competency in application of content knowledge, data literacy, use of collaborative skills, and application of technology. The EPP uses multiple measures to provide evidence of the effective preparation of and competency of candidates completing the Educational Leadership and Career Education Director programs.

The EPP's advanced programs disaggregate and monitor admission and completion data as well as performance on assessments. As noted previously, the Educational Leadership program has experienced growth in the percent of both admissions and completion of candidates of color (RA3.1.2 Progression Checkpoints). However, the majority of candidates admitted to and completing the programs are white. The Recruitment Plan (RA3.1.3 Recruitment Plan) includes a goal to increase the diversity of the candidates in the programs. The Educational Leadership program continues to admit more females than males and this is likely due to the fact that 78% of Missouri's educators are female and current educators are the pool from which the program draws most of its candidates. The Career Education Director program has greater numbers of males, since many skilled and technical areas have male instructors, but the program enrolled 38% female candidates over the past three years with this most recent year (2021-22) at 50%.

 

Formative Assessment

The EPP uses multiple sources of information to provide evidence completers are prepared for certification upon completion of the program. The programs' assessment plans provide formative and summative assessments used to measure competency in data literacy, use of research, leading and supporting collaborative efforts, developing supportive environments, applying technology, and applying professional dispositions (RA1.1.9 Assessment Plan and Curriculum Map). Formative assessments are used to provide opportunities for program faculty to provide growth feedback for candidates and to support them as they prepare for summative key assessments. The EPP selected three of these competencies for a full analysis and the formative data is provided here (RA1.1.10 CTE Data Literacy, RA1.1.11 EDAD Data Literacy 1, RA1.1.12 EDAD Data Literacy 2, RA1.1.13 CTE Data Analysis, RA1.1.14 EDAD Data Analysis 1, RA1.1.15 EDAD Data Analysis 2, RA1.1.16 CTE Collaboration, RA1.1.17 EDAD Collaboration 1, RA1.1.18 EDAD Collaboration 2).

 

Summative Assessment

The Educational Leadership and Career Education Director programs use three key summative assessments to provide evidence candidates are prepared for certification and for leadership roles. Two of the three key assessments, the Missouri Content Assessment (MOCA) and the Missouri Performance Assessment for Aspiring Building Administrators (MPEA-PA), are aligned to the Missouri Leadership Development System (MLDS) domains. The competencies within the MLDS domains are aligned with the National Educational Leadership Preparation (NELP) Standards. The third key assessment, the Educational Leadership Dispositions Assessment (ELDA) is also aligned with the NELP standards so data on the key assessments provide triangulated evidence that candidates are prepared for certification at completion.

 

Content Assessment

Candidate content knowledge is measured using the appropriate Missouri Content Assessment (MOCA) (RA1.1.4 Superintendent MOCA, RA1.1.5 Building Level Admin MOCA). Across three cycles of data, the first time pass rate for the Building Level Administrator MOCA (completed by candidates in the K-12 School Leader option of the Educational Leadership program and Career Education Director program) is 75%. The first time pass rate on the MOCA for superintendent candidates is 66.7% across three cycles. Because candidates are not required to provide information regarding passing the MOCA on later attempts, the EPP is not always made aware if candidates pass on subsequent attempts. Coursework in the programs is designed to develop the content competencies measured on the MOCA. When candidates struggle with the MOCA, program faculty are able to direct them back to course documents to assist them in preparing.

 

Performance Assessment

A second key assessment, the Missouri Performance Assessment for Aspiring Building Administrators (MPEA-PA) (RA1.1.7 MPEA-PA) provides evidence of candidates' ability to apply the content knowledge measured by the MOCA. The overall pass rate on this assessment, 99.34%, is strong evidence of the competency of candidates and their ability to achieve certification as building level leaders. The programs provide support for candidates in completing the MPEA-PA by building practice on the four steps of the assessment into the internships required in the programs. This allows candidates to receive formative feedback that enables them to be successful on the summative MPEA-PA.

 

Disposition Assessment

Finally, the third key assessment, the ELDA, provides evidence of the dispositions of candidates. In conjunction with their faculty, candidates in the Educational Leadership and Career Education Director programs complete the assessment at entry to the program, mid-program, and program completion. This provides program faculty with the opportunity to address areas of concern with candidates. On a scale of 0 to 2, scores on the entry-to-program administration of the ELDA range from 1.11(embedding vision in all decisions) to 1.79 (accepting constructive feedback) with a mean of 1.47 across all indicators. When a concern is noted for a specific disposition, program faculty and district mentors build experiences into internships to help the candidate grow in the area of concern. Scores on the completion-of-program administration of the ELDA range from 1.77 (embedding vision into all decisions) to 1.99 (creating positive relationships) with a mean of 1.91 across all indicators. The increase in scores from program entry to completion provides evidence of candidates' development of the dispositions required of effective school leaders (RA1.1.6 ELDA).

The University of Central Missouri (UCM) Educator Preparation Program (EPP) offers a wide variety of graduate programs. In conjunction with CAEP, the Missouri Department of Elementary and Secondary Education (MoDESE) defines an advanced program falling under CAEP review as one leading to a certification for which MoDESE requires an advanced degree (RA1.1.1 DESE Memo). Using this definition, UCM's Educational Leadership, Career Education Director, and Counseling programs are to be reviewed by CAEP. However, the EPP's Counseling program is accredited by the Council for Accreditation of Counseling and Related Educational Programs (CACREP) (RA1.1.2 CACREP Letter, RA1.1.3 Counseling Self-Study). Therefore, this report will focus specifically on the EPP's Educational Leadership and Career Education Director programs.

The UCM EPP prepares K-12 school leaders, superintendents, and career education directors who are well-prepared for the tasks and responsibilities associated with serving in leadership positions in school districts. The completers and those who supervise them in their leadership roles consistently report satisfaction with the preparation of the candidate

The Missouri Department of Elementary and Secondary Education (MoDESE) collaborates with the University of Missouri's Office of Social and Economic Data Analysis (OSEDA) to administer an annual survey of first-year principals and career center directors in Missouri public schools. The employers of these administrators are also surveyed. Similar surveys are administered to first year teachers and their principals. The Technical Manual and the most recent updates to it provide information about the development processes used to create the surveys, administrative procedures, and evidence of the reliability and validity of the instruments (RA4.1.1 Technical Manual, RA4.1.2 Technical Manual Updates).

The survey of the supervisors of first year principals includes 22 questions organized by Missouri Leadership Development System (MLDS) competencies (Visionary Leadership, Instructional Leadership, Managerial Leadership, Relational Leadership, and Innovative Leadership). Each item begins with "The principal was prepared to..." with supervisors asked to indicate the degree of their agreement with the statement. The items are presented with a 5-point Likert scale ranging from 5 representing "Strongly Agree" to 1 representing "Strongly Disagree." Two additional questions ask supervisors to rate the overall quality of the administrator preparation program completed by the principal and how the supervisor rates the principal's impact on the students, teachers, and community. These items also use a 5-point Likert scale ranging from 5 representing "Very good" to 1 representing "Very poor." The final two questions on the survey are open ended and allow the supervisors to provide perceived strengths of the program completed by the principal as well as perceived areas for improvement. The items on this survey are connected to the criteria in CAEP Standard R1 (the learning and learning, content, instructional practice, professional responsibilities, and technology) and highlight the importance of administrators understanding how to support teaching and learning, possessing leadership content knowledge, being knowledgeable of sound instructional practices, demonstrating knowledge of professional responsibilities, and understanding and supporting the use of technology (RA4.1.3 Connections to R1).

The EPP receives data specific to the institution as well as the data obtained from the supervisors of principals across Missouri. This allows the EPP to compare the responses specific to its programs with those reflective of all programs in Missouri. The report includes an overall mean for each of the five MLDS domains as well as the mean for each specific indicator. To ensure an adequate number of responses for data analysis, the EPP has combined data from the 2019-2020 through the 2021-2022 school years and will report the combined data for both the EPP and the state.

For each MLDS domain, an overall mean for the domain and mean on individual indicators for both the EPP and the state as a whole are provided. The supervisors of the EPP's completers rated their principals positively with the overall means for the domains ranging from 4.57 to 4.67 out of 5. Across the state, the range of the overall means for the domains ranged from 4.31 to 4.41 out of 5. The ratings for the EPP's completers were higher than the state average for each dimension mean. The EPP's completers were also rated higher than the state mean on each individual indicator within the domains. Scores on individual indicators ranged from 4.39 to 4.76/5 (RA4.1.4 Supervisors of First Year Admin). The four most highly rated items were "The principal was prepared to maintain a safe learning environment for the school community" (4.71/5), "The principal was prepared to establish a culture that nurtures positive relationships" (4.71/5), "The principal was prepared to support positive relationships with community members" (4.73/5), and "The principal was prepared to model personal and professional ethical behavior" (4.76/5). This strength in relationship-building is also reflected in the responses to the open-ended question asking the supervisors to share perceived strengths of the preparation program completed by the principal: 22.6% of the responding superintendents listed developing and maintaining relationships as a strength of the EPP's program. The two lowest rated items were "The principal was prepared to identify key stakeholders in his/her community" (4.39/5) and "The principal was prepared to facilitate community support networks to impact student learning" (4.42/5). Although these are the lowest items for the EPP completers, they are both higher than the mean for all Missouri completers on each of these indicators. A recent curricular change moved EDAD 6160: School and Community Relations to become a required course in the Educational Leadership, K-12 School Leader option. This change was based on MOCA data as well as the feedback from advisory groups and past students. Some of the students who completed the state surveys may have completed the program prior to this course change resulting in a lower score in the survey. The EPP realizes the enhanced need for communication with parents and community members came to the forefront during the pandemic.

Additionally, the mean of the ratings of the supervisors of the EPP's completers to the question "What overall rating would you give the quality of the administrator preparation program your principal completed?" was 4.66 out of 5 compared to 4.50 for the state as a whole. The lowest rating received by the EPP's first year administrators was in response to the question "Based upon the performance based evaluation of this first year principal, how would you rate his/her impact upon students, teachers, and school community?" The EPP's candidates were rated 3.63 out of 5 while the state mean was 3.5. The EPP's program faculty attribute part of this lower score to the impact of the pandemic and the challenges faced by administrators. Of course, this is true across the state and is reflected in the lowest state mean occurring on this same item. The most frequent response to the open ended question "What are some areas for improvement of the Educational Leadership program that your principal completed?" was "No suggestions" (25.8%). However, 16.1% of the supervisors mentioned strengthening preparation in teacher evaluation and coaching as an area for improvement and program faculty will work with advisory boards to address this recommendation.

While the EPP values the data provided through this state-administered survey and believes the data indicate the quality of the EPP's programs, program faculty acknowledge the limited context and depth the items allow. Because of this, both the Educational Leadership and Career Education Director programs have plans in place to organize focus groups of employers of completers and current advisory groups to add to the data from the survey items. Area superintendents meet regularly on the EPP's campus and this will enable program faculty to host focus groups easily and conveniently for the superintendents.

The EPP collects data to learn of the perception of completers regarding their preparation to fulfill the responsibilities of serving as a school leader. Data from exit surveys and state-administered surveys of first year administrators provide evidence that the EPP's completers are prepared for their leadership roles.

As previously noted, MoDESE collaborates with the University of Missouri's OSEDA to administer an annual survey of first-year principals in Missouri public schools. The survey of first year principals includes 22 questions organized by Missouri Leadership Development System (MLDS) competencies (Visionary Leadership, Instructional Leadership, Managerial Leadership, Relational Leadership, and Innovative Leadership). The items on the survey mirror those on the survey of supervisors of first-year administrators as described previously. For example, an item from the Relational Leadership domain is phrased "The principal was prepared to establish a culture that nurtures positive relationships" on the survey of supervisors and as "I was prepared to establish a culture that nurtures positive relationships" on the survey of first year administrators. The items are presented with the same 5-point Likert scale previously described. An additional question asks new administrators to rate the overall quality of the administrator preparation program they completed using a 5-point Likert scale ranging from 5 "Very good" to 1 "Very poor." The final two questions on the survey are open-ended and allow the administrators to provide perceived strengths of the program as well as perceived areas for improvement. Because the items mirror those on the survey of supervisors, connection to CAEP Standard R1 is strong (RA4.1.3 Connections to R1).

As is true of the survey of supervisors of first year administrators, the EPP receives survey response data specific to the EPP's completers as well as statewide data. To ensure an adequate number of responses for data analysis, the EPP has combined data from the 2019-2020 through the 2021-2022 school years and will report the combined data for both the EPP and the state.

The data obtained from the EPP's completers indicate they were well-prepared for their work in educational leadership positions. Overall means for the EPP completers on the MLDS domains ranged from 4.35 out of 5 on both Instructional and Managerial Leadership to 4.54 for Innovative Leadership. Overall state means on the MLDS domains ranged from 4.10 to 4.30 and each domain mean for the state was lower than the mean for the EPP on that domain. For each individual item, the EPP mean was higher than the statewide mean. Within the EPP's completer data, the two most highly rated indicators were "I was prepared to model personal and professional ethical behavior " (4.66/5) and "I was prepared to establish a culture that nurtures positive relationships" (4.64/5). These indicators were also among the most highly rated on the survey of supervisors of EPP completers (RA4.2.1 First Year Admin). The four lowest rated items were "I was prepared to build partnerships with community members" (4.09/5), and "I was prepared to facilitate community support networks to impact student learning" (4.15/5), "I was prepared to work with personnel to develop growth plans for improvement of student learning" (4.17/5) and "I was prepared to facilitate effective evaluation processes" (4.19/5). Although these are the lowest items for the EPP, they are each higher than means for all Missouri completers on each of these indicators. The EPP believes the previously described curricular change moving EDAD 6160: School and Community Relations to a required course in the K-12 School Leader option will help address this. The EPP will monitor ratings on the two indicators related to working with personnel to develop growth plans and to facilitating effective evaluation processes. While the pandemic may have contributed to a lack of confidence in these areas, if ratings remain low program faculty will review curriculum and make changes as needed.

When asked "What overall rating would you give the quality of the administrator preparation program you completed?" the EPP's completers' rating was 4.57 out of 5 as compared to the statewide mean of 4.28 out of 5. The EPP's completers suggested that the opportunity for collaboration within the program (41.4%), thorough preparation (24.1%), and the practical nature of the program (24.1%) are strengths of the EPP's program. While "No suggestions" (20.7%) was the most frequent response to the question asking for areas for improvement, 10.3% of the completers mentioned handling difficult conversations, increased clinical experiences, and additional opportunities for networking as areas for improvement.

Survey data for the EPP's completers and their supervisors were compared (RA4.2.2 Comparison of Admin and Super). Of note, the supervisors rated the new administrators higher on each item than did the administrators themselves. The EPP believes this may be partially due to the increased challenges administrators faced as a result of the pandemic and for new administrators these challenges would have been magnified.

In the Spring of 2021, the Educational Leadership and Career Education Director programs implemented an exit survey to gather data on the perceptions of candidates immediately upon completion of the programs. The items on the exit survey are identical to those on the state administered survey of first year administrators and responses are gathered using the same 5-points Likert scale used on the state survey. Thirty-nine candidates completed the survey in Spring 2021 and the EPP is currently collecting data from Spring and Summer 2022 completers (RA4.2.3 Exit Survey). As more cycles of data are completed, the EPP will be able to monitor trends in completers' perceptions of preparation. In addition, a comparison of completers' perceptions immediately upon completion with their perceptions after a year in the leadership position will be possible. This additional information will be valuable in guiding changes in curriculum to even better prepare candidates.

The University of Central Missouri (UCM) Educator Preparation Program (EPP) offers a wide variety of graduate programs. In conjunction with CAEP, the Missouri Department of Elementary and Secondary Education (MoDESE) defines an advanced program falling under CAEP review as one leading to a certification for which MoDESE requires an advanced degree (RA1.1.1 DESE Memo). Using this definition, UCM's Educational Leadership, Career Education Director, and Counseling programs are to be reviewed by CAEP. However, the EPP's Counseling program is accredited by the Council for Accreditation of Counseling and Related Educational Programs (CACREP) (RA1.1.2 CACREP Letter, RA1.1.3 Counseling Self-Study). Therefore, this report will focus specifically on the EPP's Educational Leadership and Career Education Director programs.

The Quality Assurance System (QAS) developed and utilized by the Education Preparation Program (EPP) at the University of Central Missouri (UCM) provides a sustainable process to review and document operational effectiveness. Because the EPP continues to respond to changes (at the institutional, local, regional, state, and national levels) and to feedback (from completers, stakeholders, and partners), the QAS continues to evolve to address these changes and to ensure continuous improvement. The EPP's QAS includes both initial and advanced programs and this is reflected in the similarities in the Standard 5 narratives for each.

The UCM EPP's QAS is a model for assessment that addresses two specific reasons the EPP conducts assessment. First, the EPP conducts program area level assessment to determine candidate achievement of the specific learning outcomes for their area of study. Data collected provides evidence that guide changes at the programmatic level. Second, the EPP conducts unit-wide assessment to ensure candidates are developing the knowledge, skills, and dispositions required of all educators. Unit-wide data collected provide the evidence required to guide changes at the unit level. Both program area and unit assessment provide evidence to meet the needs of accrediting bodies such as CAEP and HLC.

 

College Level

The QAS is monitored by the Quality Assurance Workgroup (QAW), a group with membership representing both initial and advanced programs. The Dean of the College of Education (COE) is a member of this group as is the chair of the College of Education Assessment Committee (COEAC). The QAW meets regularly to ensure the QAS is functioning to promote continuous improvement (R5.1.1 Selected Minutes QAW) The QAW led the development of the Continuous Improvement Cycle (CIC) that forms the basis of program area and unit level assessment (R5.1.2 CI and Unit Data Cycles). The CIC reflects the importance of using data as a basis for change at the program area and unit levels, and the need for this process to be ongoing, systematic, and continuous. After careful review by the Professional Education Faculty (PEF) and other stakeholders, this CIC was adopted by the EPP's Teacher Education Council (TEC) in Fall 2021.

The QAS includes two parallel and closely related data cycles with responsibility for oversight falling to two groups. Unit level assessment and operations are monitored by the QAW. The QAW reviews unit processes related to the QAS, oversees the administration of key assessments, and oversees data collection, analysis, and documentation at the unit level. In conjunction with the Dean's Office, the QAW plans and conducts two meetings of the PEF each year. Program level assessment is monitored by the COEAC with one member from each of the COE departments. The Coordinator of College of Education Assessment is a member of this committee. This committee reviews and provides feedback on program area assessment and provides professional development to improve assessment plans and processes.

 

Unit Level

At the unit level, the data cycle is an ongoing process overseen by the QAW with multiple levels of review and reporting (R5.1.2 CI and Unit Data Cycles). The COE Coordinator of Data Management and Technology (CDMT) and the QAW compile unit wide data for the academic year and share the data with the COE Advisory Board, the PEF, and the Secondary and K-12 Group. The CDMT is a newly created position to provide technical support and expertise in data collection, analysis, and management. The CDMT disaggregates the data by program area and shares program data with department chairs and the appropriate program coordinators. The CDMT provides the QAW with unit-wide data disaggregated by race, gender, first generation status, program type, and certification area. After receiving feedback from the COE Advisory Board (R2.1.6 COE Advisory Board Minutes) and the PEF (R5.1.3 PEF Minutes), the QAW prepares an executive summary (R5.1.4 Executive Summary) of the academic year data and shares this with the COE Advisory Board, PEF, and TEC. The TEC uses this summary to provide data-based recommendations to the Dean of the COE, the head of the EPP unit.

 

Program Level

At the program level, the ongoing process reflected in the data cycle is overseen by the COEAC and includes levels of review and reporting (R5.1.5 Program Data Cycle and Schedule). Data on key assessments are collected in the fall and spring semesters. The CDMT shares this data disaggregated by program, gender, race, first generation status, and program type with the program coordinators. Simultaneously, the program coordinators collect program-specific assessment data. The program coordinators compile key assessment and program-specific assessment data and share the data with program faculty and advisory board members. Using input from these groups, the program coordinators enter a program report into the UCM assessment management system (Nuventive). These reports are entered annually and the report and data are accessible to program faculty, unit administrators, and university administrators for review. A formal review of program assessment reports is conducted by the COEAC on a three year cycle (R5.1.5 Program Data Cycle and Schedule) and, for advanced programs, by the UCM Graduate Council on a five year cycle. In addition, faculty for the EPP advanced programs under CAEP review, the Educational Leadership and Career Education Director programs, meet regularly to discuss the common key assessments used and data collected from them (RA5.1.1 Compiled Meeting Minutes). This provides an extra layer of review for the EPP's advanced programs.

The data collected from the EPP's key assessments are an integral part of the QAS (RA5.1.2 Assessment Grid, RA1.1.4 Superintendent MOCA, RA1.1.5 Building Level Admin MOCA, RA1.1.6 ELDA, RA1.1.7 MPEA-PA). The key assessments used by the EPP advanced programs are aligned to the Missouri Leadership Development System (MLDS) standards which were created from the National Policy Board of Educational Administrators (MPBEA) including the National Educational Leadership Preparation (NELP) standards/Professional Standards for Education Leaders (PSEL) (RA1.1.8 Standards Crosswalk). The MLDS Standards consist of 32 competencies organized into five domains: Visionary Leadership, Managerial Leadership, Instructional Leadership, Relational Leadership, and Innovative Leadership. Aggregate and disaggregated data from these assessments provide the basis for continual review and improvement of unit and advanced program level processes, and for decision making at the unit and program levels.

EPP program faculty play an important role in the quality assurance process and shared beliefs about the importance of assessment is key to continuous improvement. The COEAC has worked to create a mindset of valuing assessment and using data as part of the process of continual improvement. The committee distributed a Culture of Assessment Survey to all PEF members and the results of the survey are being used to plan professional development for the upcoming year (R5.1.6 Culture of Assessment Survey).

UCM's EPP works to ensure all interpretations of data are valid and consistent through a reliance on relevant, actionable, verifiable, representative, cumulative, and actionable measures. Measures include proprietary instruments required by the state and EPP adopted or developed measures (RA5.1.2 Assessment Grid). Multiple measures are used to gather data on the knowledge, skills, and dispositions of candidates. Formative assessments used in the Educational Leadership and Career Education Director programs are course-embedded assignments. As is typical for these types of formative assessments, the assessments include less attention to quantitative reliability and validity metrics, but have good face validity and practical utility. The summative measures used for the EPP's advanced programs have established reliability and validity.

 

Relevant

Two of the summative measures used by the EPP for advanced programs are aligned to the MLDS Standards. The MLDS Standards include 32 competencies organized into five domains and provide a description of the essential knowledge, performances, and critical dispositions required for effective educational leadership that supports learning and learner development. The MLDS Standards are aligned to the Professional Standards for Education Leaders (PSEL) (RA1.1.8 Standards Crosswalk). Because of the alignment to the MLDS, the EPP ensures there is a clear link between what is measured on each assessment and what the EPP intends to measure. The QAS provides for the ongoing review of assessments and the data they produce.

 

Verifiable

Valid and reliable measures are essential to ensure quality. The EPP uses a mix of propriety and EPP adopted key summative assessments at the advanced level (RA5.1.2 Assessment Grid). The proprietary measures used by the EPP, the MOCA and the MPEA-PA, are required by the state of Missouri and work at the state level provides evidence of validity and reliability (RA1.1.4 Superintendent MOCA, RA1.1.5 Building Level Admin MOCA, RA1.1.7 MPEA-PA). The Missouri Professors of Educational Administration (MPEA) is authorized by the MoDESE to develop, administer, score, and facilitate quality control of the performance assessment (MPEA-PA) required for certification. MPEA offers bi-annual professional development conferences to ensure the assessment is consistent and valid. Annual scorer training and calibration are required for every participating EPP. The Educational Leadership Dispositions Assessment (ELDA) was adopted by the EPP as a measure of the professional dispositions of educational leaders and is administered at three points in the program: entry, mid-program, and completion. The ELDA was originally developed at the University of Tampa and the ELDA technical guide provides evidence of validity and reliability (RA5.2.1 ELDA Technical Manual, RA1.1.6 ELDA). The EPP advanced programs administer an exit survey to learn of the perceptions of candidates immediately upon completion of the program (RA4.2.3 Exit Survey). The QAW applied the CAEP criteria for EPP-created assessments to the Exit Survey and made no recommendations for changes in content (RA5.2.2 Sufficiency Criteria Exit Survey).

 

Representative

The data collected on summative assessments are representative of all candidates in the Educational Leadership and Career Education Director programs. Summative assessments are administered to all candidates regardless of program and this ensures analysis of the data encompasses all programs. While candidates take the MOCA appropriate for desired certification, every candidate must pass the appropriate MOCA for certification. For all other key assessments, all candidates complete the same version of the assessment.

 

Cumulative

Because summative assessments are administered every cycle, the EPP is able to view and analyze data across successive administrations for advanced programs under CAEP review.

 

Actionable

The data generated through the administration of summative assessments provide a foundation for EPP decision making. The EPP utilized the feedback from the NCATE Legacy Report in 2016 to make a number of updates to advanced programs. The EPP now regularly reports data each year and utilizes data to improve unit operations. For example, the Educational Leadership and Career Education Director programs, along with stakeholders, routinely examine data and make decisions in response. These programs follow the data cycle plan described previously. Dispositions are now assessed at three points throughout the programs using the ELDA. Data from the ELDA are used to inform candidates of their strengths and areas for improvement (RA5.4.2 Advanced Data Based Decisions).

The UCM EPP engages both internal and external stakeholders in program design, evaluation, and continuous improvement processes. This is accomplished internally through the work of groups such as the Professional Education Faculty (PEF) and the Teacher Education Council (TEC). Each of these groups provides input that is vital to the continuous improvement of the EPP. Externally, stakeholders participate in a variety of advisory groups including the College of Education Advisory Board and program advisory boards. The advisory boards include members who hold a variety of roles and responsibilities and, thus, provide varied perspectives to the EPP (RA2.1.1 EDAD Advisory Boards, RA5.3.1 COE Advisory Board, RA2.1.7 CTE Advisory Board).

Both internal and external stakeholders provided feedback that led to the revision of the EPP's mission, vision, and goals. The College of Education Advisory Board provided input via a survey in Spring 2021 sharing the knowledge, skills, and dispositions needed by educators. This input was used and is evident in the goals developed and adopted by the EPP (R5.3.1 Mission Vision Goals). The PEF also reviewed and provided feedback on the EPP conceptual framework that includes the mission, vision, and goals at the Spring 2021 meeting (R5.1.3 PEF Minutes). The PEF reviewed the revised mission, vision, and goals in Fall 2021 before they were presented to the TEC for adoption. The internal and external feedback received was vital in creating the mission, vision, and goals that are the foundation of the EPP.

 

Assessment Design

The Professors of Educational Administration (MPEA), including faculty from 17 Missouri universities' EPP programs, collaboratively developed the first version of the MPEA-PA, which was approved by the educator certification unit at MoDESE. Faculty (including UCM EPP faculty) also collaborated for the two revisions that were driven by stakeholder feedback and the need to better assess MLDS components. Quality assurance is maintained by yearly collaboration training conducted by MPEA and faculty attend these training sessions.

 

Advisory Boards

The Educational Leadership and Career Education Director programs have advisory boards that meet regularly and provide valuable guidance to program leadership. The advisory boards' memberships reflect rural, urban, and suburban school districts of varying sizes. Both programs utilize the expertise of their partners to collaboratively design meaningful clinical experiences for candidates.

The Educational Leadership program has established two advisory boards that play complementary roles in supporting the program. One board is composed of current district principals, many of whom are program graduates, who provide guidance and support for the K-12 School Leader program option. Members of the other advisory board are current school superintendents who are able to provide a perspective different from that of the principals. The composition of the boards is intentionally designed to include diverse representation in district size and nature (RA2.1.1 EDAD Advisory Boards). Minutes from the meetings of the advisory boards provide evidence of the valuable role partners play in providing feedback to the EPP (RA2.1.2 EDAD Advisory Minutes).

The Career and Technical Education (CTE) Graduate Programs Advisory Board provides input and guidance to the Career Education Director program. The CTE Advisory Board has diverse membership including members from Area Career Centers, school districts, MoDESE, graduates of the program, and current students (RA2.1.7 CTE Advisory Board). Minutes from this group's meetings are evidence of the importance of the guidance provided by this group (RA2.1.8 CTE Minutes). 

 

Clinical Experiences

Internal and external stakeholders also provide guidance on clinical experiences. In conjunction with EPP faculty, the districts hosting the advanced programs internship identify an on-site mentor to work with the candidate during the internship. For all internships, the clinical experience is designed through collaboration among the candidate, the on-site mentor, and the EPP faculty member who together determine the tasks to be completed, plan for the completion of the tasks, and evaluate task completion. As such, the clinical experience of each candidate in the Educational Leadership and Career Education Director programs is individually constructed to meet the needs of the candidate and the district hosting the candidate for the internship.

As an example of stakeholder involvement in the design of clinical experiences, recently the Superintendent Advisory Board provided feedback on potential topics and tasks to be included in internship courses. The Board was presented with a list of the potential topics/tasks and identified those tasks and topics that were essential for inclusion in an internship (RA2.1.3 Internship Essentials, RA2.1.4 Feedback on Internship). The candidate, district mentor, and EPP faculty use this prioritized list to plan internship tasks to be completed.

The UCM EPP is committed to developing and maintaining a culture in which all stakeholders, both internal and external, are involved in an ongoing review of the processes of the unit and the performance of the EPP's candidates. Because of this commitment to continuous improvement, the EPP has established a structure that supports a set of systems and processes to assess EPP performance towards achieving its goals and meeting relevant standards.

 

Continuous Improvement Process

The EPP has adopted a continuous improvement process (R5.1.2 CI and Unit Data Cycles) that provides the foundation for ongoing review and data-based revision as needed of processes and systems. Within this cycle of continuous improvement, data review processes established by the EPP are in place to gather, input, analyze, interpret, and use information from the QAS effectively (R5.1.2 CI and Unit Data Cycles). Data on key assessments is collected in the fall and spring semesters. Aggregate unit-wide data and data disaggregated by program, program type, race, gender, and first-generation status are shared with administrators, program coordinators, program faculty, unit and program advisory boards, and the PEF. After review of the data, feedback is gathered and analyzed by the QAW; the QAW then creates an executive summary based on their analysis to be shared with the TEC, PEF, and advisory boards. The TEC as the recommending body of the unit uses the analysis included in the executive summary to provide recommendations to the Dean and the unit as a whole. A parallel process at the program level allows additional review, analysis, and interpretation of program specific data and processes (R5.1.2 CI and Unit Data Cycles).

 

CAEP Aligned Goals

The EPP has adopted a set of goals that fall under five areas aligned with CAEP standards. These five areas also form the structure for the College of Education's Strategic Plan currently under development. The EPP assesses progress towards these goals, identifies completed steps, and proposes next steps needed to work toward achieving the goals (RA5.4.1 Progress Goals). Because continuous improvement must be ongoing and always a priority, the EPP acknowledges the goals in this area will not and should not ever be considered fully met. What follows is a summary of steps taken by the EPP's advanced programs to meet the EPP's goals.

Goal Area 1 - Professional Knowledge, Skills, and Dispositions

Advanced programs have worked to integrate educational technology throughout the program of study and aligned with ISTE standards. Upcoming goals include creating a technology survey for school administrators to determine what new technologies are being utilized in school districts for updates to coursework. Advanced programs have also integrated best practices in supporting special needs students and classroom/behavior management. Monitoring and staying current in these areas, as well as adding new resources and coursework on trauma informed practices are next steps in moving forward. Advanced programs are also reviewing the MPEA-PA to ensure it is the best measure as a summative assessment to align with employing data analysis to develop supportive, diverse, equitable, and inclusive school environments. While this was the selected measure over these past three data cycles, the EPP is planning to review this measure, along with the data with stakeholders (RA5.4.1 Progress Goals).

Goal Area 2 - Student Recruitment, Retention, and Success

The EPP advanced programs plan to continue current recruitment efforts, while also exploring and engaging with organizations to increase the number of candidates of diverse backgrounds. The EPP will also continue to communicate with candidates to share information and will monitor candidate progress through regular checkpoints (RA3.1.3 Recruitment Plan, RA5.4.1 Progress Goals).

Goal Area 3 - Access, Opportunity and Community

The EPP has formed a diversity workgroup to make recommendations to create more inclusive environments. For advanced programs, the EPP's statement on diversity (R1.1.10 Diversity Workgroup) and the MLDS diversity statements are guiding course updates and internship experiences. The EPP plans to implement a diversity survey to ensure inclusive practices and that students monitor their own biases. Advanced programs are also reviewing the unit definition of diversity for alignment with the MLDS diversity standards, as well as exploring additional professional development opportunities for advanced candidates and faculty. The EPP also plans to review coursework to ensure that candidates have the knowledge to support differentiation across learning environments (RA5.4.1 Progress Goals).

Goal Area 4 - Collaborative Partnerships

Program advisory boards at the advanced level meet regularly with collaborative partners who guide coursework by providing input to desired knowledge, skills, and dispositions of administrators employed by their districts. Statewide partnerships provide opportunities to collaborate across institutions and organizations. The EPP plans to continue providing training and support for mentors, and soliciting feedback from partners on ways to benefit the field (RA5.4.1 Progress Goals). 

Goal Area 5 - Continuous Improvement

The EPP has established a Quality Assurance Workgroup to provide oversight of unit operations and assessment. A two-year cycle has been employed to administer a Culture of Assessment Survey. The results guide training and communication efforts for faculty to maintain a culture of assessment within the College of Education. Documentation of this cycle has been developed to ensure faculty participation and knowledge of assessment plans. Stakeholder collaboration is also embedded into the Continuous Improvement Cycle. Advanced programs will continue to utilize advisory boards in improvement efforts and value the role stakeholders play in educator preparation (RA5.4.1 Progress Goals).