The research design used in this study was quasi experimental with multiple quantitative analytic techniques including Single Group Pretest Posttest Design (Campbell & Stanley,1963) and Single Cross-Sectional Interrupted Time Series (Glass, Willson, & Gottman, 1975). The research design section is organized by research question and research design. That is, each research question is discussed utilizing one of the research designs referenced earlier. See Table 8 for details.

Research Questions-Design-Measures for the Study

This study was conducted with three specific time frames in relation to measures and principal-teacher interactions:

1.      Prior to the pilot year (prior to fall 2007)

2.      Pilot year (2007-2008 school year)

3.      Year of full implementation (2008-2009 school year)

Two principal-teacher interactions (snapshots and data reviews) were implemented during the pilot year and the full set of four principal-teacher interactions (one-on-one summer meetings, snapshots, data reviews, and teacher self-assessment) were implemented in the year of full implementation. Student grade and discipline data were available and collected from each of these three time frames. Teacher and student surveys data were only available in the two time frames of the pilot year and the year of full implementation. Principal and teacher QIR data were only available in the year of full implementation. See Table 9 for details on available date for each time period.

Available Data for Associated Time Periods of the Study

Research Question One (Principal & Teacher QIR)

Procedures. Before and after the year of full implementation principal and teacher QIRs were completed for all teachers. In addition, in August 2008 and again in May 2009, each teacher completed a QIR as an assessment of their own instructional practices. At the same time each of the four principals independently completed a QIR for each teacher as an assessment of instructional practices as referenced in the treatment and measures section of this chapter.

Research design. The principal-teacher interactions and QIR data were used in a single group pretest posttest research design as modeled in Figure 2.

Figure 2 Exploration of Impacts of Treatment on Teacher Instructional Practices

Research Question Two (Classroom Grade Distributions & Classroom Discipline Referrals)

Procedures. The researchers collected four years of student performance data, classroom grade distributions and classroom discipline referrals prior to the pilot year in order to obtain a baseline of performance. Establishing a baseline of student performance data provided support for a connection between changes in student performance to the principal-teacher interactions.

Classroom grade distributions and discipline referrals were collected and analyzed for the six school years of 2003-2004 through 2008-2009, prior to the pilot year, during the pilot year, and during the year of full implementation. Two of the principal-teacher interactions associated with treatment investigated in this study (snapshots and data reviews) were implemented during the pilot year (2007-2008) and continued through the year of full implementation (2008-2009). The second two principal-teacher interactions associated with the treatment investigated in this study (summer meetings and teacher self-assessment) were implemented during the summer before and during the year of full implementation (2008-2009). For the purpose of this single cross-sectional interrupted time series design, student performance measures, classroom grade distributions and discipline referrals from school years 2003-2004, 2004-2005, 2005-2006, and 2006-2007 were used as pretreatment data. Student performance measures from school years 2007-2008 and 2008-2009 were used as post treatment data.

Research design. In order to detect changes in student performance, single cross-sectional interrupted time series, as modeled in Figure 3 was used to analyze data collected from classroom grade distributions and classroom discipline referrals.

                         -Observations                                        - Principal-Teacher Interactions

Figure 3 Exploration of Impacts of Treatment on Student Performance

Research Question Three (Teacher & Student Surveys)

Procedures. Teacher surveys were sent electronically to all teachers, and student surveys were given to random samples of 200 students, during each of the three specific time frames for this research in order to utilize a single group pretest posttest research design as modeled in Figures 4 and 5. The same teacher and student surveys were administered electronically in May 2007 as the pre-test and then again in May 2008 at the end of the pilot year (post-test for pilot year, pretest for the year of full implementation). These same surveys were given to the same teachers and a different random sample of students again in May 2009 (post-test for the year of full implementation).

Research design. The assumption associated with the use of single group pretest posttest design was that the pretest scores were representative of the group in years prior to the treatment being implemented. As discussed in chapter two, evidence supports that without a change agent teacher behavior reflects little change from year to year (Marshall, 2003; Toch & Rothman, 2008).  

As modeled in Figures 4 and 5, student and teacher surveys from before and after the pilot year as well as before and after the year of full implementation were used in the single group pretest posttest design as measures of the frequency and focus of teacher conversations. Data from student and teacher surveys was used specifically to analyze the changes in the frequency and focus of principal-teacher conversations, teacher-teacher conversations, and teacher-student conversations.

Figure 4 Exploration of Impacts of Treatment on the Frequency and Focus of Teacher Conversations During the Pilot Year

Figure 5 Exploration of Impacts of Treatment on the Frequency and Focus of Teacher Conversations During the Year of Full Implementation

The year by year analysis from Figure 4 permitted exploration of changes in the frequency and focus of teacher conversations that occurred over the course of the pilot year. The year by year analysis from Figure 5 permitted exploration of changes in the frequency and focus of teacher conversations that occurred over the course of the year of full implementation.