Research Design        

A study of how specific principal behaviors affect teacher and student performance

 


Home
Dissertation Defense
Introduction
Significant Quotes
Context
Origin of Study
Conceptual Framework
Literature Review
Research Design
Measures & Insturments
Treatments
Data & Analysis
Support Documents
Results
Discussion
About Authors
Contact Information

The research design used in this study was quasi experimental with multiple quantitative analytic techniques including Single Group Pretest Posttest Design (Campbell & Stanley,1963) and Single Cross-Sectional Interrupted Time Series (Glass, Willson, & Gottman, 1975). The research design section is organized by research question and research design. That is, each research question is discussed utilizing one of the research designs referenced earlier. See Table 8 for details.


Table 8

Research Questions-Design-Measures for the Study

 

Research Question

Design

Measures

       

RQ1

How will the set of principal-teacher interactions affect teacherís instructional practices?

Single Group Pretest Posttest Design

 

Principal and Teacher QIR

 

RQ2

How will the changes in teachersí instructional practices, initiated by the set of principal-teacher interactions, affect student performance?

Single Cross-Sectional Interrupted Time Series

Longitudinal Classroom Grade Distributions and Student Discipline Referrals

       

RQ3

How will the changes in principal-teacher interactions, affect the frequency and focus of teacher conversations?

Single Group Pretest-Midtest- Posttest Design

Student and Teacher Surveys

 

This study was conducted with three specific time frames in relation to measures and principal-teacher interactions:

1.      Prior to the pilot year (prior to fall 2007)

2.      Pilot year (2007-2008 school year)

3.      Year of full implementation (2008-2009 school year)

Two principal-teacher interactions (snapshots and data reviews) were implemented during the pilot year and the full set of four principal-teacher interactions (one-on-one summer meetings, snapshots, data reviews, and teacher self-assessment) were implemented in the year of full implementation. Student grade and discipline data were available and collected from each of these three time frames. Teacher and student surveys data were only available in the two time frames of the pilot year and the year of full implementation. Principal and teacher QIR data were only available in the year of full implementation. See Table 9 for details on available date for each time period.


Table 9

Available Data for Associated Time Periods of the Study

 

Principal and Teacher QIR

Classroom Grade Distributions and Discipline Referrals

Student and Teacher Surveys

 

(Measures for RQ1)

Single Group Pretest Posttest Design

(Measures for RQ2)

Single Cross-Sectional Interrupted Time Series

(Measures for RQ3)

Single Group Pretest Posttest Design

Prior to Pilot Year

 

X

 

(2003-2004 through 2006-2007)

       

Pilot Year

 

X

X

(2007-2008 School Year)

       

Year of Full Implementation

X

X

X

(2008-2009 School Year)

Research Question One (Principal & Teacher QIR)

Procedures. Before and after the year of full implementation principal and teacher QIRs were completed for all teachers. In addition, in August 2008 and again in May 2009, each teacher completed a QIR as an assessment of their own instructional practices. At the same time each of the four principals independently completed a QIR for each teacher as an assessment of instructional practices as referenced in the treatment and measures section of this chapter.

Research design. The principal-teacher interactions and QIR data were used in a single group pretest posttest research design as modeled in Figure 2.

Figure 2 Exploration of Impacts of Treatment on Teacher Instructional Practices

 

Research Question Two (Classroom Grade Distributions & Classroom Discipline Referrals)

 

Procedures. The researchers collected four years of student performance data, classroom grade distributions and classroom discipline referrals prior to the pilot year in order to obtain a baseline of performance. Establishing a baseline of student performance data provided support for a connection between changes in student performance to the principal-teacher interactions.

Classroom grade distributions and discipline referrals were collected and analyzed for the six school years of 2003-2004 through 2008-2009, prior to the pilot year, during the pilot year, and during the year of full implementation. Two of the principal-teacher interactions associated with treatment investigated in this study (snapshots and data reviews) were implemented during the pilot year (2007-2008) and continued through the year of full implementation (2008-2009). The second two principal-teacher interactions associated with the treatment investigated in this study (summer meetings and teacher self-assessment) were implemented during the summer before and during the year of full implementation (2008-2009). For the purpose of this single cross-sectional interrupted time series design, student performance measures, classroom grade distributions and discipline referrals from school years 2003-2004, 2004-2005, 2005-2006, and 2006-2007 were used as pretreatment data. Student performance measures from school years 2007-2008 and 2008-2009 were used as post treatment data.

Research design. In order to detect changes in student performance, single cross-sectional interrupted time series, as modeled in Figure 3 was used to analyze data collected from classroom grade distributions and classroom discipline referrals.

 

Oval:  

                         -Observations                                        - Principal-Teacher Interactions

Figure 3 Exploration of Impacts of Treatment on Student Performance

 

Research Question Three (Teacher & Student Surveys)

Procedures. Teacher surveys were sent electronically to all teachers, and student surveys were given to random samples of 200 students, during each of the three specific time frames for this research in order to utilize a single group pretest posttest research design as modeled in Figures 4 and 5. The same teacher and student surveys were administered electronically in May 2007 as the pre-test and then again in May 2008 at the end of the pilot year (post-test for pilot year, pretest for the year of full implementation). These same surveys were given to the same teachers and a different random sample of students again in May 2009 (post-test for the year of full implementation).

Research design. The assumption associated with the use of single group pretest posttest design was that the pretest scores were representative of the group in years prior to the treatment being implemented. As discussed in chapter two, evidence supports that without a change agent teacher behavior reflects little change from year to year (Marshall, 2003; Toch & Rothman, 2008).  

As modeled in Figures 4 and 5, student and teacher surveys from before and after the pilot year as well as before and after the year of full implementation were used in the single group pretest posttest design as measures of the frequency and focus of teacher conversations. Data from student and teacher surveys was used specifically to analyze the changes in the frequency and focus of principal-teacher conversations, teacher-teacher conversations, and teacher-student conversations.

Rounded Rectangle: Pilot Year

Figure 4 Exploration of Impacts of Treatment on the Frequency and Focus of Teacher Conversations During the Pilot Year

Rounded Rectangle: Year of Full Implementation

Figure 5 Exploration of Impacts of Treatment on the Frequency and Focus of Teacher Conversations During the Year of Full Implementation

 

The year by year analysis from Figure 4 permitted exploration of changes in the frequency and focus of teacher conversations that occurred over the course of the pilot year. The year by year analysis from Figure 5 permitted exploration of changes in the frequency and focus of teacher conversations that occurred over the course of the year of full implementation.


Home | Dissertation Defense | Introduction | Significant Quotes | Context | Origin of Study | Conceptual Framework | Literature Review | Research Design | Measures & Insturments | Treatments | Data & Analysis | Support Documents | Results | Discussion | About Authors | Contact Information

 This research project is sponsored by the University of Louisville, The Kenton County School District, & the head research (Kim Banta/Brennon Sapp)
For problems or questions regarding this Web site contact bsapp@bsapp.com or kim.banta@kenton.kyschools.us
Last updated: 07/09/10.