What is Fidelity of Implementation?
Fidelity of implementation is the delivery of instruction in the way in which it was designed to be delivered (Gresham, MacMillan, Boebe-Frankenberger, & Bocian, 2000). Fidelity must also address the integrity with which screening and progress-monitoring procedures are completed and an explicit decision-making model is followed. In an RTI model, fidelity is important at both the school level (e.g., implementation of the process) and the teacher level (e.g., implementation of instruction and progress monitoring).
Why is Fidelity of Implementation Important?
For valid disability determination to occur, a diagnostic team needs to be able to determine that a student has received appropriate instruction in the general education classroom. Implementing instruction with fidelity satisfies one of IDEA’s legal requirements for appropriate instruction. “In making a determination of eligibility under paragraph (4)(A), a child shall not be determined to be a child with a disability if the determinant factor for such determination is– (A) lack of appropriate instruction in reading, including the essential components of reading instruction; (B) lack of instruction in math; or (C) limited English proficiency [SEC 614.(b)(5)].” Several studies confirm the importance of fidelity of implementation to maximize program effectiveness (e.g., Foorman & Moats, 2004; Foorman & Schatschneider, 2003; Gresham et al., 2000; Kovaleski et al., 1999; Telzrow, McNamara, & Hollinger, 2000; Vaughn, Hughes, Schamm, & Klingner, 1998).
Although these studies examined various interventions, the results suggest that positive student outcomes may be attributed to three related factors:
1. Fidelity of implementation of the process (at the school level)
2. Degree to which the selected interventions are empirically supported
3. Fidelity of intervention implementation (at the teacher level)
Although both common sense and research support the concept of fidelity of implementation to ensure an intervention’s successful outcome, the practical challenges associated with achieving high levels of fidelity are well documented. Gresham et al. (2000) and Reschly and Gresham (2006) noted several factors that may reduce the fidelity of implementation of an intervention:
- Complexity
The more complex the intervention, the lower the fidelity because of the level of difficulty. (This factor includes time needed for instruction in the intervention). - Materials and resources required
If new or substantial resources are required, they need to be readily accessible. - Perceived and actual effectiveness (credibility)
Even with a solid research base, if teachers believe the approach will not be effective, or if it is inconsistent with their teaching style, they will not implement it well. - Interventionists
The number, expertise, and motivation of individuals who deliver the intervention are factors in the level of fidelity of implementation.
How Can Schools Ensure Fidelity of Implementation?
When school staffs administer a standardized assessment, the assumption is that the test is administered according to the directions in the test’s accompanying manual and that the examiner is qualified. Implementation of RTI must meet the same standard. Direct and frequent assessment of an intervention for fidelity is considered to be best practice. When researching the effectiveness of an intervention, it is critical to be able to report the fidelity with which it was implemented so that any resulting gains in student achievement can be accurately attributed to the intervention under scrutiny and so that the intervention may be replicated. When implementing an intervention, it is critical to know whether it is being implemented as designed, so that if the intervention is initially unsuccessful, schools can take appropriate measures to remedy the deficiency rather than abandoning the entire reform.
Specific proactive practices that help to ensure fidelity of implementation include the following:
- Link interventions to improved outcomes (credibility)
- Definitively describe operations, techniques, and components
- Clearly define responsibilities of specific persons
- Create a data system for measuring operations, techniques, and components
- Create a system for feedback and decision making (formative)
- Create accountability measures for non-compliance
The ultimate aim of a fidelity system is to ensure that both the school process of RTI and the classroom instruction at various tiers are implemented and delivered as intended. This aim must be balanced with the school’s existing resources. General education in Tier 1, using a standard treatment protocol, is an important beginning to the RTI process. Several key components lead to high fidelity, and several key indicators are evidence of implementation with fidelity.
What Are the Key Components and Indicators That Lead to Fidelity in General Education?
The key components that lead to RTI fidelity in general education include the following:
- Systematic curriculum
- Effective instruction
- Direct instruction
- Specified instructional materials
- Checklist of key instructional components
- CBM assessments
- Videos and/or observations of classroom instruction
- Results graphed against goals
- Data (results) graphed against goals
- Student progress monitored monthly
- Decisions regarding curriculum and instruction based on data
Key indicators
Key indicators of RTI fidelity in general education include:
- 80 percent to 85 percent of students pass tests
- Improved results over time
- High percentage of students on trajectory (Reschly & Gresham, 2006)
What Are the Three Dimensions that Keep Implementation of Fidelity Manageable for Schools?
Dimension One: Method
Checking the implementation of a process for fidelity can be an extremely complex and resource-intensive process. In the research literature, checks for fidelity typically involve frequent observations and recording of behavior, teacher questionnaires, and self-report or videotaping of lessons. The tools available to achieve fidelity can be divided into two main categories (Gresham, 1989):
1. Direct assessment. The components of an intervention are clearly specified in operational terms within a checklist based on the task analysis of the major intervention components. A qualified staff member observes the intervention and counts the occurrence of each component to determine the percentage correctly implemented and identifies those teachers needing retraining.
2. Indirect assessment. Included in this type of assessment are self-reports, rating scales, interviews, and permanent products. Of the indirect methods, permanent product assessment is thought to be the most reliable and accurate. Permanent products might include samples of student work or student performance on assessments and videotapes of instructional sessions.
Written instructional materials or manuals represent a necessary but not all-sufficient method of ensuring the fidelity of implementation of interventions. The use of such written materials or manuals should be corroborated by direct and indirect measures. In other words, in reviewing a checklist, a teacher might use student work samples as evidence of compliance with the outlined steps on the checklist (Reschly & Gresham, 2006).
Although direct assessments of an intervention are considered to be best practice, schools likely will have to prioritize the ways in which they plan to ensure fidelity of implementation of the various components of RTI. Many of the tools to begin a process of fidelity checks may already exist within a school or are “built in” within the RTI process.
Dimension Two: Frequency
The frequency with which teachers are observed to ensure fidelity of implementation will vary depending upon several factors. These factors include, but are not limited to, the following:
- Teacher experience level
- Teacher requests for support
- Overall class performance
- Degree to which special education referrals do or do not decrease
In the interest of maintaining a non-punitive viewpoint of the evaluation process, it is important that a school set up a timeline for conducting teacher evaluations at the beginning of the school year. This allows teachers to see (a) that fidelity of implementation is important to the principal, school, and district and (b) that regular observations of teachers’ implementation is a typical course of action. The person who is designated as the observer (e.g., the principal or reading specialist) would ensure that all teachers are on the schedule for at least one observation.
It is important that new staff be evaluated during the first month of the school year and then further observations can be set up throughout the year depending on need. The dates for the screenings can also be included on this timeline so that teachers are aware of when the student progress data will be collected. Throughout the year, it is also important for teachers to be able to submit comments regarding the evaluation process or the curriculum as well as requests for support in the implementation process.
Dimension Three: Support Systems
As applied by schools, fidelity of implementation serves the purpose of identifying areas of deficiency that need to be remedied. For example, a newly hired teacher may not be familiar with the school’s reading curriculum. This teacher might require professional development opportunities to become acquainted with the principles and procedures of the curriculum. Or, a particular classroom may not have sufficient resources to implement and sustain a system of progress monitoring. This deficiency would require the subsequent attainment or redistribution of resources within the school. The kinds of support systems that are required to correct areas of deficiency likely will fall into one of two categories:
1. Professional development and training. This may include formal opportunities for workshops and in-service training as well as partnership with mentor teachers or coaches.
2. Resource allocation. If teachers do not have the proper resources to implement the intervention, it is incumbent upon the school leadership to obtain or redistribute resources.
How Does a School Achieve High Fidelity?
Overall, a school’s objective is to achieve high fidelity of implementation of the curriculum and instructional practices. If there is a high rate of fidelity in the implementation of the curriculum and appropriate instruction, this enables the administration and staff to rule out this variable with regard to student achievement. Essentially, if scientifically based curriculum and instructional practices are implemented as they were designed, then the student outcomes should be better and more consistent than previous years. When student outcomes are better, the school’s instruction and curriculum program increase credibility and reliability. This credibility naturally leads to a more highly motivated staff who wish to maintain this credibility through continued faithful implementation of the curriculum and instructional practices.
Does Fidelity of Implementation Affect School Structures and Staff’s Roles, and Responsibilities?
Ensuring fidelity of implementation integrates the following three components of a school:
1. Instructional tools and strategies
2. Student achievement
3. Professional development
This integration cannot occur if teachers are threatened by the system of observation and evaluation that will accompany this process. Accountability measures related to state assessments and the No Child Left Behind Act of 2001 (P.L. 107-110) (NCLB 2001) have in many cases placed an emphasis on punitive measures for teachers. We emphasize that schools should have the opportunity to implement a system of fidelity checks within a collaborative and positive environment that promotes teacher improvement. Honest and open communication with mentors or coaches can help a school tailor its professional development resources to support its staff and ultimately improve student achievement. Evaluations and observations of teachers then need to be approached in a positive manner that emphasizes problem solving.
Teacher mentors also can play a larger role in the school environment to ensure fidelity. To make this process work, mentors or coaches will need to have authority on which to act. Mentors who have proven ability in the relevant area (e.g., additional certifications, consistently high student performance, National Board Certification) should be selected to serve as coaches to new staff. Mentors may require some training for their new role, especially if they now find themselves evaluating their peers.
Download this Issue
To Download a PDF file version of this Issue of the NASET RTI Roundtable – CLICK HERE
