Protocol Revision for Accreditation Agencies
American school accreditation offers tremendous opportunities and challenges to the implementation of any new program of school visits. Initially six regional accrediting organizations accredited almost all the public high schools in America and in some regions all schools in a district.
While protocols vary across regional associations they all consider the profession peer school visit as their signature event. It is the usually the most important instrument in the accrediting process to collect information about how well a school is performing against a set of standards set by the association.
Most researchers and policy makers do not consider the work of these associations of real value. They distrust the rigor of how information is collected and how decisions are made.
Catalpa has worked closely with the associations that have responsibility for accreditation in 41 states as well as Defense Department schools and many other schools in international settings. This work has included observation of 10 visit teams at work under an accreditation protocol in regions across the country.
Catalpa's current overall conclusion about the quality of accreditation visit is that accreditation teams can really take off and make solid and penetrating observations of schools. When they do you can see the principles of PBI at work. But reports seldom have the overall rigor and utility that they could achieve and too often appear to be driven by prescription or conforming to what the team thinks the accrediting agency wants.
The problem is that accreditation protocols are overloaded with concepts and practice that merge back into the 100 year history of American accreditation. And the central paradigms for designing protocols are not considerations of how to strengthen the team's methodology for forming a conclusion. Rather they are based in practices that protect schools from criticism, that push the team to make superficial analysis of evidence and that push the accreditation process forward to the accreditation decision (preferably a positive decision to award accreditation).
Finally, accreditation agencies do review their standards for judging school and their protocols on a regular basis. Thus protocols do change over time in response to changing issues and knowledge. For example, the planning and organizational paradigms that were designed for business are frequently incorporated into standards.
Strengthening the rigor and public acceptance of reports has led them down the path of so many educational policy researchers: variables, data, precision (or at least the illusion of precision.) Teams are asked to rate schools on four point scales (and these ratings are what counts for accreditation) and to use organizational analyses and language in place of a sharp focus on teaching and learning in their descriptions. Schools are commended for planning well, rather than for teaching well.
Catalpa has completed two major reports of the actual practice of accreditation visits. By basing the analysis on the principles of PBI, Catalpa's analysis focuses much more clearly on the visit as a research methodology than on an administrative process. Both reports were received well by the agencies involved and in one case resulted in major changes is the visit protocol.
Catalpa still has major work in this area to find ways to provoke an understanding of the loss and gains to the agencies and schools of conducting visits that are built on a stronger scientific basis.
Accreditation Standards and School Improvement: Putting Methodology in its Proper Place. NCATE (National Council for Accreditation of Teacher Education). 1998. View »
Visiting Accreditation: Strengthening the Regional Accreditation Process. NEASC (New England Association for Schools and Colleges). 1999.LAB at Brown. View »
Rigor and Utility: The Efficacy of AdvancED's Practices for Conducting School Evaluation Visits and Preparing Team Reports. AdvancED (Formerly NCA -- the North Central Association for Schools and Colleges and SACS the Southern Association for Colleges and Schools). A Special Catalpa Study, 2011. (Copyrighted and distributed by AdvancED, Alpharetta, GA.)
Program and Policy Evaluation »
Protocol Design »