But districts’ data action plans are often met with resistance and skepticism borne from stakeholders’ past experiences with failed initiatives. This resistance is understandable, given the many recent examples of ineffective school data projects and policies that were costly to budgets, teacher morale, and instructional quality.
These challenges leave many school systems grappling with complex questions:
How can we harness that baseline consensus around the belief that data matters in our schools and translate it into thoughtful data plans that inspire buy-in, adoption and desired outcomes?
How can we adapt our data collection and use practices to benefit students?
The answers are not simple or complete. But we can learn from past successes and failures to better understand the elements of effective data planning.
The Experiences Informing These Insights
To provide some context for the school data planning ideas that follow, we want to briefly describe the experiences from which we draw these insights.
Our founding Schoolzilla team members were originally part of the internal Data and Technology department at Aspire Public Schools, a high-performing public charter school system serving low-income communities. Backed with generous funding from the i3 Grant, Michael and Susan Dell Foundation, and Bill and Melinda Gates Foundation, we set out to build a school data warehouse and reporting platform. The project ultimately involved $10 million, five years, a team of 20 people, and extensive educator and administrator feedback before our stakeholders told us we had gotten it right.
In the process, we have curated the following collection of best practices and pitfall warnings.
Traits of Successful School Data Plans
Define Data’s Role
Schools with effective data practices clearly define the roles they want data to play.
They design data collection and use plans that foster:
A culture of inquiry
Concrete goals and accountability
Consistency (in grading practices, for example)
Informed pacing, alignment, and grouping for instruction
Amanda Datnow and Vicki Park outline the above “data promises” in their book, “Data-Driven Leadership,” as well as some of the “data pitfalls” discussed below.
Understand the Limits and Pitfalls of School Data Use
Perhaps equally important, people designing effective school data plans operate with the understanding that:
Students cannot be reduced to numbers. Data is important, but people have complex contexts, traits, and gifts that are hard to measure and also require attention and consideration. Data policies or practices should not suggest in any way that data can serve as a substitute for personal connections with and a deeper understanding of students.
Excessive focus on a few metrics in isolation can unintentionally encourage teachers and administrators to narrow the curriculum and dedicate academic resources disproportionately.
A holistic approach to data use is essential to serve students’ best interests.
Budget for Data Staff and Systems
The school districts that we have seen build effective data practices have at least one full-time staff member dedicated solely to “owning data.” This role’s responsibilities include managing data collection and analysis and collaborating with the administrators of any data systems. We have yet to see promising results from schools that try to make data work one of many “hats” worn by several individuals. Data managers and analysts are important investments for furthering schools’ goals, and these positions can often be at least partially funded through grants. When data is being ineffectively and inefficiently managed across a district, it costs teachers and administrators valuable time that could be better spent with students.
Similarly, we have seen very few success stories about districts building their own data software internally or contracting custom builds for their data solutions. Both options typically end up requiring more budget, on-staff expertise, and maintenance than an individual district has capacity for. There is a myriad of data tools available, and districts that carefully select from those existing systems to suit their unique needs tend to fare better in the long term.
Data Plan Implementation: Lessons Learned
SMART Goals Work
Strategic, Measurable, Ambitious, Realistic, and Time-bound
Schools with strategic action plans that work are aligned to goals that meet each of these criteria. Additionally, limiting scope by focusing on only one to three targeted, strategic goals at a time yields greater success than pursuing several data objectives at once. This practice simplifies resource allocation, improves work quality and prevents data overload.
Goals Need Resources
Coming up with goals in a central office is relatively easy, but unless those goals are tied directly to specific tools for educators and administrators, they are of little value. Consider this SMART goal, for example:
“Eighty percent of seventh grade students will advance 1.5 or more grade levels in reading during the 2014-15 school year.” The following resources would increase the likelihood of the school meeting this goal:
A breakdown of quarterly reading milestones by student - or the time, training, and support needed for teachers and school leaders to calculate milestones themselves
- Training for teachers on how to administer the 1:1 reading assessments
- Designated time for teachers to administer the assessments
- A tool for data capture
- A tool for tracking growth and progress throughout the year
- Substitutes to assist with or cover teachers’ classes while they administer the 1:1 assessments
- Time for collaborative data analysis, including data coaching
- Time with a literacy coach or master teacher to help plan and implement reading interventions
Underestimating the time and knowledge required to meet SMART goals like this one is a common source of frustration for school-level educators and administrators. When the academic and data leaders who write the plans and goals thoughtfully allocate resources, they both set the stage for success, and demonstrate to school-level teams that they have a grounded understanding of and commitment to all that goes into pursuing those ambitious goals.
Data Proficiency Takes Time and Practice
Most teacher preparation programs do not include training on working with data. More often than not, educators are willing and capable but don’t necessarily know what to do with all the data they’re given. Design your schools’ data action plans to support them in collaboratively building their data proficiency.
Administrator-led initiatives that lack teacher buy-in rarely work, and teacher-led initiatives rarely succeed without administrator buy-in. Schools we have seen thrive began with buy-in from at least a few key thought leaders at all levels of the organization. Some of the most effective methods for generating buy-in have included:
Involving teacher and school administrator representatives in district planning sessions
Having district staff present portions of plans in the making to collect and incorporate feedback. Note the incorporate step! This method doesn’t work when the feedback collection is done as lip service.
Consider the difference in attitudes, feelings, and behaviors elicited by saying, “Let’s dig into this data together to see if we can find ways to support our students’ learning,” versus, “Be prepared to demonstrate how you’ve driven student growth and account for instances where you haven’t.” While this framing may seem obvious, the latter approach is surprisingly common in schools amid the accountability movement. This approach fosters fear, mistrust, and defensiveness — setting the stage for some of the unintended consequences discussed earlier.
While data usage in schools has taken many forms and elicited substantial debate, there is little contention that, in any context, informed decisions tend to produce better results than uninformed decisions. School systems that now have thriving data practices began their work with that basic consensus and heeded the cautionary tales of past failed data initiatives. Through thoughtful goals, realistic resource allocation, and careful attention to stakeholder buy-in, these districts have built data plans that have translated to action, and those actions are now positively impacting students. No one says the process is easy or complete. All have said it is well worth the effort.