How to Evaluate Literacy Tools: What Schools & Districts Should Look For

The best literacy tool is not simply the one with the most features. Schools need to know whether a solution aligns with instruction, targets specific decoding skills, gives teachers useful visibility into student progress, and works in actual classrooms.

How to Evaluate Literacy Tools

Schools are under significant pressure to improve reading outcomes.

District leaders are reviewing curricula. Teachers are implementing more explicit foundational-skills instruction. Intervention teams are trying to identify struggling students earlier. Schools are evaluating digital tools that promise more practice, better data, personalized learning and stronger reading growth.

But choosing a literacy tool is not simply a matter of comparing feature lists.

The more important question is:

Will this tool help our teachers provide better reading instruction to our students?

For many schools, answering that question requires looking closely at three priorities:

1. Alignment between classroom instruction and student practice

2. Precision in phonics and decoding support

3. Visibility into what students can and cannot do

Once those priorities are clear, a fourth question naturally follows:

How can we determine whether this approach actually fits our school?

That is where a thoughtfully designed literacy pilot can be valuable.

The Best Literacy Tool Is the One That Solves an Instructional Problem

Schools do not need technology for technology's sake.

They need tools that solve real problems for teachers and students.

A school might be trying to address questions such as:

• How can students get more high-quality decoding practice?

• How can teachers identify which phonics skills individual students are missing?

• How can independent practice better reinforce classroom instruction?

  
  

These are instructional questions, not technology ones.

That distinction matters because a platform can be engaging, attractive, and easy to use - all without meaningfully improving instruction.

A useful literacy tool should make both the teaching and learning processes more effective and efficient.

Priority 1: Alignment Between Instruction and Practice

One of the first questions schools and districts should ask is whether student practice reinforces what is being taught.

Consider a classroom where the teacher is explicitly teaching consonant digraphs, but the student's independent reading program is having them practice r-controlled vowels. The student may still be learning, but the practice is completely disconnected from what the teacher is trying to pass along to them.

That disconnect can make it harder to build mastery.

Effective practice should give students repeated opportunities to apply skills recently taught, while also accounting for what they already know and what they are ready to learn next.

The What Works Clearinghouse practice guide on foundational reading skills recommends explicit instruction that helps students connect sounds in spoken language to letters, decode words, analyze word parts, and apply those skills in connected text.

For schools evaluating a literacy tool, alignment should therefore mean more than saying that a program is "research-based" or "science of reading aligned."

School and district leaders should ask:

Does the actual student experience reinforce the scope and sequence of the curriculum, classroom routines, and skills our teachers are currently teaching?

A tool that conflicts with classroom instruction may create unnecessary confusion. A tool that complements instruction can give students more opportunities to grow.

Priority 2: Precision in Phonics Instruction

Broad reading labels are not precise enough to guide instruction.

A student may be described as "below benchmark" or "struggling with reading," but those descriptions do not tell a teacher what to teach next.

The underlying difficulty may involve issues with:

• Phonemic awareness

• Letter-sound correspondence

• Continuous blending

• Short vs long vowels

• Consonant digraphs

• Silent-e patterns

• Vowel teams

• R-controlled vowels

• Etc...

  
  

Two students with similar overall reading scores may need completely different instruction.

One student may have difficulty hearing the individual sounds in a spoken word. Another may hear the sounds accurately but confuse specific letter-sound correspondences. A third may decode individual words successfully but fail to apply the same patterns while reading sentences.

These are not interchangeable problems.

That is why precision matters.

A strong literacy tool should help educators move from: "This student is struggling with reading."

...to: "This student needs more support accurately decoding words with short e and short i."

The first describes a problem. The second identifies a teachable skill, and creates an instructional path.

Priority 3: Visibility Into Student Decoding

A student can appear to be reading without decoding accurately.

During independent practice, students may guess from a picture, use the first letter and substitute a plausible word, skip an unfamiliar word, or rely heavily on context.

The final answer may be correct, even when the reading process is not reliable.

For teachers, this creates a visibility problem.

A completed lesson tells you that the student reached the end. A score may tell you whether answers were right or wrong. Neither necessarily tells you how the student actually processes printed word.

Useful decoding data should help answer questions such as:

• Which letter-sound correspondences do they know well?

• Which phonics patterns are consistently missed?

• Can the student apply an explicitly taught skill to unfamiliar words?

• Is performance improving over time?

When teachers can see the specific point of breakdown, they can respond more precisely. Visibility turns assessment data into an actionable plan.

Why More Data Is Not Necessarily Better Data

Schools already have enormous amounts of data.

Teachers may have universal screening scores, benchmark assessments, curriculum-based measures, intervention records, program dashboards, state assessment results, and classroom observations.

The problem is not always a lack of data.

Often, the problem is that available data does not clearly answer the teacher's next question: What should I do tomorrow with this student?

A literacy dashboard can contain dozens of graphs and still fail to provide useful instructional direction.

Good data should reduce uncertainty, not increase it.

For phonics and decoding, useful data should ideally show:

• The exact skills being measured

• Current student performance on those skills

• Common error patterns

• Change over time

• A clear connection to the next instructional step

The goal is not to produce a more impressive dashboard. The goal is to help teachers make better, quicker decisions - so they can help students catch up quickly.

Once Priorities Are Clear, Schools Can Evaluate Fit

Suppose a school has identified its core literacy priorities.

Leaders know they want stronger alignment with classroom phonics instruction. They want students to receive more decoding practice. They want teachers to see precise skill-level data. They want independent work to produce useful information rather than simply completion metrics.

The next question is straightforward: Will this particular tool work in our building?

Product demonstrations can help schools understand features. Research studies can provide evidence about outcomes in other contexts. References can reveal how other educators experienced implementation.

But none of those sources completely answer how a tool will work with a school's own students, teachers, curriculum, devices, schedule, intervention model or literacy goals.

A well-planned pilot can help answer those questions.

What Is a Literacy Pilot Program?

A literacy pilot program is a small-scale implementation of a reading tool, curriculum, intervention, or instructional approach designed to help educators gather information before deciding whether to expand its use.

The Institute of Education Sciences describes implementation pilot studies as small-scale tests with intended users in real-world conditions. Their purpose can include examining feasibility, understanding how an initiative fits existing systems, identifying needed supports or modifications, and informing decisions about whether broader implementation makes sense.

A useful pilot asks:

Does this solution address our needs, can we implement it successfully, what do our teachers and students experience, what does the evidence show, and should we continue, adapt, expand, or stop?

That is a much stronger decision-making process.

A Pilot Is Not the Same as an Efficacy Study

This distinction is important.

A short school pilot can provide valuable information about usability, teacher/student experience, participation, implementation challenges, tech requirements, alignment with instruction, and data usability.

Depending on its design, it may also provide preliminary information about changes in student performance.

But a small implementation pilot does not automatically prove that a tool causes improved reading outcomes.

Claims about efficacy require an appropriate research design. Factors such as comparison groups, sample size, assessment quality, intervention duration, implementation fidelity, student characteristics, and statistical methods all affect what conclusions can reasonably be drawn.

School leaders should therefore be clear about what they are trying to learn.

A pilot can be extremely useful without being treated as definitive proof of effectiveness.

Seven Questions Every School & District Should Ask Before Piloting a Literacy Tool

1. What problem are we trying to solve?

Do not begin with the product. Begin with the need.

For example: "Our teachers do not know which phonics patterns students struggle with"

Or: "Students need more opportunities to apply the phonics skills taught during core instruction."

A specific problem makes it possible to evaluate whether the tool actually helps.

2. Does the tool align with our instructional approach?

Review the actual student experience to understand which scope & sequence is being used, what type of errors are being identified, how that feedback is given to students, how decodable the texts are, and how actionable the data is for teachers.

Do not rely solely on a marketing statement that says "aligned with the science of reading."

Ask how alignment works in practice.

3. What will teachers be able to see that they cannot see now?

This is one of the most important questions. Will the tool reveal:

• Missed phoneme-grapheme patterns?

• Accuracy percentage by specific skill?

• Growth over time?

More data is not automatically valuable. The new visibility should support a real instructional decision.

4. What will teachers do differently because of the data?

Imagine a teacher opening the dashboard. What happens next? Can the teacher identify a skill to reteach? Create a small group? Assign focused practice? Determine whether intervention is working?

Data should lead to action.

5. How will students experience the tool?

Observe students directly. Are they independently able to use it? Do they understand the directions? Are they actually practicing reading skills, or spending significant time navigating, waiting, or playing non-instructional activities?

Engagement matters, but engagement should always be focused on learning. Avoid platforms in which students spend time playing games that are not educational. Many literacy platforms include noneducational games that students love, but teachers often get frustrated about the amount of time students are playing games with zero educational value.

6. What will success look like?

Ideally, schools and districts should define this before a pilot begins. Possible indicators might include:

• Teachers can identify students' specific decoding gaps more easily.

• Students receive more opportunities for successful targeted practice.

• Teachers report that the data changes instructional decisions.

• Student accuracy improves on specific phonics skills.

• The tool fits existing schedules without creating additional demands.

Success criteria should reflect the original problem the school was trying to solve.

7. What decision will we make with the results?

A pilot should end with a decision. Possible outcomes include:

• Adopt

• Expand

• Continue testing

• Modify implementation

• Limit use to particular grades or student groups

• End implementation

The purpose of a pilot is not simply to say that a pilot occurred. It is to reduce uncertainty before a larger decision.

What Schools & Districts Should Measure During a Literacy Pilot

A useful pilot should examine more than student logins. Consider gathering evidence in four areas:

Instructional Alignment

Does practice reinforce what teachers are teaching?

Are the phonics skills, routines, and progression compatible with the school's curriculum?

Teacher Usefulness

Can teachers quickly understand the data?

Does the information identify actionable student needs?

Does it save time, create additional work, or simply duplicate information the school already has?

Student Experience

Can students use the tool independently?

Do they receive meaningful reading practice?

Does the feedback help them correct errors?

Evidence of Learning

Are students improving on the skills being practiced?

Are gains visible at the phonics-pattern or decoding level?

Can students apply learning to unfamiliar words or connected text?

These categories give leaders a more complete view than usage data alone.

Why Real-Classroom Fit Matters

An intervention may be theoretically strong and still be difficult to implement in a particular school.

A tool may require more teacher time than expected. Device availability may be limited. Students may need more login support. Reports may not match the information interventionists need. The scope and sequence may not align with classroom instruction.

Alternatively, a tool may fit smoothly into existing routines and solve a problem that teachers have struggled with for years.

School leaders cannot learn all of this from a sales presentation. Real implementation reveals friction...but also reveals value.

This is why pilot studies can be useful before scaling. They allow educators to examine whether a new initiative works under actual local conditions and what support, training, or modifications may be necessary.

A Practical Literacy Pilot Framework

A school considering a new literacy solution can use this simple process:

Step 1: Define the instructional need

Write one or two specific problems the school wants to solve.

Step 2: Establish success criteria

Determine what evidence would indicate that the tool is helping.

Step 3: Choose an appropriate pilot group

Select classrooms, grades, intervention groups, or teachers that provide a meaningful test of the intended use case.

Step 4: Establish a realistic timeline

The timeline should be long enough for educators to observe implementation, student response, and, where appropriate, early performance trends.

Step 5: Gather multiple forms of evidence

Use quantitative and qualitative information when possible, including:

• Student performance

• Teacher feedback

• Classroom observations

• Student experience

• Implementation challenges

• Usage data

Step 6: Compare results with the original goals

Did the tool solve the problem identified in Step 1?

Step 7: Decide what comes next

Adopt, adapt, continue testing, expand, or stop.

The strongest pilots are designed around a decision, not simply access to a product.

Red Flags When Evaluating a Literacy Tool

Schools should be cautious when:

• A vendor focuses heavily on minutes logged or lessons completed, but cannot clearly explain what students learned.

• Reports show broad scores without identifying specific phonics or decoding needs.

• Claims of "personalization" are not connected to observable skill differences.

• A tool says it is aligned with reading research but does not explain how instruction reflects evidence-based practices.

• Student improvement claims are presented without information about the assessment, sample, timeline, comparison group, or research design.

• The platform introduces routines that conflict with the school's core literacy instruction.

• Teachers receive more data but no clearer instructional direction.

No single red flag automatically means a tool is ineffective. But each should prompt deeper questions.

The Most Important Question: Does the Tool Make Reading Instruction Better?

Ultimately, schools do not need the literacy platform with the longest feature list.

They need tools that help teachers and students do something important better.

That might mean giving students more targeted practice.

It might mean identifying a decoding difficulty that was previously invisible.

It might mean giving immediate corrective feedback during independent work.

It might mean helping a teacher see that one student needs more work with short vowels while another is ready for vowel teams.

It might mean making small-group instruction more precise.

The technology itself is not the goal. Better reading instruction is.

When Literacy Priorities Become Clear

When schools clearly define what they need, evaluating literacy tools becomes easier.

Look for alignment between instruction and practice, precision at the skill level, visibility into student decoding, and data that changes what a teacher does next.

Then, when appropriate, test the approach under real classroom conditions before making a larger commitment.

A well-designed pilot cannot answer every research question. But it can give school leaders something highly valuable: better evidence about whether a solution fits their students, teachers, systems, and instructional priorities.

That turns the decision from: "Does this product look promising?"

...into: "Do we have evidence that this helps solve the problem we set out to address?"

A Brief Note About English Islands

English Islands is designed to provide independent reading and writing practice, immediate corrective feedback, and visibility into the specific letter-sound patterns students struggle with. Schools evaluating English Islands, or any literacy platform, can use the framework above to examine instructional alignment, usefulness for teachers, student experience, and evidence of learning before making a larger implementation decision.

Frequently Asked Questions

What should schools look for in a literacy tool?

Schools should evaluate whether a literacy tool aligns with classroom instruction, targets specific reading skills, provides actionable data, supports meaningful student practice, and produces evidence that teachers can use to guide instruction.

What is a literacy pilot program?

A literacy pilot program is a small-scale implementation of a reading tool, curriculum, or intervention used to gather evidence about feasibility, instructional fit, teacher and student experience, and potential value before broader adoption.

How long should a literacy pilot last?

There is no universal ideal length. The appropriate duration depends on the intervention, the questions being asked, and the outcomes being measured. A pilot should be long enough to observe meaningful implementation and gather credible evidence for the intended decision.

Does a successful pilot prove that a literacy tool is effective?

Not necessarily. A pilot can provide valuable evidence about feasibility, usability, instructional fit, and preliminary student performance. Strong causal claims about effectiveness generally require more rigorous research designs.

Why is phonics data important when evaluating literacy tools?

Broad reading scores may show that a student is struggling, but specific phonics data can help identify the precise sound-spelling patterns or decoding skills that need support. That makes the information more actionable for teachers.

How can school leaders evaluate whether literacy data is useful?

Ask what a teacher can do differently after viewing the data. Useful data should help educators identify a specific need, choose an instructional response, and monitor whether the student improves.

Why test a literacy tool in a real classroom?

Real classroom implementation can reveal whether the tool fits existing instruction, schedules, technology, student needs, and teacher workflows. These factors may not be fully visible in a product demonstration or research study conducted elsewhere.

If this was helpful, feel free to share: