Is Your EdTech Company Only Committed to Making the Sale…or to Really Improving Learning in K12 Schools?
Using new EdTech tools to really improve learning, means more than just selling EdTech products to schools...
Because schools are “complex organizations” the implementation of a new instructional program, product, or solution will often set in motion complex change dynamics. These dynamics and variables can impact learning outcomes, for better or for worse, no matter how powerful or elegant the product features in question.
In fact, researchers have shown that when a school or district invests in a new instructional program, new education technology, or new instructional practice, there may be little to show for it in terms of improved academic achievement. And whether academic results diminish or improve overall, the many factors impacting (or potentially impacting) instruction and learning can make it near impossible to demonstrate a strong link between changes in academic learning and a specific EdTech product.
In this article we’ll take a brief look at some key reasons for this common disconnect between powerful EdTech innovations and improvements in learning, including:
why schools are “complex organizations”
why there is a whole consortium dedicated to trying to help school leaders quantify EdTech ROI
what kinds of implementation challenges make it hard to predict how an EdTech innovation will perform
how EdTech companies and marketing teams can respond to some of the core challenges posed by schools, organizational complexity, and local variability
So what does it mean to say schools are “complex organizations,” and why does it matter for EdTech companies and their marketing teams?
If you’re selling a software solution to a conventional B2B company, such as an auto parts maker or a large real estate firm, your clients’ organizations may be large and internally complex, but they are not a “complex organization” in the same way that schools, hospitals, and federal service agencies are typically “complex.” A key point is not to conflate size with complexity, at least not in the present context.
Let’s take the example of the large real estate firm and explain why it’s less complex than a school, despite its size. For one thing thel real estate firm it has a very simple definition of success and a relatively static decision-making structure: increased revenue and profits are the essential success metrics, and decision-making is essentially in the hands of senior leaders. Even if there are policies or processes for more delegated decision-making, any such policy or process is under the direct discretion of the senior leaders.
Schools are different…Like many organizations in highly-regulated sectors, schools have many layers of dynamic complexity and school directors have limited managerial autonomy compared to leaders in many profit-driven business models.
School organizations—big and small—are frequently buffeted by a range of shifting federal, state, and local policies, laws, and mandates. And, school leaders not only need to manage staffs and complex bargaining agreements, but they also need to serve, support, protect, and “manage” large student bodies and be responsive to a range of interest groups, both internal and external (including shifting ideological or personnel-related interests or factions, parent and student interest groups, community interests, and social and political interests, both local and national…).
Success metrics are also more ambiguous in K12 settings. For example, a school leader may institute solutions, programs, or policies narrowly focused on improving academic achievement as a key success metric, but isolating and evaluating linkages between specific inputs and outcomes can be difficult to do…Likewise, a focus on academic “success” as such, may be judged as coming at the expense of competing priorities within a values-driven organizational setting like a K12 school—competing priorities such as student wellbeing, student choice and voice, equity, or other “value-driven” interests related to the kind of broader educational experiences parents and guardians want for their children, or that students themselves care most about…
All of this means that EdTech innovations land in organizational settings beset by complexity and local variability: as a result it can be difficult to validate correlations between a specific EdTech innovation and any specific gains in academic achievement.
The challenges posed by local variables…
It is true that, conceptually, there are some uniform best practices for leading effective innovation and implementation processes in complex settings and in K12 settings in particular, but there’s one problem: individual schools often have unique kinds of overlapping, complex variables that impact or complicate the implementation of new instructional practices. In change and innovation theory, these dynamics are sometimes referred to as local variability.
These local variables are typically anchored in factors related to professional culture, differences in student, family, and community values and demographics, differences in technology readiness and technology proficiencies, differences in leadership beliefs and competencies and in educator proficiencies and prevailing organizational and instructional norms and practices…
School complexity, local variability and the advent of the EdTech Evidence Exchange…
Because schools have complex dynamics contributing to learning outcomes and often have unique organizational traits, cultures, and proficiencies, EdTech companies discover that 1) it’s hard to measure the direct impact that EdTech tools have on learning, and 2) it can be hard to predict how the same EdTech product will perform from one school to another, or even from one classroom to another…
In fact, schools are so complex and so fraught with localized variables that the federal government has established a coalition called the EdTech Evidence Exchange whose primary mission is to find ways to accurately “measure” the connection between specific EdTech tools and programs schools have purchased and the actual “ROI” they get in return, especially in the form of academic achievement gains…
At its core, the EdTech Evidence Exchange came into existence because schools, education researchers, and public funding agencies had to acknowledge that there was no way to gauge the actual benefits accrued from the millions and billions in annual education technology spending!
As a matter of fact, the experts and researchers that lead the EdTech Evidence Exchange settled on defining at least 10 types of local variability and complexity in K12 settings!
Then, for good measure, these researchers added an 11th variable—a placeholder for any additional variables yet to be discovered!
EdTech marketers, therefore, face a similar dilemma: if schools can’t figure out how to gauge the value or benefits of EdTech tools and isolate various learning influences in order to measure the academic “ROI” for their EdTech spending, then EdTech companies also can’t meaningfully draw a clear line of attribution between their own EdTech product (or any of its specific features) and verifiable gains in academic achievement (until and unless, perhaps, their product demonstrates strong and consistent correlations with academic achievement over extended periods and across a range of K12 settings).
Admittedly it may seem counterintuitive to assert that a technology tool or feature that addresses a specific instructional need or challenge will have no measurable benefit for learning, and we may assume that some products or features would inherently deliver specific benefits. But, while some kinds of benefits may be easier to isolate and measure in some settings, such as a time-saving benefit for teachers, for example, the problem remains that making a meaningful connection between an EdTech product and specific academic achievement gains can be much more difficult…
Fidelity of implementation can be challenging in its own right, nor is it a reliable predictor of specific or consistent outcomes or benefits in across different schools and districts…
Even if a single or small cluster of localized school settings appear to deliver academic achievement data linking an EdTech product to improved learning, those outcomes may prove elusive in other school settings or contexts, and even in settings that are, or that seem to be, “comparable” to one another. This is because some variables are easy to compare, like core student demographic data, for example, but a a whole host of other local variables are harder to quantify and may also be fairly invisible from the outside. The fact is that research into organizational and social innovation and change agency tell us that complex cultural, social, and organizational settings pose hard-to-predict and often counterintuitive obstacles to successful change and innovation, even with the implementation of well-designed technologies.
Why are K12 school complexity and local variability so problematic for EdTech companies and for their marketing teams?
In a nutshell, these organizational dynamics in K12 schools can (and do) make it very difficult to draw any clear linkages between the use of specific EdTech products or programs sold to schools and improvements or changes in student learning outcomes.
This problem cuts in different directions too…
For example, even if a new EdTech tool has features that make it stand apart as a learning solution, organizational complexities mean that actual changes in academic achievement may in fact result from other shifting and dynamic organizational factors or forces. In addition, these factors or forces can be hard or impossible to quantify and/or disaggregate, making it virtually impossible to link changes in student academic achievement with specific causal factors.
For those who have a fuller understanding of the kinds of leadership competencies and practices that can ensure a more effective implementation and end-use process, it remains a fact that because of local variability, there are no simple set of prescriptive implementation processes or practices that can simply be replicated across different K12 organizations with any promise of success. In fact, no less a player than Microsoft Education has argued that for innovation to bring real success, school leaders need to attend to at least twenty preliminary planning steps—that’s right, twenty—including many deliberative and open-ended planning steps, that individual K12 schools or districts should engage in prior to deploying, or even adopting and purchasing, new EdTech devices (see Transforming Education, a report by Microsoft’s Education division).
Because schools are complex and fraught with localized variables, meaningful technology innovations often fail to change schools and instructional practices for the better…
School complexity and local variables mean that even if the purchase and implementation of an EdTech tool may seem to lead to improved outcomes, there are likely to be other variables at play which may be the real drivers of changes in academic achievement.
One very simple example has to do with school cohorts. If a student who achieved “low” in reading at the end of 3rd grade enters a 4th grade class and starts using a new EdTech product and begins scoring much higher in reading at the end of 4th grade, how do we isolate the role that the EdTech product and investment played in this scenario? Obviously the change in teacher and classroom dynamics could also be change factors, or not…
That’s just one example. Now think of any number of other variables that change from one school year to the next, such as a new district reading curriculum, the introduction of new teaching methods, changes in other instructional norms or routines in the classroom from one year to the next, and so on….
This means that even in “successful” scenarios that seem to validate the performance of an EdTech product, it can be hard for educators, and by extension EdTech companies and their marketing teams, to build a strong case for what results really can be attributed to a product or any specific product features.
Complex organizations and the implementation paradox…Some organizational change researchers point to an implementation paradox when it comes to the introduction of new technologies in complex organizational settings.
Let’s imagine that teachers are struggling to provide enough one-on-one reading instruction in their classroom. An EdTech product that provides one-on-one instructional support in reading skills to individual students and aligns to each student’s individual reading level would seem certain to improve reading scores, right?
The paradox that can occur with implementation of the new instructional technology is that even when it offers solutions directly aligned with observed educational problems or challenges, the actual implementation and outcomes are often not predictable or reliable due to the way the introduction of one innovation or change can trigger new challenges and change forces.
As a result, implementation of the promising instructional technology may not only not move the needle on learning but may even result in negative outcomes, diminishing rather than improving the targeted academic knowledge or skills.
And, as strange as this may seem, researchers have also shown that in many cases, a school will purchase an EdTech tool but never puts it to use, or the school will achieve only a very limited scale of roll out, or fail to use the product or innovation with any consistency. According to reporting by researchers with LearnPlatform, for example, this may be even more common than most of us would imagine. Lack of basic implementation follow through can result from leadership challenges, logistical challenges, organizational dysfunction, training challenges, or other factors…
Whatever the cause, the fact is that implementation challenges and complexities like these mean that selling an EdTech product to schools or districts will often fail to impact and improve learning…Nor is the client school or the seller going to be able to generate valuable feedback on features and benefits flowing from the EdTech product or program, nor is the EdTech company marketing team going to benefit from a great client success story that can be used in future marketing collateral!
How EdTech Companies Can Collaborate More Effectively With Schools: Tips for Managing Complexity and Local Variability in School Settings…
In the face of the kinds of implementation challenges posed by organizational complexity and local variability, what can EdTech companies and marketing teams do?
Here are some tips that may help EdTech companies and marketing teams sell to K12 schools AND also partner effectively with K12 clients to help ensure more meaningful outcomes for student learning.
Identify how your EdTech product solves larger challenges in school settings and learn how your product offers relevant solutions to specific and current instructional challenges at local sites. The same insights that align your EdTech product with key challenges and goals in specific school settings might inform marketing content and improve how your EdTech company approaches new prospects and initiates relationships with them.
Identify and anticipate potential obstacles to full and effective implementation by taking stock of potential challenges or obstacles related to local factors such as:
a) levels of student engagement with academic content
b) teachers’ existing professional competencies and performance challenges (at the specific client school or district)
c) leadership readiness and organizational capacity generally for driving longer term innovation in instructional practices
Build partnerships in candid dialogue with school clients because each school or district will be in a different phase of its innovation journey…This journey on the road to innovation is impacted by a range of local variables related to factors such as:
a) teacher competencies and teacher professional learning modalities
b) educational values and organizational culture
c) existing IT proficiencies
d) underlying beliefs about teaching and learning
Understanding specific client schools and districts and their readiness for implementing instructional changes and innovation—where they are on their journey—is critical to building an effective implementation plan and road map.
It’s important to go beyond teacher “buy-in” when planning for the successful adoption of a new instructional technology…Teachers, or most often, a particular small cohort of teachers poised to serve as change agents, are typically in the role of “first adopters” for EdTech products and will often play a pivotal roll in implementation success. It probably pays to partner and dialogue more deeply with leaders and instructional change agents at client schools in order to support successful understanding, adoption, and end-use practices.
Today’s school leaders know (or should know) that the old saw about getting “staff buy-in” before promoting a new instructional practice or tool is simply not enough. EdTech companies need to know this too…
Technologies designed for instructional use have at least two categories of dynamic “end users”: teachers and students. In most cases, how students “receive” and engage with a new learning technology and new learning formats will be inflected significantly by how teachers themselves initially perceive, understand, and value an EdTech product and related innovations in instructional practice and principles.
And, when teachers themselves are not fully “on board” and positively “on-boarded” when it comes to understanding the larger educational or instructional principles and practices that will play a determinative role in successful implementation, then the EdTech tools may never be implemented in many classrooms or may only be implemented in a very limited fashion.
Likewise, if teachers are not receptive to or don’t fully understand the rationale for the adoption of and transition to a specific EdTech tool and any related instructional practices and routines, then implementation may be impaired as a result.
Engage in more sustained partnerships with school clients…In addition to working with school staff as you approach a school and develop plans for relevant professional support and classroom implementation, it’s also important that deeper partnerships allow your business-oriented leaders to join their managerial acumen with the school-based knowledge of school administrators to figure out with them and alongside them how to
a) actually measure the impact of your product on learning outcomes
b) achieve practical and sustained data-gathering routines
c) identify, implement, and monitor strategies for using the data to improve instructional practice and boost learning
In “complex organizations” this work can be more challenging than business leaders may realize, but investing in this kind of deeper learning and iterative implementation successes can be a boon for both schools and EdTech companies who sell to schools…
In the end…Would it be easier to merely sell your EdTech product to schools and forget all the rest?…
The answer is that it would be easier, but to what end?
Bashing or blaming schools, or simply turning our back on these challenges won’t serve schools, business, or national workforce development in any meaningful way.
There are real reasons schools, both public, private, and including those under charter school arrangements, have more often than not struggled to innovate successfully.
The need to achieve real adaptive change and innovation…
In complex organizations like schools and school districts, identifying, purchasing, and implementing tools, programs, and products without the kinds of proper planning and collaboration needed to anticipate and discover pertinent organizational variables and change dynamics is the same as believing that a mere technical change or technical fix is sufficient when a range of adaptive changes are typically needed in tandem with the deployment of a new EdTech product.
Innovation replication may be based on flawed assumptions…
As tempting as it may be to believe that there is a right solution or superior EdTech tool that simply needs to be proven and disseminated at more schools, it is probably a fallacy to believe that increasing sales and adoption of any so-called proven EdTech product across more school markets will result in significant academic gains without deeper planning and collaborations and the hard work of achieving sustained and robust adaptive change in local settings.
If school leaders and researchers like those at the EdTech Evidence Exchange can’t figure what schools are actually getting for their EdTech spending, then EdTech companies and marketing teams will also be struggling to achieve their long-term prospects for building solid business models based on selling solutions to schools before their clients figure out how to evaluate benefits in the form of real and measurable academic achievement gains.
Let us know what you think about this post!
EdPro Communications offers flexible, freelance support at competitive rates for content marketing and grant writing informed by K12 education experience and insights.
