ChartsEarlier in the week, I saw a post on EdTech times about a new survey from MCH Data about technology usage in schools. I found a post on a different blog with the same data, so I thought it might be worth digging into. As a social scientist and qualitative researcher, I’m always a bit wary of purely quantitative surveys that seek to simplify some complex topics into simple snapshots. But we’ll get to that…

First, Their “Findings”

That said, let’s start with some of the actual data. Then I’ll critique.

So, a few highlights from the report:

  • 90% of school districts reported having whiteboards, and 80% of districts reported having whiteboards in all schools. I assume they mean interactive whiteboards (SmartBoards), but it’s not entirely clear.
  • 65% of districts reported having WiFi in all of their schools.
  • About 40% of districts reported using tablets in all schools, another 15% use them in some schools, and about 10% plan on implementing them in the next 18 months.

And the quote that really piqued my interest:

Of the technologies covered in the survey, tablets/ereaders are expected to have the most significant growth.

This jives with what seems to be a general consensus that tablets are the next big thing in education, and that tablets are poised to be the device for one-to-one initiatives. I don’t think that’s a good idea, but that’s a topic for another day…

What Exactly Does “In All Schools” Mean?

Now, for some methodological concerns.

I’m curious what exactly “used in all schools” means. Does that mean they’re used by most people in all of the schools in the district? By some people in all schools in the district? By at least one person in all schools in the district?

I love the backpedaling the report does when talking about whiteboards:

Almost 80% of districts surveyed reported that [white boards] were used in all schools in the district and 87.9% of districts that use whiteboards use them in all schools. This implies some level of heavy use; however, the survey did not address the question of classroom saturation. There may be lots of room for growth.

That, to me, sums up the value of this survey. Our data says that districts use this technology, but we assume they don’t really use it that much… so there’s plenty of room to sell more of this technology.

For example, my school has interactive whiteboards in a few classrooms. I don’t know the exact count, but we’ve got somewhere between a half a dozen and a dozen whiteboards in the building. We just rolled more out this year, so the number could increase to two or three dozen.

However, that’s out of nine or ten dozen classrooms. We spend a decent bit on technology and are probably in the upper quartile in terms of hardware, yet we only have 20 to 30% saturation of interactive whiteboards. Now, imagine the variation between school districts, and tell me how is it useful to know that 90% of districts use white boards somewhere?

Similarly, I’d like to know some real saturation numbers on WiFi. It would be awesome if every school in America had WiFi. But I’m thinking that a lot of that 65% has some WiFi in their schools, with access limited to a media center or a specific room. That’s like a hotel advertising that they have free WiFi when it’s only available in the lobby. [Yes, that happened to me in China, by the way.]

And What’s With the District Classifications…?

The other thing that seems strange to me is their classification of school districts. They break them down by size, which is reasonable. However, the tiers make no sense to me at all. Their classifications are as follows:

  • 1 School = Very Small
  • 2 Schools = Small
  • 3-4 Schools = Medium
  • 5 – 9 Schools = Large
  • 10+ Schools = Very Large

Now my concern is… why differentiate between all of those small districts and then lump the big ones together?

To me, either 1 or 2 schools is extremely small. You don’t have a large enough district to have a dedicated elementary, middle, and high school. You’re tiny in my book.

A small district could include 3 to 4 schools – at least one each of elementary, middle, and high schools. A medium district might have two high schools, and a sufficient number of feeder schools (5-10 total).

Now, the “very large” distircts can be broken down into at least two categories. Consider that my district has about 20 schools and 12,000 students. That’s for a city with a population of 65,000. It’s large, I’ll give you that.

But consider that neighboring Newark has 21 charter schools. And then they have 66 “regular” public schools on top of that. That’s for an enrollment of nearly 40,000 students and a city population of over 275,000.

Does that seem big to you? Well, now consider the real behemoths in education – New York, Chicago, Los Angeles.

I just don’t see how you can lump all those kinds of districts – East Orange, Newark, New York – into the same category and have it be meaningful.

Be Wary of Numbers. Always.

The bottom line is that statistics are easily skewed, especially when methodology is questionable and unexplained. While MCH tries to make their methodology sound thorough by saying that they called 10,000 school districts, it’s the underlying questioning strategies and data aggregation that I’d be afraid of.

Clearly, these questions are open ended and open to interpretation. I highly doubt that 65% of all school districts have completely blanketed their schools with WiFi or that 40% of schools have given a tablet to every one of their students. Without some kind of breakdown within the district and the school, this data to me is useless.

It’s easy to collect, sure. I’m not sure how you’re going to get more detail out of a person in one phone call, especially if you’re talking to a Board of Education employee for a “very large” school district.

But that just goes to show you that good research is extremely time consuming. If you want to really know what’s going on, you have to send people in to observe and have thorough conversations. In other words, you need some good qualitative research to know what’s going on in schools. Surveys like these make for great headlines and simple news stories, but they don’t really tell us anything useful about the state of technology in schools.