(The following is a post by @Tony_Chauncey and his views are in no way reflective of the official views of his employer!)
These are, of course, my thoughts, and not those of my employer or any other technology director in the universe.
I live in a small town. Most of my girlfriend’s paternal family lives in the next small town over. Therefore, we spend a significant amount of time in the next small town over. The only fast food place in the next small town over is a drive in. We’ll call it the Blue Devil Drive In. In general, I like to eat at drive ins. They’re an Oklahoma company with decent food that seems to contain decent ingredients. I told you all of that to tell you this… While it’s incredibly convenient to stop at the Blue Devil Drive In and grab a burger, I’d rather drive fifteen miles north to a slightly larger town (and fifteen miles out of my way) to go to the Tornado Drive In or some other place than go the Blue Devil Drive In. Why? Because every damned time I go there, I leave angry. It either takes forever (which defeats the purpose of fast food) or they screw up our order. I don’t tell you this to discourage you from going there; I’m sure that many people could tell you great things about the Blue Devil Drive In. I tell you this so that you know how I like to go about my business, and why it bothers me so much that we continue to use CTB/McGraw Hill for online state testing in Oklahoma. Two Words: CUSTOMER SERVICE.
The state of Oklahoma has a $13.5 million per year contract with CTB/McGraw Hill to provide online testing. I personally believe that this is way too much, but that’s not the point of this post. I’m sure that if CTB/McGraw Hill provided a reliable, consistent mechanism for administering tests, no one (myself included) would bat an eye at this number. They, however, do not provide anything resembling a reliable, consistent mechanism.
I’m not talking about the catastrophic failure that was the result of CTB/McGraw Hill’s overselling of their service in the spring of 2013. While that was ridiculously egregious and should have been enough to get Oklahoma’s State Department of Education to cancel it’s contract with CTB/McGraw Hill before this year started, it’s still not what I’m talking about. After that debacle, the OKSDE and CTB/McGraw Hill came to an agreement on a $1.2 million settlement, which not surprisingly resulted in additional time away from class for students and tons of extra work for Oklahoma’s School Technology Staffs, but did not actually test what needed to be tested. It did not test CTB/McGraw Hill’s ability to handle multiple states worth of students testing on their servers simultaneously, which is what caused last year’s two-day disruption. Of course, we don’t care whether CTB/McGraw Hill can provide testing for several states worth of students at once. If we did, we’d test it. Remember, “If you don’t test it, you don’t care about it.” But, again, this isn’t what I’m talking about.
In my district, all computers used for testing on any particular day go through the same process: 1) Start (or Restart) the computer; 2) Login with credentials used only for testing; 3) Student takes the test. This happens every morning. We do these things so that every computer is at the same point every morning. We have a very stable network. While we may not have all the bells and whistles that some do, I’d put our network up against anyone’s. We use all Cisco equipment, most of which has been upgraded in the last couple of years. But, again, this isn’t what I’m talking about.
What I am talking about is the constant “Please Wait…” and other communication related errors seen on CTB/McGraw Hill test clients. These messages are disruptive to students and frustrating to technology staff. Why, if I have two identical computers sitting right next to one another, does one work and one doesn’t? If computer A never worked and computer B always worked, I’d think the problem was with computer A. However, that has never been the case. Computer A might work one minute and computer B might fail, and vice versa the next minute. If computer A is working and allows a student to begin a test, why might it fail in the middle while the entire rest of the lab works just fine the entire way through the same test? We don’t have these issues on any other software used in the district. If you’re reading this, I’m probably not telling you anything that you don’t already know.
The real problem here is customer service. Why does CTB/McGraw Hill not care to put out a better product? Why do we continue to put up with this? Please wait… messages should be an anomaly, not the norm. But, I continue to hear things like “We had some issues, but it was better than last ye ar.” I hear these things from our Building Test Coordinators and on all three of the Oklahoma Technology Director Listservs of which I am a part. “Better than last year” isn’t nearly good enough. Seeing one Please wait… message should be cause for concern, but it’s become the norm. We continue to be happy with “better.” “Better” than awful isn’t worth $13.5 million per year. This is why I don’t buy burgers at the Blue Devil Drive In. If people continue to go to a place with awful customer service, that place has no reason to serve their customers better. CTB/McGraw Hill has no reason to perform any better. Our SDE appears content with this awful level of customer service, despite the increasing importance of these tests.
Perhaps the reason our SDE puts up with this level of customer service is that our SDE, under it’s current leadership, provides the same level of customer service. Amazing and dedicated people work at the SDE, but no organization can go from 500 employees to 300 employees in just a couple of years and be expected to maintain the same level of customer service for the same number of customers. Perhaps I’m wrong about this. Perhaps the SDE was a bloated organization. I don’t believe this was the case, though.
My first year in education, which was 2005, the district that I worked for was selected for an E-Rate Selective Review by the PIA at USAC. It was a pain. It wasn’t fun, but I gritted my teeth and did it. I ended up sending in about a ream of paper to answer all of their questions. I thought “Whew… Hopefully, I’ll never have to do that again.” I was wrong. The very next year, we were selected again. I was incredibly unhappy about it. Like, a lot unhappy. I wanted to know why we were selected in consecutive years. I called the education technology guy at the SDE (who is now a colleague). He called the SLD and got an answer for me. He probably should have made me do it myself, but he listened to me vent and did something about it for me. He called me back within a couple of days with an explanation. I felt like I had an advocate at the SDE. It was important to me.
The SDE, in it’s current incredibly slim form, doesn’t seem able or willing to answer questions that are, to me, incredibly important. I’ve emailed current SDE staff on multiple occasions about technology purchasing issues and I still don’t have any answers. I’ve not asked difficult questions. I just don’t get responses. Then, we have the Wave office. Remember when you could call the Wave office and talk to people who knew something about the Wave? Call them now and you might get a call back, in a week or two, maybe, from someone who doesn’t really seem to know much about the Wave.
It’s all about customer service. It doesn’t matter if you’re talking about $15 worth of burgers, $13.5 million worth of testing software or contact from a governing body. It’s just customer service. It’s incredibly simple. People need to be held to higher standards and a high level of accountability. Like Albert Einstein said, “Insanity is doing the same thing over and over again and expecting different results.” As tax payers, parents and educators, we must stop settling for terrible service.
Director of Technology
I wrote this on Sunday night. Obviously, I also was not talking about the catastrophic failure that resulted in testing disruptions on Monday, April 21, 2014. This colossal failure looks to be more ridiculous than last year’s colossal failure. According to our State Department of Education,
“CTB said a piece of hardware – a DNS resolver – caused intermittent disruptions for about 8,100 students throughout the state. The failure affected both internal and external systems at the vendor.”
Wow… Where to start… First, does anyone know what a DNS resolver is? Also, how does a single piece of hardware cause a company that does 40% of this country’s online testing to be unable to deliver test sessions to students across the country? Does CTB/McGraw Hill not know about failover? It’s a pretty old concept. This explanation doesn’t seem to make much sense.
It’s another example of horrible customer service from CTB/McGraw Hill. Janet Costello Barresi says that she will not endorse keeping CTB/McGraw Hill as our online testing vendor for next year. Why did she recommend keeping them this year? Perhaps Mr. Einstein could shed some light on the matter…
About the February 28th stress test, according to an Oklahoma State Department of Education employee,
“(T)he stress test did take place in other states at the same time of the same day.”
This is the first time that I have heard this. We were told all along that the purpose of the stress test was to test the ability of districts to give the tests and the ability of IPS’s to provide required bandwidth. We were explicitly told that CTB/McGraw Hill’s ability to deliver test sessions to students was never under fire. If other states gave the stress test on the same day at the same time, CTB/McGraw Hill would have done themselves a favor by telling us that. It would have alleviated at least some angst toward the SDE and CTB/McGraw Hill for requiring the stress test in the first place. This is yet another example of poor customer service from our SDE or our testing vendor. You decide.