Originally published on February 6, 2020
In the beginning of January I forwarded everyone an email I received from the NY State Education Department looking for test writers for the 3-8 ELA and Math assessments. If I’m being honest, one of the main reasons I signed up to do it myself was because my younger sister, who is a teacher, said, “Wouldn’t it be fun if we both did it and I could stay with you for a few days.” As it turns out, she never applied.
I signed up for this thinking that it would be interesting to see the back end of what goes into creating the 3-8 assessments. The much-maligned assessments have been demonized by many and I have heard many descriptions of how they are created. I wanted to get some first-hand knowledge of “how the sausage gets made.” Spoiler alert, I did not know what I did not know and I’ll bet most of you don’t know either.
January 6, 2020
First of all, there was a short application process where I had to indicate my education, my certifications, etc. I completed this on a Monday night.
January 8, 2020
By Wednesday, I was notified that I made it past the first round of screening and that I next had to complete three self-paced training modules which each had assessments to measure my understanding of the modules. Oh, by the way, these were due by the end of the day on Friday which gave me 2 days to do it. Wednesday was out because I had a long-scheduled dinner with a friend I hadn’t seen in over a year, so that left Thursday night.
January 9, 2020
On Thursday I had my oldest son’s HS orientation. Since the email said each module was about 30 minutes, I did it on Thursday after I got home from the orientation and ate dinner, which meant it was a little after 8:00 pm at that point. Well, these modules took me over three hours to complete. Around 11:00, my husband called down from our bedroom, “When are you coming to bed?” (Insert crying emjoi here because I really did not want to do these things at all, let alone for three hours after work and after Freshman orientation.) Nevertheless, I completed them and sent in my completion certificates.
January 10, 2020
On Friday afternoon I received directions for the next step in the process. So, on Saturday, I watched a recorded webinar to learn even more information to prepare me to write the questions. This lasted about 90 minutes. When I emailed that I had completed that step, I was told that I would get my next set of directions either sometime that weekend or on Monday. That would be the final step and would require me to write some sample questions to put what I learned into practice and get feedback. This must be done, I was told, by that Wednesday.
January 13, 2020
I did receive the email on Monday but when I got home on Monday, I saw that I received an email telling me I had to go to court on the first day of the actual question writing (this was regarding my credit card number getting stolen in October which is a story for another day). I reached out to the test writing people and said that I’d contact the Assistant District Attorney’s office to see if I could reschedule or not come at all. I wondered if I couldn’t get out of it if that meant I wouldn’t be able to participate in the test writing. Since I wouldn’t know the answer until Tuesday, I wasn’t going to waste my time on Monday doing work that might not count.
January 14, 2020
On Tuesday I learned that I didn’t have to appear in court so the test-writing was back on. I sat down on Tuesday night to do the work only to discover that I didn’t have editing rights so I couldn’t electronically write or submit anything. I did it by hand and sent a few emails with the word “URGENT” in the subject to the test writing people and thought, “You’ve got to be kidding me!” After all, this assignment was due on Wednesday and I did not have any time to do the work during the work day. On Wednesday, they changed my rights so I could enter my info. Done. In all I spent about eight hours doing all of this work.
What I learned
All of this was required before I could even step foot into the test question writing sessions. To be honest, I was really surprised. Keep in mind, the test question writing sessions were scheduled for three days. Three days. I was shocked that so much work went into this even before we got together since I figured that the reason why we’d be together for three days was, in part, to train us. I was wrong.
I was also surprised by how much I learned about writing the test questions through this process. The level of detail and thinking that goes into it was more than I expected. For example, for ELA multiple choice questions they now try to make the answers as close to being the same length as possible and the choices should be listed in the order they would appear in the text or, when that’s not possible, by length from shortest to longest. These things sound simple, but in reality, it was not nearly as easy as it sounds.
In my letter next week, I’ll share with you some of what I learned through the process of writing the questions over those three days because that’s a letter all by itself. In the meantime, I’m wondering if you realized the front loading that goes into this or what surprised you about the process I described. I’m also curious what training you’ve had on writing standardized assessments. I find this fascinating.