50 banned words on New York tests, polling, and cultural bias
Piers Morgan examines 50 words that are banned on New York school tests. Here’s the full list:
- Abuse (physical, sexual, emotional, or psychological)
- Alcohol (beer and liquor), tobacco, or drugs
- Birthday celebrations (and birthdays)
- Bodily functions
- Cancer (and other diseases)
- Catastrophes/disasters (tsunamis and hurricanes)
- Children dealing with serious issues
- Cigarettes (and other smoking paraphernalia)
- Computers in the home (acceptable in a school or library setting)
- Death and disease
- Expensive gifts, vacations, and prizes
- Gambling involving money
- Homes with swimming pools
- Junk food
- In-depth discussions of sports that require prior knowledge
- Loss of employment
- Nuclear weapons
- Occult topics (i.e. fortune-telling)
- Rap Music
- Religious holidays and festivals (including but not limited to Christmas, Yom Kippur, and Ramadan)
- Rock-and-Roll music
- Running away
- Television and video games (excessive use)
- Traumatic material (including material that may be particularly upsetting such as animal shelters)
- Vermin (rats and roaches)
- War and bloodshed
- Weapons (guns, knives, etc.)
- Witchcraft, sorcery, etc.
As an educational researcher, I take a completely different approach. One of the main goals of testing is to assess a student’s knowledge and understanding of a particular subject. This can be a very delicate process — the ways in which questions are worded can have a significant impact on the answers given. This is a constant issue in political polling, for example. A recent YouGov poll suggests that a majority of Republicans don’t want to decrease spending to most government programs, but there’s a difference between wanting to cut spending to a government program vs. the complexity of actual issues that concern voters, such as whether additional spending is possible with a budget deficit, whether the funds are better spent elsewhere, or whether a federal program is best suited to implement the program (perhaps it’s better run as a state or local program). People’s understanding of social and economic issues is often much more complex and nuanced than polling suggests.
An infamous case of polling bias happened in 2010, when the Pew Research Center asked, “Do you happen to know what Barack Obama’s religion is? Is he Christian, Jewish, Muslim, Buddhist, Hindu, atheist, agnostic, or something else?” 34% of respondents said that Obama was Christian while 18% said he was Muslim. Time magazine used different wording: ”Do you personally believe that Barack Obama is a Muslim or a Christian?” Given only two options, 47% of respondents said he was Christian, while 24% said Muslim. The choice of wording influences how people answer a question. If the following question is asked: “Do you believe that Barack Hussein Obama is a Muslim?” then it’s likely that an even higher number of people will say yes.
When constructing a test, it’s important to remove any bias that may occur from how the questions and responses are worded. In particular, cultural bias may skew test responses. Here’s an example: “John put his hat on the table” vs. “John put his afghan on the divan.” The first sentence would be clear to most people, but the second sentence may be more difficult to understand. The extra effort required to process the second sentence may be too small to measure in individual test-takers, but looking at aggregate scores may yield more significant differences.
I think of it as noise. When constructing a test, you want to make sure that there’s as little noise as possible. Consider the following question: “Charles Manson rode the train for 3 hours. The train was travelling at 60 miles per hour. How far did the train go?” You may spend more time thinking about why Charles Manson is on the train, where he’s headed, or any number of other thoughts. Or the following question: “The farmer killed 3 chickens on Saturday, 5 chickens on Sunday, 2 chickens on Monday, and no chickens the rest of the week. On average, how many chickens does the farmer kill per day?” That question may make some people queasy, but now replace “chickens” with “kittens” and suddenly you’ll see test scores drop. The attitude that you have towards kittens being killed may be the same attitude a vegetarian has towards chickens being killed.
A poorly designed test confuses what is being testing with who is being tested. If a question singles out a particular group of people, that makes test data less reliable. It’s not that educational researchers want to avoid offending people, it’s that offending people creates noise in the data that skews results. Is it enough to make a single person answer a question incorrectly? Probably not. If I ask you the two Obama religion questions from above, chances are your answer will be consistent. However, when you total up all of the answers and compare the two surveys, you start seeing differences emerge that aren’t obvious at the individual level.
I feel that there may be some misconceptions about the NY ban on test words. Here are a few key points:
1. It’s not really a ban. The word “ban” is used by journalists who want a catchy headline. The actual document says: “As a guide, the following topics are to be avoided:” Those words can still be used in relevant test subjects.
2. The list applies to statewide standardized tests. We’re not talking about classroom tests given by teachers. A biology teacher can still give a test on dinosaurs or cancer. A history teacher can still give a test on slavery.
3. It’s not even a test that grades students:
“Students take Periodic Assessments several times throughout the school year to give teachers more information about what students have learned. Teachers use these assessments—along with other school work and what they see in class—to learn where students need more help and plan targeted instruction. The New York City DOE does not use Periodic Assessment results to grade children or schools.”
So we’re not even talking about real tests. These are a guide to help teachers with their instruction, and don’t impact a student’s grade.
I feel that people read the headlines and assume that these words are being banned from schools or from all school tests, but from what I read, that isn’t the case. They’re being banned from a very specific statewide standardized test that’s used to guide a teacher’s instruction. Teachers can use the words all they want in their own tests.
http://www.classsizematters.org/wp-content/uploads/2012/04/RFPR09111.pdf http://www.classsizematters.org/wp-content/uploads/2012/04/RFPR0911Section3Appendices1.pdf http://schools.nyc.gov/Accountability/resources/assessments/default.htm