Sunday, August 31, 2014

Analyzing the latest Bluegrass Poll, the three major newspapers' reporting of it, and the pitfalls of polling

Nothing in this post violates the guidelines, and it is almost nine years old, which leads me to think some sort of system error is responsible. —Al Cross

By Al Cross
University of Kentucky School of Journalism and Telecommunications

The latest Bluegrass Poll shows Sen. Mitch McConnell with a lead of 46 percent to 42 percent over Alison Lundergan Grimes Monday, Aug. 25 through Wednesday, Aug. 27. Here's how the state's two major newspapers, The Courier-Journal and the Lexington Herald-Leader (which co-sponsor the poll with WHAS-TV and WKYT-TV) reported the results, with some illustrations of the pitfalls of polling:

In print, the Herald-Leader put its story under a floating H-L nameplate, with news of the University of Kentucky football game above. The headline was "McConnell extends lead over Grimes," with a small-capitals, one-column subhead: "SENATOR HOLDS HUGE ADVANTAGE IN EASTERN AND WESTERN KENTUCKY." The poll was The C-J's top story, with a one-column headline, "Poll gives McConnell 4% lead on Grimes," and a subhead: "Third consecutive poll to show lead widening." (The Kentucky Enquirer in Northern Kentucky, a sister Gannett Co. paper, used the C-J's story in a very similar way.)

Technically, 4 percent is not the same thing as 4 percentage points, because of those who are undecided and chose Libertarian David Patterson; "4-pt." would have fit in the narrow width. More importantly, while McConnell has gained in the last three polls, all the results are within the polls' error margins, so the published results are only the most likely results within a range of possibilities. And the error margins apply to each candidate's percentage, not the difference, so in theory there is a small chance that Grimes could be ahead. But three straight polls showing her losing ground is a newsworthy trend.

C-J political writer Joe Gerth reflects that and the probabilities by writing at the start of his third parargaph, "It's the third consecutive Bluegrass Poll that has found McConnell improving his chances for re-election in November." In his first paragraph of his story, he writes, "Mitch McConnell has put a little more distance between him and his Democratic opponent, Alison Lundergan Grimes, but the race remains within the margin of error."

H-L political writer Sam Youngman began his story by writing that McConnell's campaign "has gained momentum in the last month, propelled by huge leads . . . in Western and Eastern Kentucky and among men." After giving the results, Youngman writes, "McConnell's showing is his best to date in the Bluegrass Poll, but the race remains well within the poll's margin of error ...". That margin means that whatever statewide momentum McConnell has is small, but he clearly does have momentum in Western Kentucky, where he posted a 23-point lead after leading by only 5 points there in July. (The error margin for the western sample is plus or minus 8.8 percentage points.)

Online, the Herald-Leader's headline,"Gaining momentum, McConnell holds 4-point advantage over Grimes," reflected Youngman's emphasis. The C-J headline took a different focus, with this headline: "McConnell favorability improves in Bluegrass Poll." That part of the poll isn't reported until the 19th paragraph of the story, which said McConnell "continues to improve his standing with voters, even though 46 percent still view him unfavorably and only 36 percent view him favorably. That's considerably better than the poll found last winter when 50 percent viewed him unfavorably and only 27 percent viewed him favorably." However, the July poll showed his rating as 43 percent unfavorable and 36 percent favorable, so his rating actually worsened in the latest poll, albeit well within the error margin of 3.9 percentage points among registered voters. Told that, the newspaper changed the headline to "McConnell expands lead in Bluegrass Poll."

The error margin for the poll's Senate-race question was 4.2 percentage points, because that question was asked only of people who said they were likely to vote in the race. But the small difference in those figures reflects the fact that of the 647 registered voters surveyed, 569 said they were likely to vote in the race. That would be a turnout of 88 percent, which history says will not happen -- far from it. Historically, turnouts in elections where a Senate race is at the top of the ballot are slightly under 50 percent. So, the poll's sample includes a lot of people who said they would vote but won't.

How does that skew the results? In recent years, turnout in non-presidential federal elections like this one has fallen most among Democratic-leaning voters, so history indicates that McConnell is even more likely to win than the poll indicates. However, he may also have a turnout problem, because 40 percent of the Republicans who voted in the May primary chose someone else. The May election was McConnell's first real primary, and may have shaken some Republicans' willingness to go to the polls for him on Nov. 4. People often act differently than they tell pollsters they will act, yet Youngman wrote that McConnell is "locking up 79 percent of Republican respondents" in the poll. (Emphasis added.)

1 comment:

  1. Your analysis is better than what appeared in the Herald or the Courier, Al, but I believe it missed a pertinent point. First, we can agree that McConnell has a modest lead at this juncture, but because of the MOE it may be no greater than it was last go-round. The important point, I think, is that the survey showed McConnell at 46 percent. He was at 47 percent last time -- not great momentum if you ask me. The increased gap can only be attributed to Grimes supporters going undecided. If true, can it be that McConnell, after 30 years in office, has a 46-47 percent ceiling? If so it's a problem since most late votes usually -- not always but usually -- trend toward the challenger. I recollect the poll further indicated Grimes maintains only a one percent lead among women. I would expect that to change and, if McConnell's tepid Tea Party supporters sit this one out, it should be a horse race.

    ReplyDelete