Monday, March 5, 2012

Questions Are Raised about Civil Beat Rail Poll Showing Alleged Slippage in Support for Project: Why Were Neighbor Islanders’ Views Solicited, Why Was Half the Electorate Excluded from Calls?

It isn’t just the Sunday newspaper that often starts the day with a bang, as it did yesterday. Mondays also are a favorite for outlets like Civil Beat to break their news, as the online subscription news service does today.

“Casual visitors” are allowed access to the site, CB says, and presumably can read this story, which tops CB's news today: Civil Beat Poll – Honolulu Voters Oppose Rail Project

April 2013 Update: See Yes2Rail's final post -- the results of the 2013 mayoral election that revealed the extent of Civil Beat's misunderstanding of public opinion on rail.
It took a second or third reading to appreciate the precision with which that headline was written. The Merriman River Group “touch-tone poll” truly did survey only voters, according to this note below the results: “No calls were made (a) to residents of the state who were not registered voters or (b) to registered voters who did not have a demonstrated record of participation in recent General Elections"

But There’s More
The reference to “residents of the state” is noteworthy, because the phrase just as easily and more precisely could have said “residents of Oahu” if that were the case. But incredibly, this survey on the Honolulu rail project solicited the opinions of neighbor island residents!

We know that for a fact because of the February 26th email we received from a friend living on Maui who maintains no residence on Oahu. After describing the questions and issues in the automated poll, he observed:

“Seemed to be pretty straight-forward -- meaning, unbiased -- but the reporting of the results could appear skewed against the project, meaning a large percentage of respondents could be "very concerned" or "somewhat concerned" about the cost, but a large percentage of that group could still favor the project.
Big question: What the hell are they doing calling someone on Maui??
What, indeed. That’s the question we’ve sent to John Temple, Civil Beat editor and author of today’s story on the survey. It seems almost inconceivable that a Maui resident and presumably other neighbor islanders would be asked about these issues.

And since it wasn't just Honolulu voters who were surveyed, the headline does appear to be misleading after all.

A List of Questions
Here’s the text of the email we’ve sent to Mr. Temple:

John, did the polling company have any comments on how the demographics and related issues may have influenced the poll's results?
• Only 52 percent of eligible voters statewide participated in the 2010 gubernatorial election. Did Civil Beat's poll not contact the other half?
• What is the presumed outcome of not contacting residents who don't exhibit an interest in "public affairs" based on their voting pattern? Did the polling company comment on that?
• The respondents appear to have been "overly educated" compared to the population at large, with 70 percent in the "college degree" and "grad degree" categories. Did the polling company have any comments on what that might suggest re their preference or lack of same for transit?
• Similarly, the household income chart says 67% of the respondents had household income above $50,000. Again, how does that compare to the entire population on Oahu?
As always, the devil is in the details, and I hope you and your colleagues can provide more of them to help the rest of us understand this poll better.

A Radical Suggestion
This isn’t the first time questions have been raised about a rail-related public opinion survey. The Star-Advertiser and Hawaii News Now sponsored a poll that began one week after anti-rail mayoral candidate Ben Cayetano officially launched his campaign.

It’s difficult to believe his strong support and the anti-rail majority found in the poll were not influenced by the heavy media coverage Mr. Cayetano enjoyed during the survey period. That poll had a poorly worded question that required an element of interpretation by the participants. At least the Civil Beat poll avoided that problem in asking its first question.

In 2008, the Honolulu Advertiser reported the results of a survey that, while showing strong pro-rail sentiment, included an extremely misleading and frankly inaccurate question that contributed to the results.

Now comes the Civil Beat survey that raises questions about how it was conducted. So here’s our radical suggestion:

The next time a media outlet wants to survey public opinion on the Honolulu rail project, it should involve both rail supporters and opponents before the survey begins on both the conduct of the poll and the questions.
Nothing would be lost in such a collaborative effort, and a lot of misinformation might be avoided. If Civil Beat had taken this approach, it presumably would not be asked to answer the questions that reasonably must be raised about the validity of this new survey.


Michael Levine said...

Hi Doug,
Michael Levine from Civil Beat here. I just spoke with Merriman River Group, who assisted us with the poll. You heard correctly from your Maui friend, as folks on the neighbor islands were called. Seven (7) people on neighbor islands completed the survey, and it's likely that many more were called and did not complete the poll. Your friend appears to be one of them.

However, MRG caught the presence of neighbor island respondents during the survey, and we removed all of them from the final results. So the poll results you see on Civil Beat today are based only on Oahu voters, though some neighbor island residents were called.

Thanks for your questions, and your coverage of The Civil Beat Poll.


Anonymous said...

Doug, would you feel better if the headline said "Majority of highly educated public affairs minded Oahu residents think rail is a bad idea?"

I know it's hard for your side to swallow, but can you just not even grasp that most people think this is a bad idea? have you no water cooler to hang out at?

I don't think it's a lack of communication strategy on the city's part. I think people are smarter than you all think. ya'll complain about misinformation, but i'd say the city has been promoting more of this than other group. and purposefully hiding reports they commissioned with public funds that they don't like the results of. that's just not on the up and up, and people are getting sick and tired of it.

Doug Carlson said...

Anonymous, your headline would be more accurate than implying, as Civil Beat did, a "tide" of anti-rail sentiment among the population as a whole. You're right: I don't believe this poll and the most recent one were valid reflections of what "most people" think about rail, for reasons I've already written about, as you've seen.

Can you get beyond your own anti-rail tunnel vision to see that my criticisms of Civil Beat's poll today are valid?

Doug Carlson said...

Michael, thanks for your responses today. They were helpful.