Civil Beat Publishes Survey Questions on Day 4; Respondents Were Asked Only about ‘Concerns’ and Nothing about ‘Benefits’ in ‘Push Poll’ – a Style Condemned by National Research Group

One thing’s certain – and maybe it’s the only thing – about Civil Beat’s public opinion survey on the Honolulu rail project that the news service has highlighted all week at its website: CB staffers may have learned something about the perils of opinion polling.

Civil Beat publishes the questionnaire online today; infrequent visitors to the website might be able to read it without signing up for the subscription service.

This survey is known as a “push” poll’ inside the business. It “pushes” a concept or issue at the recipients and asks them to respond. In Civil Beat’s case, the poll pushed “concerns” about the Honolulu rail project.

Not one “benefit” of building rail or riding the train was mentioned in this survey. It pushed only negatives and excluded anything that would qualify as a positive about the project, such as completely avoiding traffic congestion on surface streets, saving considerable time and money with each trip, and more.

AAPOR on ‘Push’ Polls 

The American Association for Public Opinion Research has a statement on “push” polls that “explains how to tell the difference between fraudulent political polls – commonly referred to as ‘push polls’ – and legitimate polling, including message testing. AAPOR condemns political telemarketing under the guise of research….”

Let’s assume Civil Beat harbors no anti-rail attitudes – not consciously, at least. By sponsoring a poll that includes only “concerns” and no “benefits” of rail, the news organization nevertheless participated in what the AAPOR statement calls “an insidious form of negative campaigning, disguised as a political poll.”

Beginning with question 7 and continuing through number 10, the questionnaire asked the respondents to express a level of concern about the rail project – “too expensive,” “will damage Oahu’s natural beauty and landscape,” “not enough people would use the rail line if it is built” and “will disturb Native Hawaiian burials.”

Question 11 asked respondents “which of those issues concerns you most about the rail project” and then listed the concerns again -- won’t solve problems, cost, beauty, not enough use, burials, not concerned, not sure.

Participants in this automated telephone poll heard only concerns – enough presumably to sink a project like this one if not offset by a host of benefits, not the least of which is fast, frequent, reliable and safe travel through our increasingly congested urban corridor.

One-Sided Polling

The AAPOR statement suggests ways to message-test a survey to determine if it’s legitimate. Civil Beat’s questionnaire seems to satisfy five of the six so-called tests, but not this one: “The questions usually ask about more than one candidate or mention both sides of an issue.”

Quite obviously, the Civil Beat poll that mentions only “concerns” and nothing about “benefits” failed the both-sides test.

The AAPOR might also have an opinion about a questionnaire that first mentioned candidates in the one-issue mayoral race (all rail, all the time) before continuing on to the list of concerns about rail. Doing so undoubtedly influenced the responses to the “concerns” questions.

Civil Beat may have innocently stumbled into the opinion survey swamp by commissioning this automated-call push poll, but however it got there, this particular survey quite reasonably can be dismissed as a legitimate and accurate representation of Oahu residents’ attitudes about the Honolulu rail project. 

That's not a conclusion we come to gladly. We've admired Civil Beat for its independence and space-unconstrained investigative coverage of the rail project and many other local issues since it launched its online service nearly two years ago.

But like other Honolulu media, Civil Beat can trip itself up when covering Honolulu rail – a project too big and complicated to be easily explained in two-minute TV stories (where most people get their news) or even in long newspaper and online pieces.

All media are encouraged to devote more time becoming acquainted with rail before they report on it, including rail's goals and benefits. Citizens need complete coverage if they're expected to have informed opinions about the biggest construction project in state history.