Last Friday, on the Guardian Australia site, blogger Celeste Liddle expressed cynicism about polling released by the Recognise movement that showed high levels of support for a “yes” vote in a referendum to recognise the first Australians in the Constitution.
She juxtaposed our research with a self-selected social media poll released earlier in the week by Indigenous X, which showed much lower levels of support. In her blog, Celeste accuses the Recognise movement of not listening to dissenting voices.
So whose survey is right?
Both polls are simply a reflection of their method. They are not the same as each other – and that’s key to understanding why the results are different.
Let’s turn firstly to the poll we commissioned for the Recognise movement – a non-government, not-for-profit organisation which receives public funding (as the Expert Panel recommended) to help build public awareness and support ahead of a referendum vote.
For three years we’ve commissioned voter research so that we can be accurate and confident of the opinions of both Aboriginal and Torres Strait Islander people and non-Indigenous Australians.
This work is done by experienced, professional researchers who are members of the Australian Market and Social Research Society. The most recent survey, undertaken in March, was done by Polity Research – a small, independent research company.
We insist on best practice market research methodology. Our surveys are built on nationally representative samples with strict sample quotas set by age, gender, income, profession, education, and location. The national sample is selected randomly from the Australian population; contacted by phone and via online market research panels. This is done to avoid survey bias. The surveys are large enough to draw statistically significant conclusions (in the most recent survey, the sample size was 2,700 people from the general Australian population and 750 Aboriginal and Torres Strait Islander respondents).
The participants are not drawn from our own supporter database nor are our networks used to promote the surveys. Great care is taken to ensure that nobody responds more than once. This research over several years reaffirms continuing strong support amongst Aboriginal and Torres Strait Islander people.
Polity took great care to make sure the results were as robust and representative as possible. For example, the Indigenous survey required more than 8,000 phone calls to be made in order to reach the required random participation of a representative sample of 750 respondents. No respondent knew what the survey was about to ensure no self-selection based on their views.
The margin of error was +/- 1.9 percent for the general population and +/- 3.6 percent for the Aboriginal and Torres Strait Islander population.
People were asked: “If a referendum were held today, how would you vote on the proposals to recognise Aboriginal and Torres Strait Islander people in the Constitution?”
In the general community, 75 percent of voters said they would vote ‘yes’ and 25 percent said they’d vote ‘no’. In the Aboriginal and Torres Strait Islander community, 87 percent said they’d vote ‘yes’ and 13 percent said they’d vote ‘no’.
When gauging public opinion, we don’t opt for self-selecting polls. They are notoriously inaccurate as a measure of the views across a population. In self-selected polls (often used by talkback radio and other platforms with a motivated audience with a particular view) respondents are recruited by calls to action (being asked to call-in or complete a survey online to express a view). This means respondents are recruited by their own like-minded networks – a practice which very quickly distorts results. Put simply, it’s like asking your own friends about an issue and then saying “everyone I have spoke to agrees with me”.
The respected US journalist and pollster Richard Morin of the Pew Research Centre, a former polling director for The Washington Post, has written that self-selecting surveys “litter misinformation and confusion across serious policy and political debates, virtually wherever and whenever they are used”.
Now to the Indigenous X poll. This was a self-selecting poll (or “self nominating” to use Celeste’s term). It was promoted repeatedly through Twitter and social media amongst like-minded networks of activists, including some like Celeste who are longstanding critics of the Recognise movement. It was not a nationally representative sample and there were no quotas set to ensure demographic representation.
In addition, on Survey Monkey, depending on the settings chosen for a poll, it’s possible for people to vote more than once.
So both polls were a reflection of the views of those that completed them – one randomly-selected, one self-selected. To compare them is to compare apples with oranges. To be absolutely clear, though, neither should be used to denigrate the motives of those commissioning either of them.
Emeritus Professor Murray Goot of Macquarie University, a leading expert on polling, notes the particular challenges of sampling in surveying Aboriginal and Torres Strait Islander community opinion:
“All attempts to survey Indigenous opinion need to acknowledge the difficulties of the task and the limitations of the findings. Approaches that seek to reach as wide a range of respondents as possible in proportion to their presence in the population are to be preferred to approaches organised around particular websites, news outlets or media audiences. And methods that lower the probability of self-selection are to be preferred to methods where self-selection is built-in.”
Finally, we take issue with the accusation that Recognise is “just not listening” to dissenting voices. This is simply incorrect.
Over the past two years, the Recognise movement has held more than 270 events across the nation to engage in detailed conversations on the Journey to Recognition relay. We have also provided or helped to source speakers for hundreds more events hosted by other organisations. Our relay teams (made up of volunteers and Recognise staff) have now travelled more than 32,000 kms and held over 270 community events in 212 places which have been attended by more than 20,000 people. Many of these events, especially those involving smaller and longer discussions, have featured dissenting voices.
In addition, Recognise campaigners are addressing hundreds of people every week at public forums and debates (including a recent, informative and respectful panel discussion that included Tanya Hosch, Larissa Behrendt and Celeste Liddle in Melbourne).
Our staff team, and particularly joint campaign director Tanya Hosch, continues to engage with Aboriginal and Torres Strait Islander peoples who oppose or are undecided, as did the Expert Panel. So, too, have many other Aboriginal and Torres Strait Islander leaders who have given this cause countless hours of their time and leadership over many years. We have also sought to engage with non-Indigenous voices of opposition.
And some who declare themselves to be opponents of this coming referendum allow that they would vote to clean up the race discrimination in Sections 25, and 51 (xxvi). I’d note very respectfully that there’s only one way to do that – and it’s by having a referendum.
We have never denied there is dissent nor that there are critics of the movement – we simply disagree on the scale of it. We listen to our opponents and we respect their right to take a different view.
That said, we also intend to honour the decades of hard work, leadership, resilient campaigning and dedication of so many Aboriginal and Torres Strait Islander people who have worked for this opportunity to secure proper recognition and deal with the discrimination in the Constitution.
Tim Gartrell is the Joint Campaign Director for Recognise, a position he shares with Tanya Hosch. Tim has over two decades of experience running local and national election campaigns.