January 19, 2025
Our latest Abacus Data poll finds the Conservatives maintaining a 26-point lead, even as two recent polls suggest the gap may be tightening.
Naturally, people are left wondering: which of these estimates is more likely to be correct, and why can polls differ so dramatically? One key explanation lies in the methods used to collect data and the type of respondent each method tends to capture.
When pollsters talk about Interactive Voice Response (IVR) polling versus online polling, they are describing two distinct processes to collect data. IVR relies on automated phone calls, where recipients press buttons to register their opinions. Because most people either ignore or hang up on automated calls, the response rate tends to be very low. Those who do answer these calls are often the most motivated, most animated, or in some cases, the most upset. This can lead to a form of response bias, where the people who stay on the line are not necessarily representative of the wider population, but rather a vocal subset that feels strongly enough to participate.
One notable example comes from Canada’s 2021 federal election, when various IVR polls projected much higher vote shares for the People’s Party of Canada (PPC) than the party actually received. Some surveys suggested the PPC might capture nearly twice the percentage they ultimately secured. Why did this happen? PPC supporters were highly engaged and dissatisfied with mainstream options, so they were more likely to respond to phone polls. Their strong enthusiasm translated into an exaggerated presence in IVR data. Although IVR can sometimes detect an emerging wave—if those passionate views eventually spread to a broader audience—it can also simply measure the fervor of a minority, rather than accurately gauge mainstream opinions.
Final polls in the 2021 Canadian General Election Source: Wikipediahttps://en.wikipedia.org/wiki/Opinion_polling_for_the_2021_Canadian_federal_election
The start of the Liberal leadership race and Prime Minister Trudeau’s resignation could have a similar energizing effect on Liberal-oriented voters, prompting them to become more active participants in polling. Suddenly, a group that may have been apathetic or less engaged finds a renewed motivation to express its views, whether through IVR calls or SMS invitations to online surveys. While that surge in responses can create the impression of a major shift in voter support, it may simply reflect temporary excitement or newfound engagement around leadership changes rather than a lasting realignment in the electorate.
On the other hand, online polling generally tends to secure a more stable sample by offering incentives such as reward points, small payments, or gift cards. These incentives encourage participation from a wider range of individuals, including those who may be less motivated by politics alone. Consequently, online panels can be better equipped to provide a representative cross-section of the electorate, smoothing out the peaks of enthusiasm or anger that can dominate in IVR surveys.
So, returning to the question of which polls are correct regarding the Conservatives’ lead: it might depend on how each poll is conducted and whose voices it manages to capture.
An IVR poll might overestimate support among a certain group if that group is particularly animated, while an online poll might present a more consistent picture if it successfully engages a diverse set of participants. Neither method is entirely without flaw—IVR might detect a growing passion before it becomes mainstream, while online polling might miss some very niche but fervent opinions.
My experience is that online polling tends to be more consistent and slower to shift while random sampling using IVR or live interview swings more. Just look at the differences in mode during the last federal election to see evidence of this.
Ultimately, informed voters and observers should look at the methodology used in each survey and weigh how that method’s strengths and weaknesses may affect the reported numbers. It is also wise to compare multiple polls over time and the performance of the mode in the past. By doing so, you can better discern whether a dramatic lead is likely to hold or if it reflects a momentary snapshot of a particularly vocal minority.
ABOUT ABACUS DATA
We are Canada’s most sought-after, influential, and impactful polling and market research firm. We are hired by many of North America’s most respected and influential brands and organizations.
We use the latest technology, sound science, and deep experience to generate top-flight research-based advice to our clients. We offer global research capacity with a strong focus on customer service, attention to detail, and exceptional value.
And we are growing throughout all parts of Canada and the United States and have capacity for new clients who want high quality research insights with enlightened hospitality.
Our record speaks for itself: we were one of the most accurate pollsters conducting research during the 2021 Canadian election following up on our outstanding record in the 2019, 2015, and 2011 federal elections.
Contact us with any questions.
Find out more about how we can help your organization by downloading our corporate profile and service offering.
