Kanika Samuels-Wortley, Ontario Tech University
The first time my home security system flagged me as an animal, I was sitting at my patio table with my morning coffee a few years ago. I figured it was a glitch. A few months later, it happened again as I walked across the yard to catch the afternoon sun.
More recently, while I was on vacation, the same system logged my South Asian teenaged neighbour, whom I’d hired to cut the grass, as an animal too. This has never happened to my husband, who is white. Two racialized people appear to produce one pattern.
My son, who is in elementary school, also told me he couldn’t chat with a friend online. I had parental controls set up on the platform, and the age verification had read his friend, a Black boy the same age, as much older than him and blocked them from communicating.
This is no longer a glitch, I realized. We have a real problem.
I keep returning to these moments following Prime Minister Mark Carney’s first Canada Investment Summit, which asked the world’s largest investors to put money into Canadian advanced technology. Energy, critical minerals, defence and technology are the four things Canada is selling, yet nobody on the summit agenda appeared to ask whose faces these systems fail to recognize.
As a Canada Research Chair examining how bias shapes technologies adopted by the criminal justice system, I wonder: Can we trust these systems built to keep us from harm when they can’t reliably tell us apart from wildlife? Can we trust these systems that reinforce the adultification of Black boys?
Algorithms that cause harm
Canada recently released a national AI strategy, premised on AI for all. While this is well intended, “for all” is hard to imagine when systems read me, or anyone with darker skin, as something other than human.
The strategy recognizes that “biases built into algorithms can cause harm to vulnerable communities,” but it stops short of acknowledging how that harm erodes trust in these systems among racialized people in particular.
Some may question the focus on race. However, it’s necessary when system errors keep reproducing long-standing stereotypes that cast Black and brown bodies as animalistic, or Black children as older than they are. And this matters as AI-powered surveillance cameras become more popular in Canada.
Support for AI in policing
Take policing. Canadian law enforcement agencies remain among the most trusted institutions in this country. That trust, however, has always been lower among Black and Indigenous people.
I was surprised, then, when my national study on public perceptions of AI in Canadian policing across racial groups found that 60 percent of Black and 62 percent of South Asian respondents supported AI in policing, even at a potentially high financial cost, compared to only 46 percent of white respondents.
Furthermore, white respondents were more likely than Black and Indigenous respondents to rate transparency, ethical safeguards, regulation, and public input as important.
Read quickly, that might look like indifference among racialized people. But perhaps white respondents expect transparency and ethical conduct by the police, whereas for Black and Indigenous respondents, those principles have historically been absent. Their support is conditional on meaningful demonstration rather than a stated promise.
Bias is not a moral footnote
Where these racial groups did agree, in my research, was on government regulation of AI in policing. Every group rated it as important, suggesting that Canadians see AI as a viable public safety tool, but want real accountability.
Trust is the variable that decides whether these systems earn legitimacy, but whose trust does the government seek? The federal AI Strategy Task Force did not include any Black members until a prominent legal scholar pushed for one. Now there is one single Black voice advising the country on a technology whose failures may affect an entire community.
The government is right to push for more home-grown technology, since much of what enters Canadian institutions is built abroad and trained on populations that look nothing like our own. But building in Canada has to mean building with people at the table who are empowered to ask hard questions. While this does not guarantee better systems, it can create the conditions for real oversight and measured deployment instead of legal repercussions later.
So, as Ottawa goes looking for billions over the next five years with advanced technology on the pitch list, bias cannot be a moral footnote. It must be central to the agenda. We need to ask upfront who these systems are most likely to fail, how we will know and what it will take to address those failures.
If we don’t, Canada risks not only perpetuating bias but eroding its citizens’ trust. People like me would like to sit in their own backyard without being mistaken for wildlife. That’s a low bar for a country that says it’s building technology for all.
Kanika Samuels-Wortley is an associate professor in the Faculty of Social Science and Humanities at Ontario Tech University. This article is republished from The Conversation under a Creative Commons license. Read the original article.
The Toronto Spark is on Bluesky @torontospark.bsky.social.
