“Under most normal conditions, the driverless car will recognise a stop sign for what it is. But not all conditions are normal. Some recent demonstrations have shown that a few black stickers on a stop sign can fool the algorithm into thinking that the stop sign is a 60 mph sign. Subjected to something frighteningly similar to the high-contrast shade of a tree, the algorithm hallucinates.
But in practice, algorithms are often proprietary black boxes whose updating is commercially protected. Cathy O’Neil’s Weapons of Math Destruction (2016) describes a veritable freakshow of commercial algorithms whose insidious pathologies play out collectively to ruin peoples’ lives. The algorithmic faultline that separates the wealthy from the poor is particularly compelling. Poorer people are more likely to have bad credit, to live in high-crime areas, and to be surrounded by other poor people with similar problems. Because of this, algorithms target these individuals for misleading ads that prey on their desperation, offer them subprime loans, and send more police to their neighbourhoods, increasing the likelihood that they will be stopped by police for crimes committed at similar rates in wealthier neighbourhoods. Algorithms used by the judicial system give these individuals longer prison sentences, reduce their chances for parole, block them from jobs, increase their mortgage rates, demand higher premiums for insurance, and so on.
This algorithmic death spiral is hidden in nesting dolls of black boxes: black-box algorithms that hide their processing in high-dimensional thoughts that we can’t access are further hidden in black boxes of proprietary ownership. This has prompted some places, such as New York City, to propose laws enforcing the monitoring of fairness in algorithms used by municipal services. But if we can’t detect bias in ourselves, why would we expect to detect it in our algorithms?
By training algorithms on human data, they learn our biases. One recent study led by Aylin Caliskan at Princeton University found that algorithms trained on the news learned racial and gender biases essentially overnight. As Caliskan noted: ‘Many people think machines are not biased. But machines are trained on human data. And humans are biased.’ […]
The problem is not visible in our hardware. It’s in our software. The many ways our minds go wrong make each mental-health problem unique unto itself. We sort them into broad categories such as schizophrenia and Asperger’s syndrome, but most are spectrum disorders that cover symptoms we all share to different degrees. In 2006, the psychologists Matthew Keller and Geoffrey Miller argued that this is an inevitable property of the way that brains are built.
There is a lot that can go wrong in minds such as ours. Carl Jung once suggested that in every sane man hides a lunatic. As our algorithms become more like ourselves, it is getting easier to hide.”
read more here: original article
picture source: unknown to me