ProPublica – by Julia Angwin, Terry Parris Jr. and Surya Mattu
YOU MAY NOT REALIZE IT, but every website you visit is created, literally, the moment you arrive. Each element of the page — the pictures, the ads, the text, the comments — live on computers in different places and are sent to your device when you request them.
That means that it’s easy for companies to create different web pages for different people. Sometimes that customization is helpful, such as when you see search results for restaurants near you. Sometimes it can be creepy, such as when ads follow you around from website to website. And sometimes customization can cost you money, research has shown. Orbitz showed higher-priced hotels to owners of Mac computers, for instance. Staples offered the same products at higher prices to people living in certain ZIP codes.
Last year, we found that The Princeton Review was charging different prices for its online SAT tutoring course in different ZIP codes. In some ZIP codes, the course cost $6,600; in others that same course was offered for as much as $8,400.
Charging different prices to different geographic regions is regulated in Europe, but is not in the United States. In this case, it resulted in inadvertent discrimination. Our analysis found that Asians were nearly twice as likely to get that higher price from The Princeton Review than non-Asians. Asians make up 4.9 percent of the U.S. population overall, but they accounted for more than 8 percent of the population in areas where The Princeton Review was charging higher prices for its SAT prep packages.
Consider the difference between two ZIP codes with similar incomes in Texas. In Houston’s ZIP code 77072, with a relatively large Asian population, the Princeton Review course was offered for $7,200. While in Dallas’ ZIP code 75203, with almost no Asians, the course was offered for $6,600. And in heavily Asian, low-income Queens ZIP code 11355, the course was offered for $8,400.
Princeton Review told us the pricing differences reflected the varying costs of running its business and did not reflect any discrimination on their part.
But that’s the thing with algorithms — they can discriminate unintentionally. And as we enter a world of mass customization, we need to be on the lookout for this kind of discrimination.
https://www.propublica.org/article/breaking-the-black-box-when-algorithms-decide-what-you-pay