Cathy O'Neil

NEW YORK TIMES  BESTSELLER  •  A former Wall Street quant sounds the alarm on Big Data and the mathematical models that threaten to rip apart our social fabric—with a new afterword   “A manual for the twenty-first-century citizen . . . relevant and urgent.”— Financial Times   NATIONAL BOOK AWARD LONGLIST • NAMED ONE OF THE BEST BOOKS OF THE YEAR BY  The New York Times Book Review  • The Boston Globe • Wired  •  Fortune  •  Kirkus Reviews  •  The Guardian  •  Nature  •  On Point   We live in the age of the algorithm. Increasingly, the decisions that affect our lives—where we go to school, whether we can get a job or a loan, how much we pay for health insurance—are being made not by humans, but by machines. In theory, this should lead to greater fairness: Everyone is judged according to the same rules.   But as mathematician and data scientist Cathy O’Neil reveals, the mathematical models being used today are unregulated and uncontestable, even when they’re wrong. Most troubling, they reinforce discrimination—propping up the lucky, punishing the downtrodden, and undermining our democracy in the process. Welcome to the dark side of Big Data.

A clear-eyed warning about the increasingly destructive influence of America’s “shame industrial complex” in the age of social media and hyperpartisan politics—from the New York Times bestselling author of Weapons of Math Destruction Shame is a powerful and sometimes useful tool: When we publicly shame corrupt politicians, abusive celebrities, or predatory corporations, we reinforce values of fairness and justice. But as Cathy O’Neil argues in this revelatory book, shaming has taken a new and dangerous turn. It is increasingly being weaponized—used as a way to shift responsibility for social problems from institutions to individuals. Shaming children for not being able to afford school lunches or adults for not being able to find work lets us off the hook as a society. After all, why pay higher taxes to fund programs for people who are fundamentally unworthy? O’Neil explores the machinery behind all this shame, showing how governments, corporations, and the healthcare system capitalize on it. There are damning stories of rehab clinics, reentry programs, drug and diet companies, and social media platforms—all of which profit from “punching down” on the vulnerable. Woven throughout The Shame Machine is the story of O’Neil’s own struggle with body image and her recent decision to undergo weight-loss surgery, shaking off decades of shame. With clarity and nuance, O’Neil dissects the relationship between shame and power. Whom does the system serve? Is it counter-productive to call out racists, misogynists, and vaccine skeptics? If so, when should someone be “canceled”? How do current incentive structures perpetuate the shaming cycle? And, most important, how can we all fight back?

Now that people are aware that data can make the difference in an election or a business model, data science as an occupation is gaining ground. But how can you get started working in a wide-ranging, interdisciplinary field that’s so clouded in hype? This insightful book, based on Columbia University’s Introduction to Data Science class, tells you what you need to know. In many of these chapter-long lectures, data scientists from companies such as Google, Microsoft, and eBay share new algorithms, methods, and models by presenting case studies and the code they use. If you’re familiar with linear algebra, probability, and statistics, and have programming experience, this book is an ideal introduction to data science. Topics include: Statistical inference, exploratory data analysis, and the data science process Algorithms Spam filters, Naive Bayes, and data wrangling Logistic regression Financial modeling Recommendation engines and causality Data visualization Social networks and data journalism Data engineering, MapReduce, Pregel, and Hadoop Doing Data Science is collaboration between course instructor Rachel Schutt, Senior VP of Data Science at News Corp, and data science consultant Cathy O’Neil, a senior data scientist at Johnson Research Labs, who attended and blogged about the course.