![\"An](\"http:\/\/www.washnow.me\/wp-content\/uploads\/2024\/03\/AI_LEAD.0.jpeg\")
Researchers tried to get AI optimists and pessimists on the same page. It didn\u2019t quite work.<\/p>\n
I\u2019ve written<\/a> a lot<\/a> about AI<\/a> and the debate over whether it could kill us all<\/a>. But I still don\u2019t really know where I come down.<\/p>\n There are people who deeply understand advanced machine learning systems who think they will prove increasingly uncontrollable<\/a>, possibly \u201cgo rogue,\u201d<\/a> and threaten humanity with catastrophe<\/a> or even extinction. There are other people who deeply understand how these systems work who think that we\u2019re perfectly able to control them<\/a>, that their dangers do not include human extinction<\/a>, and that the first group is full of hysterical alarmists<\/a>.<\/p>\n How do we tell who\u2019s right? I sure don\u2019t know. <\/p>\n But a clever new study<\/a> from the Forecasting Research Institute tries to find out. The authors (Josh Rosenberg, Ezra Karger, Avital Morris, Molly Hickman<\/a>, Rose Hadshar, Zachary Jacobs, and forecasting godfather Philip Tetlock<\/a>) had previously asked both experts on AI<\/a> and other existential risks, and \u201csuperforecasters\u201d with a demonstrated track record of successfully predicting world events in the near term, to assess the danger that AI poses. <\/p>\n The result? The two groups disagreed a lot<\/a>. The experts in the study were in general much more nervous than the superforecasters and put far higher odds on disaster.<\/p>\n The researchers wanted to know why these groups disagreed so profoundly. So the authors set up an \u201cadversarial collaboration\u201d: They had the two groups spend many hours (a median of 31 hours for the experts, 80 hours for the superforecasters) reading new materials and, most importantly, discussing these issues with people of the opposite view with a moderator. The idea was to see if exposing each group to more information, and to the best arguments of the other group, would get either to change their minds. <\/p>\n The researchers were also curious to find \u201ccruxes\u201d: issues that help explain people\u2019s beliefs, and on which new information might change their mind. One of the biggest cruxes, for example, was \u201cWill METR<\/a> [an AI evaluator] or a similar organization find evidence of AI having the ability to autonomously replicate, acquire resources, and avoid shutdown before 2030?\u201d If the answer turns out to be \u201cyes,\u201d skeptics (the superforecasters) say they will become more worried about AI risks. If the answer turns out to be \u201cno,\u201d AI pessimists (the experts) say they will become more sanguine.<\/p>\n So did everyone just converge on the correct answer? \u2026 No. Things were not destined to be that easy. <\/p>\n The AI pessimists adjusted their odds of a catastrophe before the year 2100 down from 25 to 20 percent; the optimists moved theirs up from 0.1 to 0.12 percent. Both groups stayed close to where they started.<\/p>\n But the report is fascinating nonetheless. It\u2019s a rare attempt to bring together smart, well-informed people who disagree. While doing so didn\u2019t resolve<\/em> that disagreement, it shed a lot of light on where those points of division came from.<\/p>\n The paper focuses on disagreement around AI\u2019s potential to either wipe humanity out or cause an \u201cunrecoverable collapse,\u201d in which the human population shrinks to under 1 million for a million or more years, or global GDP falls to under $1 trillion (less than 1 percent of its current value<\/a>) for a million years or more. At the risk of being crude, I think we can summarize these scenarios as \u201cextinction or, at best, hell on earth.\u201d<\/p>\n There are, of course, a number of other different risks from AI worth worrying about, many of which we already face today. <\/p>\n Existing AI systems sometimes exhibit worrying racial and gender biases<\/a>; they can be unreliable<\/a> in ways that cause problems when we rely upon them anyway; they can be used to bad ends, like creating fake news clips to fool the public<\/a> or making pornography with the faces of unconsenting people<\/a>. <\/p>\n But these harms, while surely bad, obviously pale in comparison to \u201closing control of the AIs such that everyone dies.\u201d The researchers chose to focus on the extreme, existential scenarios.<\/p>\n So why do people disagree on the chances of these scenarios coming true? It\u2019s not due to differences in access to information, or a lack of exposure to differing viewpoints. If it were, the adversarial collaboration, which consisted of massive exposure to new information and contrary opinions, would have moved people\u2019s beliefs more dramatically.<\/p>\n Nor, interestingly, was much of the disagreement explainable by different beliefs about what will happen with AI in the next few years. When the researchers paired up optimists and pessimists and compared their odds of catastrophe, their average gap in odds was 22.7 percentage points. The most informative \u201ccrux\u201d (an AI evaluator finding that a model had highly dangerous abilities before 2030) only reduced that gap by 1.2 percentage points. <\/p>\n Short-term timelines are not nothing, but it\u2019s just not where the main disagreements are.<\/p>\n What did seem to matter were different views on the long-term<\/em> future. The AI optimists generally thought that human-level AI will take longer to build than the pessimists believed. As one optimist told the researchers, \u201clanguage models are just that: models of language, not digital hyperhumanoid Machiavellis working to their own end\u201d; this optimist thought that fundamental breakthroughs in methods of machine learning are necessary to reach human-level intelligence.<\/p>\n Many cited the need for robotics to reach human levels, not just software AI, and argued that doing so would be much harder<\/a>. It\u2019s one thing to write code and text in a laptop; it\u2019s quite another to, as a machine, learn how to flip a pancake or clean a tile floor or any of the many other physical tasks at which humans now outperform robots<\/a>.<\/p>\n The most interesting source of splits the researchers identified was what they call \u201cfundamental worldview disagreements.\u201d That\u2019s a fancy way of saying that they disagree about where the burden of proof lies in this debate. <\/p>\n \u201cBoth groups agree that \u2018extraordinary claims require extraordinary evidence,\u2019 but they disagree about which claims are extraordinary,\u201d the researchers summarize. \u201cIs it extraordinary to believe that AI will kill all of humanity when humanity has been around for hundreds of thousands of years, or is it extraordinary to believe that humanity would continue to survive alongside smarter-than-human AI?\u201d<\/p>\n It\u2019s a fair question! My experience is that most laypeople outside AI find \u201cthe machines will kill us all\u201d to be the more extraordinary claim. But I can see where the pessimists are coming from. Their basic view is that the emergence of superhuman AI is like the arrival on earth of a superhuman alien species. We don\u2019t know if that species would want to kill us all. <\/p>\n But Homo sapiens didn\u2019t necessarily want<\/em> to kill all the Homo erectus or Neanderthals hundreds of thousands of years ago, when multiple intelligent species of large apes were coexisting. We did, however, kill them all off. <\/p>\n Extinction tends to happen to dumber, weaker species when a smarter species that\u2019s better at claiming resources for itself emerges. If you have this worldview, the burden of proof is on optimists to show why super-intelligent AI wouldn\u2019t<\/em> result in catastrophe. Or, as one pessimist in the study put it: \u201cthere are loads of ways this could go and very few of them leave humans alive.\u201d<\/p>\n This is not the most encouraging conclusion for the study to arrive at. A disagreement driven by concrete differences of opinion about what will happen in the next few years is an easier disagreement to resolve \u2014 one based on how the next few years proceed rather than on deep, hard-to-change differences in people\u2019s assumptions about how the world works, and about where the burden of proof should fall.<\/p>\n The paper reminded me of an interview I saw a long time ago with the late philosopher Hilary Putnam. Working in the late 20th century, Putnam was a believer that philosophy could make progress, even if the big questions \u2014 What is truth? How does the mind work? Is there an external reality we can grasp? \u2014 feel as hard to answer as ever. <\/p>\n Sure, we don\u2019t know those answers, Putnam said<\/a>. But we know more about the questions. \u201cWe learn more about how difficult they are, and why they\u2019re so difficult. That is maybe the lasting philosophical progress.\u201d<\/p>\n That\u2019s how I felt reading the Forecasting Research Institute paper. I don\u2019t feel like I know for sure how worried to be about AI. But I do feel like I know more about why this is a hard question.<\/p>\n A version of this story originally appeared in the <\/em>Future Perfect<\/strong><\/em><\/a> newsletter. <\/em>Sign up here!<\/strong><\/em><\/a><\/p>\n","protected":false},"excerpt":{"rendered":" Javier Zarracina\/Vox Researchers tried to get AI optimists and pessimists on the same page. It didn\u2019t quite work. I\u2019ve written a lot about AI and the debate over whether it could kill us all. But I still don\u2019t really know…<\/p>\n","protected":false},"author":1,"featured_media":355,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[9],"tags":[],"_links":{"self":[{"href":"http:\/\/www.washnow.me\/index.php\/wp-json\/wp\/v2\/posts\/353"}],"collection":[{"href":"http:\/\/www.washnow.me\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.washnow.me\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.washnow.me\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.washnow.me\/index.php\/wp-json\/wp\/v2\/comments?post=353"}],"version-history":[{"count":2,"href":"http:\/\/www.washnow.me\/index.php\/wp-json\/wp\/v2\/posts\/353\/revisions"}],"predecessor-version":[{"id":356,"href":"http:\/\/www.washnow.me\/index.php\/wp-json\/wp\/v2\/posts\/353\/revisions\/356"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/www.washnow.me\/index.php\/wp-json\/wp\/v2\/media\/355"}],"wp:attachment":[{"href":"http:\/\/www.washnow.me\/index.php\/wp-json\/wp\/v2\/media?parent=353"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.washnow.me\/index.php\/wp-json\/wp\/v2\/categories?post=353"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.washnow.me\/index.php\/wp-json\/wp\/v2\/tags?post=353"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Why people disagree about AI\u2019s dangers<\/h3>\n
When disputes go deep<\/h3>\n