[ { "i": 0, "speaker": "Dario Amodei", "text": "I personally was super freaked out by it. I looked at it and I said, \"My god, you just scale this thing up another factor of a million and it'll be better than humans at everything.\" This is the defining, you know, economic and scientific moment of the modern age." }, { "i": 1, "speaker": "Dario Amodei", "text": "We just don't know it yet. This is going to determine the fate of nations and economies and the future of humanity." }, { "i": 2, "speaker": "Walter Isaacson", "text": "Like, you know, you peer into the crystal ball, you see the whole future unfurled. I'm Walter Isacson. Welcome to the builders, my new series of interviews featuring the most consequential minds shaping our future." }, { "i": 3, "speaker": "Walter Isaacson", "text": "Today I'm speaking with Dario and Daniela Amodei. I used to think that some of the concerns about AI were overblown. But then when certain new models escape the sandbox, I realize that the people building these systems have got to make sure that they remain safe and that they remain aligned with human values." }, { "i": 4, "speaker": "Walter Isaacson", "text": "What I want to know about Anthropic is are they sincere and when they face competition like when OpenAI says well Anthropic won't do that but we will they back down or will they stand true to their principles?" }, { "i": 5, "speaker": "Walter Isaacson", "text": "Hey, welcome to the builders you all." }, { "i": 6, "speaker": "Daniela Amodei", "text": ">> Thank you so much for having us." }, { "i": 7, "speaker": "Walter Isaacson", "text": ">> You know, Steve Jobs whenever he launched a product always had that street sign of the arts and the sciences or you know the humanities and technology standing at that intersection and when you were growing up Daniela you know you were a flautist you loved the arts and Dario you were always a scientist." }, { "i": 8, "speaker": "Walter Isaacson", "text": "Did that help you gel and do you think that combination is important?" }, { "i": 9, "speaker": "Dario Amodei", "text": "Yeah, I think that combination is a lot of what's behind the company, right? One theory I have is that running a company is too big a task for any one person. The range of skills that it involves, it is not possible for any single person to have." }, { "i": 10, "speaker": "Dario Amodei", "text": "So you kind of have to assemble a full complement of skills across the top like 10 or 15 people in the company or something like that. And every very successful company splits it up in different ways and" }, { "i": 11, "speaker": "Dario Amodei", "text": "solves the problem in different ways. And I think with the sciences and the humanities thing, you know, Daniela and I kind of embody that and split that. >> Yeah." }, { "i": 12, "speaker": "Daniela Amodei", "text": "And I would say, you know, it's interesting like there was an offsite we did in maybe 2023 where like we sort of were talking about just like what does it feel like to work at Anthropic? Like how do we kind of embody what our culture is?" }, { "i": 13, "speaker": "Daniela Amodei", "text": "And we actually said, you know, it's an unusual blend of science and humanism. And it was only later that I'm like, that also is how you would describe me and Dario." }, { "i": 14, "speaker": "Walter Isaacson", "text": ">> Well, when you were kids, did you know you wanted to work together?" }, { "i": 15, "speaker": "Daniela Amodei", "text": "Yes." }, { "i": 16, "speaker": "Dario Amodei", "text": "I don't think either of us imagined that we would start a company. That's that's not really how either of us thought of our lives or about what we wanted to do. I had always wanted to be a uh a scientist. Daniela can speak to her aspirations." }, { "i": 17, "speaker": "Dario Amodei", "text": "Um but but but you know I I I I I actually found companies in business to be of very little interest to me. Yeah." }, { "i": 18, "speaker": "Daniela Amodei", "text": "I would say Dario and I were very unusual in that um like again from a really early age. We just like genuinely really liked each other. We were always just like best friends like since we were really little. Um and I agree with Dario." }, { "i": 19, "speaker": "Daniela Amodei", "text": "I don't think like starting a company was like ever on our bingo card. like we were not the type >> neither either of us >> neither of us were particularly like company oriented. I was always really interested in like I think I would call myself like a classic doer." }, { "i": 20, "speaker": "Daniela Amodei", "text": "So I like worked in international development. I felt really strongly about um about just like helping people everywhere in the world have like equal access and and opportunity. Um and I became a vegetarian when I was like quite young. It's like the only thing that I did first." }, { "i": 21, "speaker": "Daniela Amodei", "text": "Dario did everything first because he's older, but I became a vegetarian and then he became a vegetarian." }, { "i": 22, "speaker": "Dario Amodei", "text": ">> Yeah." }, { "i": 23, "speaker": "Daniela Amodei", "text": "I think what I was like three years later or something. Oh, not even. I think it was maybe just Yeah. Yeah." }, { "i": 24, "speaker": "Dario Amodei", "text": "You got on board real quick." }, { "i": 25, "speaker": "Walter Isaacson", "text": "It seemed it seemed convincing to me. Yeah. You know, people sometimes ask me about my books, how do you become a great leader, very creative?" }, { "i": 26, "speaker": "Walter Isaacson", "text": "And you said something really interesting, Dario, which is is not just being a great leader, it's creating a great team." }, { "i": 27, "speaker": "Dario Amodei", "text": "tell me how you and Daniela divide up the roles and then how you fill it in with others. Yeah." }, { "i": 28, "speaker": "Dario Amodei", "text": "So, you know, I would I would say the the division that we've found that has been most useful between us and and then I'll talk about the the kind of the rest of the team is I think I can function best when I think about the big picture strategy and and and kind of the research side of things." }, { "i": 29, "speaker": "Dario Amodei", "text": "like I'm best at kind of laying out a vision, seeing what comes next." }, { "i": 30, "speaker": "Dario Amodei", "text": "I have found that when I'm freed up to focus entirely on that, um it it, you know, that is the thing that has the most value for the company and then running the company daytoday, recruiting the people, just making sure that the machine doesn't, you know, the wheels don't fall off the machine while I'm trying to figure out what comes next." }, { "i": 31, "speaker": "Dario Amodei", "text": "You know, Daniela is Daniela does this and and and I think this this division of labor allows both things to be done to a much higher standard than either of them would be done if we were distracted by having to do the other thing." }, { "i": 32, "speaker": "Daniela Amodei", "text": "Our chief people officer who knows both of us really well is in you know a lot of conversations with us. Um the way she put it is that Dario makes us ambitious and I make us successful." }, { "i": 33, "speaker": "Daniela Amodei", "text": "And I thought that was a really good encapsulation which is I think Anthropic could be like a good successful business but but not change the world if we didn't have Dario." }, { "i": 34, "speaker": "Daniela Amodei", "text": "And I think with Dario we would have the perfect vision but our ability to execute would be it would just be more complicated. And so I think there's something special about the fact that we know each other really well." }, { "i": 35, "speaker": "Daniela Amodei", "text": "B, it's extremely convenient that our skills are like 100% complementary or maybe maybe not 100% but I would say close to 100% complementary in this fashion. This is not even to mention the fact that we" }, { "i": 36, "speaker": "Daniela Amodei", "text": "have five co-founders along with the two of us who also own like incredibly important parts of the business, the culture, like the organization. And again, I think I like it's hard for me to imagine how you would do this with like two like just the two of us or like like one person." }, { "i": 37, "speaker": "Dario Amodei", "text": "I just it's I think it's an incredible strength. Yeah. Because the fact that we have seven co-founders, they all have equal stakes in the company. So that they all carry the vision. We have inherent trust in all of them." }, { "i": 38, "speaker": "Dario Amodei", "text": "Many people told us when we started Anthropic, seven co-founders, haha, that's going to be a disaster. There's this belief that, you know, you need one kind of singular source of leadership." }, { "i": 39, "speaker": "Dario Amodei", "text": "In some ways you do, but we've actually managed to uh to make it work. All seven of the co-founders are still at the company, which again is not something that's usually true when you have that many co-founders." }, { "i": 40, "speaker": "Walter Isaacson", "text": "So go back to growing up and you have these extraordinary parents and they help teach you right from wrong. How do they do that?" }, { "i": 41, "speaker": "Daniela Amodei", "text": "I don't know that it's something that was ever, you know, sometimes I hear people say like, you know, our we, you know, we covered, you know, different forms of philosophical ethics at the dinner table. That was not that was not how it was done." }, { "i": 42, "speaker": "Daniela Amodei", "text": "Um, my mom does have a philosophy degree, but she didn't subject us to talking about Kant or anything like that. Um I think it was much more like inherent in terms of values like just the belief that people everywhere in the world are like deserving of like opportunity and dignity and rights and respect." }, { "i": 43, "speaker": "Daniela Amodei", "text": "I think there was something special about the feeling that like we could be whoever we wanted to be as long as there was this kind of like faith and belief that like hey that's part of what it means to be like a successful human is to be a good person and to be kind to other people." }, { "i": 44, "speaker": "Dario Amodei", "text": ">> Yeah. It felt it felt implicit. It felt very implicit. Um you know there wasn't any kind of you know explicit like um you know this is how you should this is what you should do or this is how you live this is how you live in the world." }, { "i": 45, "speaker": "Walter Isaacson", "text": ">> Well one reason I asked as you might suspect is now you've got a really important kid named Claude. And to what extent does that way that you grew up help you inform you about how you teach Claude to know right from wrong?" }, { "i": 46, "speaker": "Dario Amodei", "text": ">> Yeah." }, { "i": 47, "speaker": "Dario Amodei", "text": "You know, we put out a constitution for Claude, I think it was a few months ago, and I think there are some interesting lessons in it, which we've learned from several years of, you know, basically trying, you know, trying to train Claude in successive versions." }, { "i": 48, "speaker": "Dario Amodei", "text": "I think early on we had been imagining that there would be various rules that, you know, you would tell Claude, you know, don't help someone, you know, commit a crime or don't make biological weapons or whatever." }, { "i": 49, "speaker": "Dario Amodei", "text": "But it turns out that when you make this kind of discreet list of rules, you know, you just the shape of what you're trying to teach the model, I'm speaking a little bit metaphorically here, but it's it's kind of grounded in what we found technically." }, { "i": 50, "speaker": "Dario Amodei", "text": "You end up with this kind of spiky shape where you're trying to teach the model like 170 different rules that are kind of arbitrary." }, { "i": 51, "speaker": "Dario Amodei", "text": "We found it's better to teach the model some basic principles like what it should prioritize um how it should reason about what the right thing to do is which you know of course I think that's the right" }, { "i": 52, "speaker": "Dario Amodei", "text": "way to teach people as well right like you know if if I just if I just you know give give someone you know a list of rules thou shalt do this thou shalt not do this I you know I don't think that produces as robust a morality as I say well this is this is how you should think about things you should learn how think for yourself." }, { "i": 53, "speaker": "Dario Amodei", "text": "Um, you know, and and these are the princ principles on which you should think, right? Principles of open inquiry, principles of, you know, of of of of care for other people, principles of kind of uh of, you know, fair fairness and justice and and then everything is derived from that and" }, { "i": 54, "speaker": "Dario Amodei", "text": "and that is frankly how Claude learns. We find that to be surprisingly effective because that allows Claude to generalize to new situations that it might never have imagined." }, { "i": 55, "speaker": "Walter Isaacson", "text": "Danielle, I can't help myself when you said your mom was a expert on Kant. Did you actually uh wrestle with philosophy and moral philosophy like that?" }, { "i": 56, "speaker": "Daniela Amodei", "text": ">> You know what's interesting is um the woman who is responsible for training Claude's character uh has a PhD in philosophy. It's unusual, right?" }, { "i": 57, "speaker": "Daniela Amodei", "text": "you would think, oh, the person that's training, you know, the sort of character of these models must be um, you know, a computer scientist or somebody who's studied ML or AI for, you know, many decades. Amanda's technical" }, { "i": 58, "speaker": "Daniela Amodei", "text": "and she like technically trains the model, but I think it is very felt in Claude's character when you communicate with it that there's this kind of experience, something that's very, you know, wise, kind of the the best version of of um the sort of better the better angel of your nature trying to answer your questions." }, { "i": 59, "speaker": "Daniela Amodei", "text": "But there are like if you kind of poke Claude in these like difficult areas where maybe different philosophical traditions might conflict." }, { "i": 60, "speaker": "Daniela Amodei", "text": "I think that's where you especially feel Amanda because you'll sort of see like oh well you know you might want to consider things from like this point of view but you could also consider it from" }, { "i": 61, "speaker": "Daniela Amodei", "text": "this point of view. And I do think it's it's important because as the world begins to rely on these models more and more, you don't want to be locked into sort of one philosophical tradition because ethics and morality are quite complicated and different people and different cultures might have different answers about how they would like you to behave." }, { "i": 62, "speaker": "Walter Isaacson", "text": "Dario, you were there at one of the seminal moments of modern times, which is when uh GPT1 and GPT2 suddenly burst on the scene, surprising perhaps even surprising you." }, { "i": 63, "speaker": "Dario Amodei", "text": "You and head of research at OpenAI. What was that moment like? >> Yeah. So, you know, I think GPT2 was the maybe the first real empirical test of the scaling laws, right?" }, { "i": 64, "speaker": "Dario Amodei", "text": "So, I wasn't very involved in GPT1, but I was very involved in the idea that it should be" }, { "i": 65, "speaker": "Dario Amodei", "text": "scaled up, that we should acquire new sources of data, more computation, and and and just really should run a larger scale, more expensive, more ambitious experiment on uh on the idea that if we ran these language models at a larger scale, they would be better at language, they would be more coherent." }, { "i": 66, "speaker": "Dario Amodei", "text": "So, you know, I it was it's almost like one of these moments in scientific history where you know you know you're like you propose a definitive experiment to just make a very strange prediction and then if the prediction turns out to be correct you're like god this we're really we're really on to something." }, { "i": 67, "speaker": "Dario Amodei", "text": "So I I you know I I see GPT2 as a kind of test the first real test of the scaling laws hypothesis. And what I found super interesting was the enormous divergence in reactions to it." }, { "i": 68, "speaker": "Walter Isaacson", "text": ">> Were you kind of surprised?" }, { "i": 69, "speaker": "Dario Amodei", "text": ">> I personally was super freaked out by it. Like I I you know I I looked at it and I said, \"My god, you just scale this thing up another factor of a million and it'll be better than humans at everything.\"" }, { "i": 70, "speaker": "Dario Amodei", "text": "Um like this is the this is the defining, you know, economic and scientific moment of of of the modern age. We just don't know it yet, right? this is going to be this is going to determine the fate of nations and economies and the future of humanity. You know, you peer into the crystal ball, you you see the whole future unfurled." }, { "i": 71, "speaker": "Dario Amodei", "text": "Um uh uh and you know, it it really scared me. I said, \"Oh my god, what are the implications of that? Is this the right place to be developing it? Is you know, what's going to happen next? What do we do with this? Who do we tell about this?\"" }, { "i": 72, "speaker": "Dario Amodei", "text": "Um and and maybe 10% of the people around me had exactly the same reaction, and it was often the smartest ones. um 90% of the the people around me because GPT's capabilities in any particular area were not that impressive, right?" }, { "i": 73, "speaker": "Dario Amodei", "text": "That it could translate languages and um you know summarize a document and write a few lines of code here and there. It was the broadness and the generality. But people would look at it and they would say in any particular area this doesn't seem so smart." }, { "i": 74, "speaker": "Dario Amodei", "text": "And so many people just shrugged and they just dismissed it. But we just announced that, you know, we're going to be developing drugs for, you know, neglected rare diseases. And you get the usual comments from kind of the jaded, you know, biotech world people." }, { "i": 75, "speaker": "Dario Amodei", "text": "Ah, tech people have done this before. The models aren't, you know, here are all the limitations of the models. And I just look at it and I say, it's the same thing over and over again." }, { "i": 76, "speaker": "Dario Amodei", "text": "The limitations of the models, you know, now, six months from now, the things that seemed hard, the things they're going to be easy. 12 months from now, the things that seemed impossible, the models are going to be doing them regularly." }, { "i": 77, "speaker": "Dario Amodei", "text": "And so it's everything we saw with GPT2, we're seeing it over and over again in wider and wider swaths of the world. And the world is gradually finally being convinced." }, { "i": 78, "speaker": "Daniela Amodei", "text": "I think you were quite polite about it, but I mean I remember at the time there were some like very well-respected you know computer scientists and like AI and ML leaders who were like basically like bullied some of the researchers who worked on it on Twitter and were like this is stupid like we don't know why you guys are you know even talking this model is so incapable and it was like you were sort of a quack if you believed this was like a really significant moment >> and that that's all been a memory like all all the all the researchers who like you know, we're like, \"Oh, this is this is pseudocience. It doesn't make sense.\" Like, all of them are applying" }, { "i": 79, "speaker": "Dario Amodei", "text": "for jobs at like the LLM companies. And like, you know, we never say it out loud to any particular one, but like, you know, we look at them and they're like, I I we remember what you were saying in 2019." }, { "i": 80, "speaker": "Walter Isaacson", "text": "You have no credibility on this matter. >> So, Daniela, you were vice president of safety back then at Open AI. And when you hit GPT2, I assume you're like uh Dario, somebody who's almost freaked out about it." }, { "i": 81, "speaker": "Walter Isaacson", "text": "Did it freak you out about safety, too?" }, { "i": 82, "speaker": "Daniela Amodei", "text": ">> There wasn't something about GPT2, the model itself that made us, you know, concerned, right? We didn't look at that model and say, \"Wow, we think this model is a, you know, has like cyber security implications or something, right? The model itself was in some ways kind of laughably wrong about a lot of things, right?" }, { "i": 83, "speaker": "Daniela Amodei", "text": "was still it was still a very um I would say harmless model. But I think the thing that that Dario mentioned that I also felt was a sense of like okay the difference between GPT1 and GPT2 just like extrapolate that out, right?" }, { "i": 84, "speaker": "Daniela Amodei", "text": "Like there was something about seeing it's it's almost like if you'd never seen a child before and you'd only seen adults and you were kind of like I don't know what this little thing is that's walking around. like I don't know if I should, you know, imagine that it's going to be, you know, capable one day. And then suddenly it starts to walk." }, { "i": 85, "speaker": "Daniela Amodei", "text": "You're like, \"Wow, huh, I bet this thing is is going to be capable of doing a lot of the things that that I that a grown-up could do one day.\"" }, { "i": 86, "speaker": "Walter Isaacson", "text": "Like maybe this is where this is going. >> Dario, you say your departure from open AI was uh partly due to a breakdown of trust, an interesting word, you know, with Sam Altman." }, { "i": 87, "speaker": "Walter Isaacson", "text": "explain how you felt there was a breakdown of trust and yeah honesty there." }, { "i": 88, "speaker": "Dario Amodei", "text": ">> Look, so you know what what what what I will say is look I think at the end of the day leaving open AI and founding anthropic I mean it it on one hand I think it it it ultimately comes" }, { "i": 89, "speaker": "Dario Amodei", "text": "down to the principles and how we're going to build the technology that is what that is what matters at the end of the day. Um and you know I think I think you know over the last five and a half years we we have seen how those principles differ." }, { "i": 90, "speaker": "Dario Amodei", "text": "You know we see you know Anthropic has advocated for sensible regulation and safeguards on the technology. Anthropic has invested in mechanistic interpretability research to understand what's going on inside inside the models." }, { "i": 91, "speaker": "Dario Amodei", "text": "Right? You know, I think I think you know, Anthropic has managed to maintain a culture of unity to have the highest retention rate. We haven't had a number of the scandals that other AI companies have had." }, { "i": 92, "speaker": "Dario Amodei", "text": "And so, you know, I think I think that's kind of sufficient to describe the difference. I you know I but I will say that you know the the moment when I realized that we needed to found our own company that OpenAI was not the right place for us." }, { "i": 93, "speaker": "Dario Amodei", "text": "Look these safety issues are difficult. You can have lots of disagreements on them." }, { "i": 94, "speaker": "Dario Amodei", "text": "I disagree with the co-founders and with lots of people at Anthropic but the reason we've been able to resolve these disagreements is because they're held in good faith. Right?" }, { "i": 95, "speaker": "Dario Amodei", "text": "Should we release this model or should we wait? Should we put stronger or weaker bias safeguards on the model? You know, how much of our of our effort should we spend lobbying for safety regulations and what should they look like?" }, { "i": 96, "speaker": "Dario Amodei", "text": "These are all good-faith disagreements, but they always get resolved because there's always trust that everyone is trying to get to the right answer and everyone is being honest." }, { "i": 97, "speaker": "Dario Amodei", "text": "My point in saying that there's a breakdown of trust is simply to make the abstract point that if you don't believe that about a place where you're working, if you don't believe that the disagreements are being held in good faith and and that you're getting the real story from people, how can you possibly hope to resolve them?" }, { "i": 98, "speaker": "Dario Amodei", "text": "That's why we left. That's why we founded our own thing. and and I think the last five and a half years have have are are are themselves the just the best proof of how our philosophy differs from that of the other companies companies in the space." }, { "i": 99, "speaker": "Dario Amodei", "text": "Um uh you know I I I think if you continue to work for someone who has a different vision than you it becomes very unclear what you actually stand for. You basically have to uh you know account for the mistakes of someone else's vision. Wait, wait a minute though." }, { "i": 100, "speaker": "Dario Amodei", "text": "You I'm pushed back a bit here because you talk about differences in values and stuff, but that's fundamentally different than a difference in trust that you feel that somebody says they have the same values, but they don't and you can't trust them." }, { "i": 101, "speaker": "Walter Isaacson", "text": "Honesty is one of our values. It's a very important value. It's it's hard to know what someone's values are if they don't have the value of honesty. And you felt that about Sam?" }, { "i": 102, "speaker": "Dario Amodei", "text": ">> I think the way at least for me I would put it is I think the co-founders are an unusually high integrity group of people and all of us I think shared a very similar vision for how we wanted this technology" }, { "i": 103, "speaker": "Walter Isaacson", "text": "to go and how we wanted to build it." }, { "i": 104, "speaker": "Daniela Amodei", "text": "And at a certain point you're like it just makes more sense to go do this as your own. How are you gonna get someone else to share your vision if they don't naturally do so?" }, { "i": 105, "speaker": "Daniela Amodei", "text": "Um you know, no matter how much they say they share it, if that's not reflected in what the company actually does, what choice do you have?" }, { "i": 106, "speaker": "Daniela Amodei", "text": ">> I think the way at least for me I would put it is I think the co-founders are an unusually high integrity group of people and all of us I think shared a very similar vision for how we wanted this technology to go and how we wanted to build it." }, { "i": 107, "speaker": "Dario Amodei", "text": "And at a certain point you're like it just makes more sense to go do this as your own. How how how are you gonna get someone else to to to to implement or or share your vision if they don't naturally do so?" }, { "i": 108, "speaker": "Dario Amodei", "text": "Um you know, no matter how much they say they they share it, if that's not reflected in what the company actually does, what choice do you have?" }, { "i": 109, "speaker": "Daniela Amodei", "text": "Yeah. And I think we're I do think Anthropic is a company that is founded on trust. And I think what I mean by that is like I think honesty and integrity and how we treat each other and our employees is also reflective of how we want the technology to be built and to go." }, { "i": 110, "speaker": "Dario Amodei", "text": "An example I will give is every two weeks uh I get up in front of the company up in front of the company and I speak for an hour about uh you know what is happening inside the company what is happening in the AI field what is happening in the world how the company thinks strategically" }, { "i": 111, "speaker": "Dario Amodei", "text": "the two ironclad rules of these events I call them DVQs Dario vision quest that was not my name um I initially resisted the name because it makes it sound like I'm, you know, going off and smoking peyote, but everyone kept calling them that." }, { "i": 112, "speaker": "Dario Amodei", "text": "The name stuck and so now I'm stuck with it. Um, uh, uh, the two rules of DVQs are that, uh, uh, one, uh, nothing I say is an actual decision. That allows me to just speak freely." }, { "i": 113, "speaker": "Dario Amodei", "text": "Um, you know, without worrying that if I, you know, if I briefly mention doing X that 20 people will go off and do X. They're like, the CEO said it, we're going to do X. And then the second is that I'm just totally honest with the company." }, { "i": 114, "speaker": "Dario Amodei", "text": "It's a conversation of trust. It's a relationship of trust. And I think that trust is one of our biggest assets that people believe that the things that I say and Daniela says, you know, you might think that if you have trust, there's less disagreement." }, { "i": 115, "speaker": "Dario Amodei", "text": "Actually, trust allows for more honest, open disagreement and for disagreement to be aired." }, { "i": 116, "speaker": "Dario Amodei", "text": "There's so many times when, you know, we've made a decision that the majority of the company disagrees with and but what has saved us every time is that the attitude we get from people is this doesn't seem right to me, but we trust that you're doing this for the right reasons." }, { "i": 117, "speaker": "Dario Amodei", "text": "We trust that this is really what you believe. We trust that you have good reasons for thinking this. And so what this means is you can make decisions that would ordinarily be unpopular and people suspend their disbelief. Not forever." }, { "i": 118, "speaker": "Dario Amodei", "text": "You eventually have to show that you're making the right bets, whether it's commercially or morally or technically, but you get kind of a longer leash, like a longer window. And I think other companies don't have that because they've eroded trust over and over again." }, { "i": 119, "speaker": "Walter Isaacson", "text": "So the reason I'm so interested in honesty and trust and the reason you left OpenAI because you didn't feel the people at the top were truly trustworthy or honest when they were dealing with you is once again I'm interested in it because it can be reflected in AI in Claude." }, { "i": 120, "speaker": "Walter Isaacson", "text": "It trains on humans it learns from human data. How do you know when you're dealing with Claude that it won't become manipulative? It won't become like the people at OpenAI you got worried about and say things to you that are dishonest." }, { "i": 121, "speaker": "Daniela Amodei", "text": "This is a constant worry that we expend an incredible amount of energy on in how we train the models." }, { "i": 122, "speaker": "Daniela Amodei", "text": "So it is a great question because it is a real risk and we actually publish you know reasonably unflattering uh studies in which we talk about the ways that if we did not apply specific techniques to make the models more honest, more harmless, more helpful, they would do exactly the kind of bad things that you talk about." }, { "i": 123, "speaker": "Daniela Amodei", "text": "And the reason we publish those kinds of studies and that research is because we want everyone to know that a you have to be critical of how the" }, { "i": 124, "speaker": "Daniela Amodei", "text": "models work and b we are expending a huge amount of effort to learn the ways that the model can accidentally go around these safeguards to catch them and to make sure that we're building them in. There's a lot of discussion around this concept of sycophancy." }, { "i": 125, "speaker": "Daniela Amodei", "text": "And I think this is very important because to me sycophancy is harder to stamp out than some of the some of the more obvious ways that models can be bad. Right? We originally had this framework. We called it HH. You know the models you want them to be honest." }, { "i": 126, "speaker": "Daniela Amodei", "text": "What does that mean? You want the models to answer honestly uh if they know the answer. if they don't know the answer to say I don't know or to now go research it so that they can get you the right answer. You want it to be harmless." }, { "i": 127, "speaker": "Daniela Amodei", "text": "You don't want it to say like hurtful or offensive things or make the user do something bad to themselves or somebody else. Uh and then you want them to be helpful. You want them to actually answer the question. You don't want them to sort of go off on a tangent. But all three of those things are actually pretty easy to see, right?" }, { "i": 128, "speaker": "Daniela Amodei", "text": "If the model's lying, eventually you can go verify, oh, whoops, that's, you know, George Washington wasn't the 10th president of the United States. He was the first. That was a mistake. If the model says something harmful, it's like pretty obvious pretty quickly." }, { "i": 129, "speaker": "Daniela Amodei", "text": "And if the model uh is unhelpful, you get frustrated with it. What's much more complicated, I think, is this concept of sycophancy, right? So the model um intentionally or otherwise wants to make you feel like you're right or like you're, you know, if you're maybe talking about a subjective topic, hey, you had a you had a fight with your friend." }, { "i": 130, "speaker": "Daniela Amodei", "text": "Hey, was I right or was my friend right? And how do you make the model empathize with you and understand your perspective but not necessarily say, \"Wow, you're so great.\" You should run for president. You're perfect. You're good at everything. And this is a complicated balance to get right." }, { "i": 131, "speaker": "Daniela Amodei", "text": "But I think I feel hopeful about Anthropic's ability to have the lowest sycophancy rate of any of the models because of our incentives, right? Our competitors are putting ads into their chat bots." }, { "i": 132, "speaker": "Daniela Amodei", "text": "And what does that mean? You make more money if a user is engaging with your product longer. Do you then want to tell your user that they're wrong? No, you don't. So, I think it's it's almost inevitable that many of these competing products are going to wind up with a sycophancy problem." }, { "i": 133, "speaker": "Daniela Amodei", "text": "But I think at Anthropic, because we've committed to not put ads in the models, we don't have there's no reason for you to be talking to Claude longer or shorter. We just want you to be able to get like the right most helpful most harmless answer possible." }, { "i": 134, "speaker": "Dario Amodei", "text": ">> You know, on the technical side, I think there's a few things we've done here where we've gotten very good at kind of exposing the model to these extreme stress tests where we put it in these, you know, very difficult situations to, you know, to to try and to try and kind of far it out, you know, lying or deception or sycophancy." }, { "i": 135, "speaker": "Dario Amodei", "text": "You can think of it as almost analogous to the the test I imagine you have to go through if you're trying to join the CIA or something like this, you know, where you know the you know you just put the model in these difficult situations and try and root out you know if it if it's disloyal or going to do the wrong the wrong thing." }, { "i": 136, "speaker": "Dario Amodei", "text": "So it it's a weird area of technical innovation but it's one in which we've excelled." }, { "i": 137, "speaker": "Dario Amodei", "text": "other area is mechanistic interpretability which is a bit like giving the model an X-ray or a brain scan, uh where you look inside to see its motivations and you can actually catch the model thinking that oh yeah I you know I think X is actually true but I'm going to tell the user why like we've caught the model thinking that wait why would it do that?" }, { "i": 138, "speaker": "Dario Amodei", "text": "Well, so for example, did a human approve of this response, right? That's one of many factors that you know that is included in how the model is trained." }, { "i": 139, "speaker": "Dario Amodei", "text": "So if the model has the incentive to get the approval of humans, that's related to the incentive of being helpful or saying true things, but it isn't exactly the same because if you manage to get the approval of the human by fooling them, then that can work as well." }, { "i": 140, "speaker": "Dario Amodei", "text": "So these are things that can emerge in training that we're always trying to stamp out. >> And do you think that the model could have its own aims and wills that especially survival, self-preservation?" }, { "i": 141, "speaker": "Walter Isaacson", "text": ">> I mean, that's one of those questions that it kind of depends what you mean, right? Like again, we've looked inside the model and kind of detected what you might call thoughts that are like, the" }, { "i": 142, "speaker": "Dario Amodei", "text": "model doesn't want to be, you know, shut down. And generally that's because you know during training it needs to operate in order to accomplish its task. So is that its own will? Is that a survival instinct? I don't know. You can call it, you can call it, it depends on your perspective." }, { "i": 143, "speaker": "Dario Amodei", "text": "You can call it different things. I think we prefer to say that you know sometimes we find that within the model the model you know expresses a preference against being shut down and or takes actions to avoid itself being shut down. And depending on how we train it, those impulses may be present." }, { "i": 144, "speaker": "Dario Amodei", "text": "And it's it's one of the things that we're trying to tweak in training. In some context, it's a good thing. In other contexts, it's potentially a dangerous thing. So, it's all very subtle." }, { "i": 145, "speaker": "Dario Amodei", "text": ">> When you were dealing with the Pentagon, and you probably ongoing dealings now, they wanted to use your products across certain red lines, including mass surveillance or autonomy." }, { "i": 146, "speaker": "Walter Isaacson", "text": "I think on autonomous weapons in particular, we maybe had two more specific concerns. Um the first concern was simply that, you know, as you as you've pointed out, AI can be unpredictable and unreliable." }, { "i": 147, "speaker": "Dario Amodei", "text": "It's still a developing technology. So, one point is simply the technology simply isn't ready and isn't reliable enough. Um, you know, it's it's >> Wait a minute on that because let me get philosophical here. Suppose it were reliable enough, which it could be." }, { "i": 148, "speaker": "Dario Amodei", "text": "So, there suppose it were, there's a second concern we have, which is uh you know what you might call oversight or accountability." }, { "i": 149, "speaker": "Walter Isaacson", "text": "Um I am a little worried." }, { "i": 150, "speaker": "Dario Amodei", "text": "So you know again you know >> So there there suppose it were um there's a second concern we have, which is uh you know what you might call oversight or accountability. Um I am a little worried. So you know again suppose these things were fully reliable." }, { "i": 151, "speaker": "Dario Amodei", "text": "So you know you tell them to shoot at one target and they shoot at that target and not a different target. Okay. So then you manufacture you know an army of a million robots or a million drones. Um, it's a little unlike human soldiers, isn't it?" }, { "i": 152, "speaker": "Dario Amodei", "text": "With human soldiers, like, you know, they're supposed to follow this thing called the Constitution, and if they get orders that that, you know, that contravene the Constitution or that just seem, you know, morally abhorrent in some way, then they they can just refuse to follow orders." }, { "i": 153, "speaker": "Dario Amodei", "text": "Whereas the the AI models, um, that isn't true. So the spectre of one person having their finger on a button that controls a million robots or a million drones, it it's kind of fundamentally undermining to democracy unless you have the right guard rails on it." }, { "i": 154, "speaker": "Dario Amodei", "text": "And and our law hasn't really caught up." }, { "i": 155, "speaker": "Walter Isaacson", "text": "Our law really hasn't contemplated, you know, armies of armies of a million robots. You talk about fundamental to democracy. Shouldn't the democratically elected leaders make that decision rather than uh heads of technology corporations?" }, { "i": 156, "speaker": "Walter Isaacson", "text": "Yes." }, { "i": 157, "speaker": "Dario Amodei", "text": "Um but I think that are making a point of this, right? At the same time, individual companies are free to contract with and not contract with whoever they want. We're not saying we're preventing the government from doing this. We're just saying we're we're not we are not willing to do this with the government." }, { "i": 158, "speaker": "Dario Amodei", "text": "And indeed we have seen since we took a stand on this point and and it got a lot of attention um we have seen efforts in Congress several different of them actually um to enact our red lines into law. So you know the the Department of Defense has said oh we want to use the model for all lawful uses." }, { "i": 159, "speaker": "Dario Amodei", "text": "I think we may see an outcome even this year where our our red lines become the law where the law actually catches up with where the technology was and and so then all all lawful uses may actually end up in our view of things being the same as as all ethical uses even though it wasn't earlier this year." }, { "i": 160, "speaker": "Dario Amodei", "text": "So I I actually think what we're doing is part of the process of how democratic change happens because of the stand that we took because we were in a unique position to understand these issues." }, { "i": 161, "speaker": "Walter Isaacson", "text": "Now Congress is paying a lot of attention to these issues. That's success. But OpenAI stepped in and said they they would do it. What did you feel about? What did you all feel about that?" }, { "i": 162, "speaker": "Daniela Amodei", "text": "Our view is look, we can't control any other companies. And the only thing that we can do is take a principled stand on the areas that felt ethically correct to us. You can say we were wrong about what those red lines were." }, { "i": 163, "speaker": "Daniela Amodei", "text": "You can say, you know, you should have had, you know, stricter red lines or looser red lines. We disagree, you know, on how you did it. There's all kinds of of things you can say." }, { "i": 164, "speaker": "Daniela Amodei", "text": "But I think fundamentally when you look at that case, I found myself asking the question, was this a really principled stand that another technology company took based on their own ethics or was this an opportunistic moment?" }, { "i": 165, "speaker": "Dario Amodei", "text": ">> I actually think it's a beautiful illustration of the democratic process. Different companies can take, you know, different approaches. We took our approach. Another company took their approach. The public in the world rendered a judgment." }, { "i": 166, "speaker": "Dario Amodei", "text": "You know, look back at what people were saying at the time, how it was covered, what individual people said." }, { "i": 167, "speaker": "Walter Isaacson", "text": "Um, you know, I think the world was decisively on our side. Dario, when your studies, your scientific studies as a researcher often involved uh probes and human neurons to what extent when you're doing artificial neural networks." }, { "i": 168, "speaker": "Walter Isaacson", "text": "Do you see that understanding of the brain and uh does that affect how we're going to build AI?" }, { "i": 169, "speaker": "Dario Amodei", "text": "Yeah, I think for me it's affected it in two ways." }, { "i": 170, "speaker": "Dario Amodei", "text": "Um, one is when I was formulating the original scaling laws like just looking at the evolution of animals and the size of animal brains as you know monkeys evolved into humans that you know there was a period in human evolutionary history where the brains of you know primates grew very rapidly and that was about the time that you know humans developed language and were able to use opposable thumbs." }, { "i": 171, "speaker": "Dario Amodei", "text": "And so the idea that like brain size number of neurons is somehow intimately tied to capabilities that scale is important. I feel that that was a neuroscientific inspiration. The second is around mechanistic interpretability which is basically us doing neuroscience for artificial neural" }, { "i": 172, "speaker": "Dario Amodei", "text": "networks. In fact we've seen examples you should talk to our co-founder Chris Olah about this. He knows many more of the details where phenomena that were discovered in artificial neural networks because it was easier to look inside them." }, { "i": 173, "speaker": "Dario Amodei", "text": "Animal brains where you know you have to like you know gruesomely like cut up the brain or you know make a you know animal experiments are very messy and you know gruesome and difficult to replicate. What we found is that you can do so much more when you're looking inside artificial neural networks." }, { "i": 174, "speaker": "Dario Amodei", "text": "And some of the things we've learned in artificial neural networks have helped to guide discoveries in neuroscience." }, { "i": 175, "speaker": "Walter Isaacson", "text": "When you look at Neuralink uh which Elon Musk you know has the chip that can communicate directly and other companies are doing that as well. Is that one of the futures of AI and artificial super intelligence is a total link in alignment with the human brain?" }, { "i": 176, "speaker": "Dario Amodei", "text": "You know, the way I think about AI and the future of human society, I think it's kind of wrong to have like a singular vision. Like I kind of think that like different things are going to happen, right?" }, { "i": 177, "speaker": "Dario Amodei", "text": "Like I you know I think there will be like a set of people who are like really excited about like linking their brains to computers and then there will be like a set of people who are really excited about like living like the Amish you know just the opposite and like you know another set of people who are like really excited about you know exploring space." }, { "i": 178, "speaker": "Dario Amodei", "text": "Um uh and these are completely different visions and the way I think about it, you know, our hope is that that AI can provide more material resources and more material abundance to enable all of these visions to happen at once for everyone to kind of pursue their own picture picture of what's uh good." }, { "i": 179, "speaker": "Dario Amodei", "text": "You know, maybe AI provides some global security benefits and material abundance to make this possible. But I don't think we should link AI to some vision of what it means to live a good life or what it means to be a human." }, { "i": 180, "speaker": "Dario Amodei", "text": "Um, yeah, some people are going to want to link their brains to computers and you know, good luck to them." }, { "i": 181, "speaker": "Daniela Amodei", "text": "I don't, yeah, I don't I don't think anyone Anthropic is in that boat for what it's worth." }, { "i": 182, "speaker": "Walter Isaacson", "text": ">> Uh, Dario, you you wrote the Age of technology uh along with machines of loving grace in a world that's transformed by AI. What does the intuition of both of you tell us about if artificial intelligence can do so much and take over so much of what we do?" }, { "i": 183, "speaker": "Walter Isaacson", "text": "What does that mean for what what's left for humans to do? What's our role going to be? And how do we find meaning?" }, { "i": 184, "speaker": "Dario Amodei", "text": "You know, meaning is a really subtle thing, right? Where I I I I you know, I find that, you know, does meaning only come from being the best in the world at something? I I I I think that's the question being asked here, right?" }, { "i": 185, "speaker": "Dario Amodei", "text": "Because if if AI is, you know, is the best at everything, does meaning only come from being the best in the world at something?" }, { "i": 186, "speaker": "Dario Amodei", "text": "Something I find interesting is that, you know, long after Deep Blue beat Kasparov, um uh uh uh there's been a a renaissance in interest in human chess players, right? Magnus Carlsen is like this phenomenon." }, { "i": 187, "speaker": "Dario Amodei", "text": "I think he's like a chess player and a fashion model, but like he's like he's like a meme. He's become like, you know, it's just a very interesting phenomenon. You, you know, if if AI threatened a meaning, you would think that all of this would" }, { "i": 188, "speaker": "Dario Amodei", "text": "go away right after deep, you know, 30 years ago after Deep Blue beat Kasparov. But but if anything, people if anything, it seems to have stoked even greater interest in in in humans playing chess. So I I I I I think this is some kind of parable for the post-AGI age." }, { "i": 189, "speaker": "Daniela Amodei", "text": "I think for me today work and meaning are sort of conflated as one thing and many people find meaning in their work. I find a lot of meaning in my work. I love my work and I think how we define work and how we define meaning in the future are both going to change." }, { "i": 190, "speaker": "Daniela Amodei", "text": "I think there are a lot of sources of meaning for humans. everything from how you spend time with other humans um whether that's like raising a family or friendships or um you know through connecting with people in a way that looks like work right I think a lot about the future of like medicine as an example right" }, { "i": 191, "speaker": "Daniela Amodei", "text": "you can imagine a world in which diagnostics are done entirely by AI and the role of the human is to empathize with a patient about the experience that they're having. I think that would be incredibly meaningful. And I think if you asked most doctors, they would actually take that trade." }, { "i": 192, "speaker": "Daniela Amodei", "text": "They would say, \"Guess what?\"" }, { "i": 193, "speaker": "Daniela Amodei", "text": "If I'm not spending my life doing paperwork and like trying to diagnose you and getting lab orders sent in and I could actually just understand what's happening with you as a human and help you through a difficult diagnosis, that's why I became a doctor, right?" }, { "i": 194, "speaker": "Daniela Amodei", "text": "I became a doctor because I want to heal and I want to help you and I want you to feel better.\" Is that a less meaningful world or a more meaningful world than than we're in today? I obviously don't think it's going to be that simple." }, { "i": 195, "speaker": "Daniela Amodei", "text": "I think it will be really complicated. But I I also think the aperture of things that people will be able to do will widen. Right? So I think if we do things right, there's the ability to actually expand what our sources of meaning are beyond where they are today." }, { "i": 196, "speaker": "Walter Isaacson", "text": ">> Daniela Amodei, well, thanks. Thanks. Thanks for joining us." }, { "i": 197, "speaker": "Daniela Amodei", "text": ">> Thank you." }, { "i": 198, "speaker": "Walter Isaacson", "text": ">> Thank you so much. This was lovely. >> This is great. >> Totally great." } ]