[ { "speaker": "Speaker 1", "text": "I don't know about you guys, but this is the coolest tech conference I've ever been to, and I I have had nothing to do with it. So, before we get started, I do want to shout out everyone who's working" }, { "speaker": "Speaker 1", "text": "behind the scenes. In particular, Jess, Julia, Jacob, and Brianna for making this event fantastic. So, let's give them a big round of applause." }, { "speaker": "Speaker 1", "text": ">> [applause] >> Now, yesterday we talked a lot about what building agents today looks like and how a lot of the most successful teams are bringing agents into production." }, { "speaker": "Speaker 1", "text": "And so, for today, I want to talk a little bit about the future." }, { "speaker": "Speaker 1", "text": "One of the things that we think a lot about at LangChain, it's actually in our our mission and vision statement, is is what do the agents of the future look like? And so, for for for the rest of" }, { "speaker": "Speaker 1", "text": "this talk, I want to pretend like we're a year in the future. It's Interop 2027, and and what are the topics that we're going to be talking about? What are the topics we're going to be discussing? And" }, { "speaker": "Speaker 1", "text": "I'm sure I'm going to be wrong in a lot of these. This is going to look really silly and really stupid. But hopefully this gives you a glimpse into where we think that the space in the industry is going." }, { "speaker": "Speaker 1", "text": "One thing that I think will start to happen is there will be a divergence in types of in the in the types of agents that people are building. So, I think there will be two types of agents, and" }, { "speaker": "Speaker 1", "text": "we're already starting to see some of this emerge." }, { "speaker": "Speaker 1", "text": "So, one of the types I think will be these long horizon style agents. So, these are agents that run for minutes and hours and maybe even in the future days. They do code execution, they do" }, { "speaker": "Speaker 1", "text": "planning, they use sub-agents, they maybe use multi-agent systems, they use skills, and they operate over longer and longer time horizons. Things like outcomes and goals are starting to" }, { "speaker": "Speaker 1", "text": "become ways to increase the horizons at which these can operate. And so, I think we'll see a big push in these as they do more and more valuable knowledge work." }, { "speaker": "Speaker 1", "text": "At the same time, there's a completely different set of agents that latency is is a huge factor for, and these often end up looking like customer experience agents. So, this could be customer" }, { "speaker": "Speaker 1", "text": "support, this could be sales, this is places when you're talking to an end user, brand is really important here, uh voice becomes a really interesting modality, maybe video will in the future" }, { "speaker": "Speaker 1", "text": "as well. And so, I think these are that these are these two directions that agents are going in. There is a shared stack underneath. There are also differences in terms of the technology." }, { "speaker": "Speaker 1", "text": "And I and I think one of the big questions that that we'll be thinking about over the next year is is how common is this shared stack versus how particular are the are the technology" }, { "speaker": "Speaker 1", "text": "pieces that you need for each." }, { "speaker": "Speaker 1", "text": "Speaking of voice, I think we'll be talking a lot more about voice in a year. This is a place where we're starting to invest more and more. We see this be particularly relevant for these" }, { "speaker": "Speaker 1", "text": "customer experience style agents. And voice is a great modality." }, { "speaker": "Speaker 1", "text": "The typical voice pipeline today looks something like this." }, { "speaker": "Speaker 1", "text": "There's this speech-to-text, text-to-speech kind of like sandwich." }, { "speaker": "Speaker 1", "text": "So, the user speaks, that's transcribed into text, that's passed to an agent, the agent operates in kind of like the text space, outputs some text, and then that text is transformed back into" }, { "speaker": "Speaker 1", "text": "speech, which is which is the audio that you hear from the agent as you're as you're talking to it." }, { "speaker": "Speaker 1", "text": "And so, this is the pipeline that we see today, but there are these native speech-to-speech uh native voice models that are coming out. So, Open AI released V2 of theirs I think 2 weeks" }, { "speaker": "Speaker 1", "text": "ago. And and and while I would I would say the the the consensus today seems to be for for for applications where you really care about having control, they're not quite steerable enough yet." }, { "speaker": "Speaker 1", "text": "We do expect that to change. And so, I think one of the big things that we're looking at in the future for voice agents is which will it be? Will it be this pipeline approach? Will it be this" }, { "speaker": "Speaker 1", "text": "native voice-to-voice approach? Will it be a combination? What are the pros?" }, { "speaker": "Speaker 1", "text": "What are the cons? I think we'll be talking a lot more about voice in the next year or so." }, { "speaker": "Speaker 1", "text": "We think all agents will need a sandbox." }, { "speaker": "Speaker 1", "text": "Uh especially these long horizon style agents. So, coding's really good for a variety of tasks. It's not just writing software. It's for data analysis. It's for web browsing. It's for image gen," }, { "speaker": "Speaker 1", "text": "for deep research. I was talking to someone yesterday who said that when they think about building agents for their marketing team, one of the ways that they think about it is they think" }, { "speaker": "Speaker 1", "text": "about giving them that marketing team a software engineer. What what would that software engineer build? What apps would it build to make the marketing teams uh job easier? That that's what giving an" }, { "speaker": "Speaker 1", "text": "agent the ability to write and execute code is. That's why we think sandboxes are really important and that's why we launched sandboxes yesterday. So, we we I I actually think agents are are are" }, { "speaker": "Speaker 1", "text": "very early on. Sandboxes are very very early on. It's still just the dawn of those. I think we'll be talking a lot about them over the next year." }, { "speaker": "Speaker 1", "text": "Open models are a place that we expect to see significant increase in over over the next year." }, { "speaker": "Speaker 1", "text": "So, the performance of these base open models without any type of post training for particular tasks is already approaching uh that of frontier models." }, { "speaker": "Speaker 1", "text": "So, this is some benchmarking that we did on deep agents comparing uh closed frontier models to open models. And you can see that that they does lag in some places, but it's starting to get really" }, { "speaker": "Speaker 1", "text": "really close to that frontier." }, { "speaker": "Speaker 1", "text": "The other big thing that is happening is cost is starting to become more and more of an issue. So, there are multiple cases where uh in particular for coding agents, these agents are burning through" }, { "speaker": "Speaker 1", "text": "a ton of tokens really really fast. I think open models offer a a cheap alternative to these. And we expect that to be a big driver of open models. A third reason that we think open models" }, { "speaker": "Speaker 1", "text": "are really interesting is because they can be trained for your particular domain." }, { "speaker": "Speaker 1", "text": "And so, I think that's yet another reason as as as companies build up a lot of these uh traces, a lot of these agent runs, how can you use those? How can you use those to improve the model over" }, { "speaker": "Speaker 1", "text": "time? I'll talk a little bit more about that in a second, but but this type of post training is something that you can do with open models. And so we expect a big rise in interest in open models." }, { "speaker": "Speaker 1", "text": "Agent identity is really, really interesting to us. So, as agents start to do more work in the real world, they take actions." }, { "speaker": "Speaker 1", "text": "How do they take those actions? On whose behalf do they take those actions? I think it's still early here. We see two emerging trends. One is when the agents act on behalf of individual users. So," }, { "speaker": "Speaker 1", "text": "if I'm accessing an agent and that agent has access to Slack, and I tell it to look up something in Slack, it will use my credentials and it will have access to whatever I see in Slack. And so, if" }, { "speaker": "Speaker 1", "text": "if if Julia or a colleague uses it, they might get a different answer because it might see different things." }, { "speaker": "Speaker 1", "text": "Another type of off is when they have a fixed set of credentials, usually a service account or something like that." }, { "speaker": "Speaker 1", "text": "And then whoever is interacting with that agent, they will always use that fixed set of credentials, so they'll see the same responses. This, I think, started to get really, really popular" }, { "speaker": "Speaker 1", "text": "with things like Open Claw, where you had this concept of an agent that was its own thing and it had its own fixed set of credentials and you you'd expose it through separate channels and you'd" }, { "speaker": "Speaker 1", "text": "interact with it. And we started to see actually some some SaaS providers make it really easy for agents to create their own accounts, so that they could have their own fixed set of credentials." }, { "speaker": "Speaker 1", "text": "And and so, I think this will be an interesting trend to watch. I actually think we'll see both of these in the future. I think we'll have agents that act on behalf of users, but also agents" }, { "speaker": "Speaker 1", "text": "with their own fixed set of credentials." }, { "speaker": "Speaker 1", "text": "And I think being really, really precise about when to use which one and which one which one is and making that clear to users will be important." }, { "speaker": "Speaker 1", "text": "Continual learning is maybe one of the areas that we're most excited about as a company." }, { "speaker": "Speaker 1", "text": "So, when we think about continual learning, we think about improving the agentic system over time." }, { "speaker": "Speaker 1", "text": "And there's three layers to this agentic system, and all of them can be improved." }, { "speaker": "Speaker 1", "text": "There's the layer, there's the harness layer, and there's the context layer." }, { "speaker": "Speaker 1", "text": "So, the model, I mean things like Sonnet, GLM-5, GPT-5. Harness, this is the code surrounding the model that connects it to the environment. This is Deep Agents, Claude Code Py. And then" }, { "speaker": "Speaker 1", "text": "context might be what we provide to the harness as ways to guide it on particular tasks. So, agent.md, skills." }, { "speaker": "Speaker 1", "text": "You can't edit Claude Code directly, but you can give it skills and give it an agent.md that that tunes it to your particular task. And so, these are three different layers of the agentic system," }, { "speaker": "Speaker 1", "text": "and we think all of them are are involved in continual learning." }, { "speaker": "Speaker 1", "text": "So, maybe starting at the model layer first, this is taken from research done by Ramp and Prime Intellect that came out last week, where they fine-tuned a model to be really good at Ramp sheets." }, { "speaker": "Speaker 1", "text": "And you can see here that the latency is very, very low, and the accuracy is very, very high. And this is an advantage of using an open-source model, Claude 3.5, and fine-tuning it for their" }, { "speaker": "Speaker 1", "text": "particular domain." }, { "speaker": "Speaker 1", "text": "And so, this is an example of continual learning at the model level." }, { "speaker": "Speaker 1", "text": "You can also learn at the harness level." }, { "speaker": "Speaker 1", "text": "So, there's a great paper out of MIT and Stanford called Meta-Harness, where they used an agent to optimize a coding harness." }, { "speaker": "Speaker 1", "text": "They optimized it on Terminal Bench 2, and you can see that it outperforms human-written harnesses." }, { "speaker": "Speaker 1", "text": "And so, this is an example where they used the feedback from the environment, the feedback from running it on Terminal Bench. They took that feedback, they passed it to an agentic system itself" }, { "speaker": "Speaker 1", "text": "that would make edits to the harness." }, { "speaker": "Speaker 1", "text": "So, it wasn't changing the model at all, it was just editing the harness, and and and they got a really good score there." }, { "speaker": "Speaker 1", "text": "My background's in in classical machine learning, and there's a lot of similarities between how you use data there and how you use it in this new world of agent development. And so, in" }, { "speaker": "Speaker 1", "text": "classical machine learning, you you you have the model, you have the training data, you do some gradient descent, and that updates the weights of the model." }, { "speaker": "Speaker 1", "text": "When you're updating the agent in general, depending on the layer you're at, if you're if you're working at the harness or context layer, it's not exactly gradient descent, but the but the evals that you write act as" }, { "speaker": "Speaker 1", "text": "a forcing function. So, in the example of the meta harness that I that I just gave, you have these evaluations, Terminal Bench 2. You run the agent on Terminal Bench, you get some feedback," }, { "speaker": "Speaker 1", "text": "you then pass that into the analytics system, and updates it. So, those evals are providing a similar type of of of this uh training gradient. And so, evals and traces are incredibly important for" }, { "speaker": "Speaker 1", "text": "this learning." }, { "speaker": "Speaker 1", "text": "We did this ourselves, and so this is a this is a few months old at this point, and and so all of these uh you know, the leaderboard no longer looks like this because the space moves really fast, but" }, { "speaker": "Speaker 1", "text": "we moved from from top 30 on Terminal Bench 2 to top 5 just by changing the the harness itself. So, no changes to the model, only changes to the harness, and we saw a big increase in" }, { "speaker": "Speaker 1", "text": "performance. And so, we think that more and more companies will be doing this type of continual learning, whether it's at the model layer, the harness layer, or the context layer for their" }, { "speaker": "Speaker 1", "text": "particular use cases." }, { "speaker": "Speaker 1", "text": "We want to help people do this, and so one of the things that we're announcing today is LangChain Labs, which will be a research group inside of LangChain aimed in particular at continual learning." }, { "speaker": "Speaker 1", "text": "We think that Yeah." }, { "speaker": "Speaker 1", "text": ">> [applause] >> We think that LangSmith, which with all these traces already in there and all the feedback associated with it, this is a really, really solid foundation for doing this type of continual learning," }, { "speaker": "Speaker 1", "text": "whether it's at the model layer, or the harness layer, or the context layer. And so, we're excited to work with a bunch of our customers going forward in this direction." }, { "speaker": "Speaker 1", "text": "The last thing that I'll say is we think that everyone will help build agents in the future. And we already see this today. So, we already see that everyone in an org is involved in improving the" }, { "speaker": "Speaker 1", "text": "agent with feedback." }, { "speaker": "Speaker 1", "text": "So, you've got UX researchers, domain specialists, customer support folks, product people, engineers. They're all giving feedback in some form, whether direct feedback or by tweaking the" }, { "speaker": "Speaker 1", "text": "prompts or by just debugging in Slack." }, { "speaker": "Speaker 1", "text": "They're all giving feedback that improves these agents. And oftentimes, it's the domain experts themselves who have the best type of feedback and know how the agent should perform best." }, { "speaker": "Speaker 1", "text": "And so, we think in the future, they'll be even more involved in creating these agents. We think they're actually going to be building their agents. They're not just going to be giving feedback on them" }, { "speaker": "Speaker 1", "text": "and handing that off to a separate team." }, { "speaker": "Speaker 1", "text": "They're actually going to be involved in building these agents." }, { "speaker": "Speaker 1", "text": "We've started to see a glimpse of this at LangChain, where we have a bunch of agents created by people in all different domains and expertise. And we've seen them build these over the" }, { "speaker": "Speaker 1", "text": "past few months. And so, to talk more about what that future looks like, I want to welcome onto the stage Caroline Embrace." }, { "speaker": "Speaker 2", "text": ">> [applause] >> All right. All right. How's everyone doing?" }, { "speaker": "Speaker 2", "text": "Good. Good. Anyone else get sunburned yesterday or was that just me?" }, { "speaker": "Speaker 2", "text": "All right. Harrison painted a picture of what the future of agents in the workforce will look like. I'm excited to tell you all how we're actually doing that at LangChain today." }, { "speaker": "Speaker 2", "text": "First, I'm going to start with a few use cases." }, { "speaker": "Speaker 2", "text": "Our talent acquisition team sources candidates using our sourcing agent." }, { "speaker": "Speaker 2", "text": "Our sales team automates outbound and account intelligence with our go-to-market agent." }, { "speaker": "Speaker 2", "text": "Our marketing team researches the competitive landscape with our Intel bot." }, { "speaker": "Speaker 2", "text": "And our engineering teams triage and fix incidents automatically with Open Suite." }, { "speaker": "Speaker 2", "text": "Across the company, agents are constantly running, monitoring the business, and providing real-time updates in Slack." }, { "speaker": "Speaker 2", "text": "What do all these agents have in common?" }, { "speaker": "Speaker 2", "text": "Well, they were all built by the LangChain team members who actually use them without having to write a single line of code." }, { "speaker": "Speaker 2", "text": "How do we do that? I'm glad you asked." }, { "speaker": "Speaker 2", "text": "They use LangSmith Fleets, our managed agent builder, which allows anyone to build agents with just natural language." }, { "speaker": "Speaker 2", "text": "They don't have to write any code." }, { "speaker": "Speaker 2", "text": "Over the last 3 years of building agents, we've learned three main things." }, { "speaker": "Speaker 2", "text": "One, the best people to build the agents are the ones who actually do the job." }, { "speaker": "Speaker 2", "text": "Two, agents need to work where you work and have access to the same systems you use." }, { "speaker": "Speaker 2", "text": "And three, as your agent scale, governance can become a bottleneck." }, { "speaker": "Speaker 2", "text": "So, why is it that the best people to build the agents are the ones who actually do the job?" }, { "speaker": "Speaker 2", "text": "Well, at the end of the day, agents are just a collection of instructions, skills, and tools." }, { "speaker": "Speaker 2", "text": "And who best to codify that than the people actually doing the jobs?" }, { "speaker": "Speaker 2", "text": "Also, just like how all of us improve over time, Fleet agents do as well." }, { "speaker": "Speaker 2", "text": "They have memory built directly into them. So, the more you use them, the better they become." }, { "speaker": "Speaker 2", "text": "Now, in order to work in conjunction with these agents, they need to have access to all the systems you use." }, { "speaker": "Speaker 2", "text": "We solve that through tools and channels." }, { "speaker": "Speaker 2", "text": "We have over 200 tools built directly into Fleets that cover the most common integrations at companies require." }, { "speaker": "Speaker 2", "text": "We also have a first-class partnership with Arcade, allowing you to access an extra 7,500 tools out of the box to cover the long tail of your integration requirements." }, { "speaker": "Speaker 2", "text": "And of course, we support everyone's favorite tool framework, MCP, meaning you can connect your custom tools and proprietary data sources directly to Fleets." }, { "speaker": "Speaker 2", "text": "Now, in order to use them easily, Fleet agents are integrated natively into Slack, Gmail, Outlook, and more." }, { "speaker": "Speaker 2", "text": "So, you can utilize the full power of Fleet without having to learn a new workflow." }, { "speaker": "Speaker 2", "text": "We also have a comprehensive user interface, which you can use to build, manage, and run your agents directly within our app." }, { "speaker": "Speaker 2", "text": "And of course, I haven't forgotten about you developers out there. We're opening up the Fleet APIs, so you can use them directly within your own production applications and harness all of the" }, { "speaker": "Speaker 2", "text": "incredible features we've built." }, { "speaker": "Speaker 2", "text": "So, people work in teams. These agents need to be able to work seamlessly alongside you." }, { "speaker": "Speaker 2", "text": "How do we handle that? Sharing, of course." }, { "speaker": "Speaker 2", "text": "You can share and collaborate on your agents and skills directly in Fleet just as easy as sharing a Google Doc with your team." }, { "speaker": "Speaker 2", "text": "When you actually go to use these agents, you need credentials and authentication management. So, they can connect to all your different services, and this can be really tricky. Trust me." }, { "speaker": "Speaker 2", "text": "Luckily, we've solved this for you in Fleet. Meaning, you can connect all your accounts and pick which agents use which accounts depending on which user is using it. Everyone still with me?" }, { "speaker": "Speaker 2", "text": "We've also seen how cost can skyrocket as your agents are adopted internally." }, { "speaker": "Speaker 2", "text": "So, for that, we've built cost tracking and usage controls directly into the app. So, you can inspect exactly how much your agents and users are spending and set spend limits on them." }, { "speaker": "Speaker 2", "text": "Human in the Loop is also a first-class feature in Fleet. So, you can give your agents access to powerful tools without having to worry." }, { "speaker": "Speaker 2", "text": "And of course, Fleet integrates natively with the rest of the LangSmith platform. So, you can look at your agent traces and see exactly how your Fleet agents are operating under the hood." }, { "speaker": "Speaker 2", "text": "Finally, Fleet's open. It's model agnostic, so you can use your favorite open or closed source models." }, { "speaker": "Speaker 2", "text": "It's built on top of everyone's favorite open source agent harness. You guessed it, deep agents." }, { "speaker": "Speaker 2", "text": "And we make it easy for you to download your agent files directly into your code, so you can make whatever modifications you would like." }, { "speaker": "Speaker 2", "text": "Now, I'm going to hand it off to Carolyn, who's going to give us a live demo. Yeah, you heard that right, live demo of Fleet." }, { "speaker": "Speaker 3", "text": "Thanks, Grace. So, we've talked >> [applause and cheering] >> So, we've talked about why we built Fleet and some of its capabilities. And I'm excited to show you what that looks like in practice." }, { "speaker": "Speaker 3", "text": "So, here we have our go-to-market agent." }, { "speaker": "Speaker 3", "text": "This agent can surface account intelligence based on data from your CRM, your call recording platform, and your data warehouse. It can research accounts and contacts, and it can draft" }, { "speaker": "Speaker 3", "text": "personalized outbound emails." }, { "speaker": "Speaker 3", "text": "In order to do that, we've connected it to the tools that we use here at LangChain. So, in our case, that's Salesforce, BigQuery, Slack, Gmail, etc." }, { "speaker": "Speaker 3", "text": "We've written certain sub-agents for common research tasks that we want it to perform." }, { "speaker": "Speaker 3", "text": "And we've given it a long list of skills for step-by-step instructions of common tasks that we want it to get right every time." }, { "speaker": "Speaker 3", "text": "We've also connected the Slack channel." }, { "speaker": "Speaker 3", "text": "So, instead of having to come into the UI here in order to use the agent, you can use it directly in Slack by tagging @GTM agent." }, { "speaker": "Speaker 3", "text": "Okay, enough said. Let's run it." }, { "speaker": "Speaker 3", "text": "Here we go." }, { "speaker": "Speaker 3", "text": "Where did we leave things off with the Pied Piper account? What should our next steps be?" }, { "speaker": "Speaker 3", "text": "As it runs, this go-to-market agent has been a game-changer for our team. The go-to-market team 84% of the go to market team uses it weekly. Lead to qualified conversion is up 240%" }, { "speaker": "Speaker 3", "text": "and it has saved on average 40 hours per month per rep." }, { "speaker": "Speaker 3", "text": "Originally, an engineer had built this agent directly in code." }, { "speaker": "Speaker 3", "text": "But when we built fleet, we rebuilt this agent directly in fleet so that the go to market team could own this agent's implementation um entirely end to end without having to write a single line of" }, { "speaker": "Speaker 3", "text": "code." }, { "speaker": "Speaker 3", "text": "Okay. So, the agent has wrapped up." }, { "speaker": "Speaker 3", "text": "We're seeing an at-risk account. That's not great." }, { "speaker": "Speaker 3", "text": "We're seeing some hesitation around pricing." }, { "speaker": "Speaker 3", "text": "And recommended next steps to re-engage Jared by email. So, let's do that." }, { "speaker": "Speaker 3", "text": "Great. Write an email to Jared, please." }, { "speaker": "Speaker 3", "text": "As Bryce mentioned, um human in the loop is first class in fleet. So, before we just send off an email to a potential future customer, this agent is going to check in with us so that we can double-check the copy," }, { "speaker": "Speaker 3", "text": "make sure everything looks good before we send." }, { "speaker": "Speaker 3", "text": "So, let's take a look here. Let's capitalize LangGraph correctly. Everything looks good. Send." }, { "speaker": "Speaker 3", "text": "Pretty cool, right?" }, { "speaker": "Speaker 3", "text": "Okay." }, { "speaker": "Speaker 3", "text": "Many of the agents Thank you." }, { "speaker": "Speaker 3", "text": ">> [applause] >> Many of the agents that we use every day here at LangChain are available as pre-built agents directly in fleet. So, there's a go to market agent, which you just saw, but also our software engineer" }, { "speaker": "Speaker 3", "text": "agent, which allows you to fully own an agent that you can tag in Slack, have it write code in a sandbox, and then put up a PR for you to merge." }, { "speaker": "Speaker 3", "text": "All you have to do to use it is create the agent, connect it to the tools that your organization uses, and then go through a short onboarding flow so it gets to know your company and then" }, { "speaker": "Speaker 3", "text": "you're good to go." }, { "speaker": "Speaker 3", "text": "And with that, I'm going to hand it back over to Harrison." }, { "speaker": "Speaker 1", "text": ">> [applause and cheering] >> So you guys can try out free today. I I use it myself for a variety of tasks. We actually added a free model in there. So for a limited time, there'll be a free model and and" }, { "speaker": "Speaker 1", "text": "and backing up what we were talking about earlier, it's an open source model powered by Fireworks, one of our one of our fantastic partners here." }, { "speaker": "Speaker 1", "text": "So we try to you know, we try to live what we preach in terms of open source models and and try and also trying to you know, have the best partners here." }, { "speaker": "Speaker 1", "text": "There's a fantastic ecosystem of folks out there at at the booths and so I'd encourage you to go and talk to them." }, { "speaker": "Speaker 1", "text": "So that's going to wrap it up for the keynote today." } ]