Jed Bartausky:

Welcome back to another episode of the React Native Radio Podcast. Brought to you by the snooze button, giving you nine more minutes to negotiate with reality. Episode 370, CopilotKit with Mike Ryan.

Robin Heinze:

So Mazen, I assume you've been watching the World

Mazen Chami:

Cup. Yes. I actually have it on in the background right now. Do you? So if I glance away from the camera, I'm paying attention.

Robin Heinze:

What game is playing right now? Just so we can date this

Mazen Chami:

Recording. Argentina, Egypt in Atlanta, round of 16. And it is 12:49.

Robin Heinze:

So now as you're listening to this, you know how long it takes us to edit

Mazen Chami:

Episodes? Yes.

Robin Heinze:

Yes. Have you been enjoying The Cup? Do you have any fun stories?

Mazen Chami:

I have been enjoying it. Yeah. It's nice because. Well, it's nice and not nice having kids that go to bed right when the big games start. So you're rushing to put them to sleep and of course they don't listen. And then you miss half the game. But then usually the second half of most of these games have been where most of the fun is. I think everyone probably wanted Cape Verde to win and keep going, to keep that story going.

Robin Heinze:

Yeah, they were the crowd favorite underdog.

Mazen Chami:

For sure. Yeah. I've been kind of telling my son, let's always root for the underdog for this World Cup. So it's been fun that way.

Robin Heinze:

Yeah.

Mazen Chami:

I came into the World Cup saying, "All right, it's your goal. It's your World Cup to lose." And they lost it last night, as you all know by now. So there goes that. But given growing up in Africa, I am Morocco and Egypt right now.

Robin Heinze:

Oh yeah, Morocco is doing well. Egypt's doing well.

Mazen Chami:

Especially because Nigeria always - Literally playing

Robin Heinze:

Right now.

Mazen Chami:

Yeah.

Robin Heinze:

As we speak, they are up one zero, just about to go down.

Mazen Chami:

Till halftime. Yeah. Yeah. And I hope that result stands. Sorry, I'm not sorry, all you Messi fans out there because I am not sorry.

Robin Heinze:

Now our guest has a very unique perspective on the Egypt-Argentina game, which is happening right this moment. Mike Ryan is with us and we'll introduce him properly in a second. But Mike, how is your experience of the Egypt-Argentina game?

Mike Ryan:

Yeah, so the office that I work from is in the middle of Pioneer Square in Seattle where we are about two blocks away from Lumen Field where we've actually been hosting a number of the World Cup matches. Nice. And so there's a massive watch party that happens in the center of Pioneer Square. And my good idea was to live close to the office, so I actually happen to live on the same block. So there's been no escape from the World Cup. No escape. None whatsoever. And while I'm rooting for team Egypt, they do have a lovely chant they play on their horns whenever they make a significant play. And so I will be able to tell you with certainty when Egypt scores this morning. No doubt about it. The horns will be blaring. There's no escape.

Robin Heinze:

Oh, vuvuzelas. It takes me back to the 2010 World Cup, which was in South Africa. Mazen, do you have a vuvuzela? You're a soccer fan. I do. So I wouldn't put it back

Mazen Chami:

To you. Yeah, I have a 2010 vuvuzela, but it's put away so my son doesn't get his hands on it. But I got one. So I was up in New York. I didn't get to go to a game, but my brother-in-law was at one, working at one, and he picked up one of these - Is it a duck? It's a duck vuvuzela with the Brazilian flag on it. It is very loud. Oh my

Robin Heinze:

Goodness.

Mazen Chami:

Jed might hate me. I

Robin Heinze:

Wish the audience could see this.

Mazen Chami:

I might tweet it out, but

Robin Heinze:

I'm going

Mazen Chami:

To squeeze it. We'll tweet a

Robin Heinze:

Picture of it.

Mazen Chami:

If you don't hear it, it's because the editors just decided to cut this part out. So sorry. So hopefully that came in. I barely squeezed it too.

Robin Heinze:

We can hear it. We cannot hear it on Zoom because Zoom

Mazen Chami:

Filters

Robin Heinze:

It out, but I'm hoping it made it into your

Mazen Chami:

Actual recording. I did see it flicker. So yeah, it's there. So this is the, I guess, American vuvuzela, you could say. I don't know if it's actually being used in the stadiums or he just found it on Amazon. I don't know the story, but he obviously, for my wife's birthday, picked the most annoying gift to give her because he knew my son would get ahold of it, and then here we are. Sure. I got my hands on it. Yeah.

Robin Heinze:

It's like uncle

Mazen Chami:

Responsibility. Exactly. It's a requirement.

Robin Heinze:

So was Pioneer Square pretty sad last night?

Mike Ryan:

It was very sad. I must admit that I'm currently rooting against teams that might be blowing the horns. So it's probably a bad American of me to say this, but I wasn't too heartbroken about their loss because the square was quiet and sleep was had. I got to sleep nice and hard. Sleep. Whereas I imagine if America had won, it would've been total chaos.

Robin Heinze:

Yes. Yeah. So yeah, for reference, last night was the US's overwhelming loss to Belgium, which Mike got to see first hand in Pioneer Square.

Mike Ryan:

Felt the building shake when the jets flew over as part of the national anthem. Oh wow. It's been a lot of fun. I did get to go see Iran, Egypt actually here. I got tickets for that, and I can confirm that the horns were in the stadium as well. So the horns are everywhere.

Robin Heinze:

So still no break.

Mike Ryan:

Oh yeah. It's been weeks now. I'm definitely at the point where I'm like, okay, let's wrap this thing up. It's time to go ahead and pick a winner out.

Mazen Chami:

Living next to Lumen Field, they've had many games there and I think they're officially done, if I'm not mistaken.

Mike Ryan:

That's right. Yeah. Yesterday's game was the final one.

Robin Heinze:

Yeah,

Mazen Chami:

Because I think the rest are like Miami, New York. LA.

Robin Heinze:

LA. They're still watching it though, it sounds like. Oh yeah. In Pioneer Square. This is not going to stop for my -

Mike Ryan:

It's honestly been one of the more fun cities, I think, because I've lived all over the United States, but Seattle has really embraced the World Cup. There's probably six major areas where there's watch parties and just hundreds, if not thousands of people are showing up at each of these locations and just having a great time.

Mazen Chami:

I mean, prior to the arrival of Inter Miami, the MLS, you had the Portland Timbers and the Seattle Sounders who both had, in my opinion, the best fan base for the MLS.

Robin Heinze:

Yeah. There's a lot of soccer culture in the Northwest. It really is.

Mazen Chami:

So it helps.

Robin Heinze:

Versus football culture. We're very much soccer culture up here.

Mazen Chami:

So helpful for that to happen. Well, okay, I think that's probably enough World Cup talk. We've lost half our audience or probably

Robin Heinze:

More than half. Some people have probably tuned out. I promise we are going to talk about CopilotKit and React Native.

Mazen Chami:

Well, okay. As we always do, I'm Mazen, staff software engineer here at Infinite Red. I'm joined by my lovely co-host, Robin, director of engineering here at Infinite Red.

Robin Heinze:

Hello.

Mazen Chami:

And joined by Mike Ryan. Mike, can you introduce yourself to our listeners please?

Mike Ryan:

Yeah, sure. My name is Mike Ryan. I'm a principal architect here at CopilotKit, working on both our SDKs and AG-UI, our protocol. Some of you may know me from the Angular community. I've written a lot of high or big open source projects in that space, including NgRx. And over the past few years, I've been working more and more in the React land, and I'm excited to be here talking about React Native.

Mazen Chami:

That's awesome. That's awesome. Awesome. Well, welcome onto the show. And before we get into our topic, let's hear from our sponsors.

Jed Bartausky:

Infinite Red. Infinite Red is a premier mobile app consultancy focused on Expo and React Native. We're a team of 30 that's located fully remote in the US with highly experienced mobile app developers. And we've been doing this for over a decade. We're also one of the first development teams to adopt agentic coding in a way that keeps high quality standards while also being able to dive in and do things the old school way if we need to. If you are looking for mobile app, Expo, or React Native expertise for your next project, hit us up at infinite.red/contact. Thanks. And now back to the episode.

Mazen Chami:

Okay. Let's get into our topic for today. As we mentioned, we're going to be talking about CopilotKit with Mike. Before we begin, I guess let's just set the scene. What is CopilotKit? What's your elevator pitch for CopilotKit?

Mike Ryan:

Sure. Our elevator pitch is that all UI will be AI-driven as more and more applications integrate AI systems into their user interfaces. And CopilotKit serves as a set of SDKs for developers to build agentic front ends where you build applications that connect that agent to your user base, and those agents can respond with more than just text. They can respond with really visual responses, including generative UI, the ability for those agents to display user interfaces to your users as part of their responses.

Robin Heinze:

No relation to GitHub Copilot?

Mike Ryan:

No relationship to GitHub Copilot whatsoever. Okay. I'm sure

Robin Heinze:

You get that question a

Mike Ryan:

Lot

Robin Heinze:

Given the - All the time.

Mike Ryan:

Yeah, whenever we're at conferences or something like that, it's like you must be related to Microsoft in some way or another. And it's like, no, no, no, no. We're our own thing. Bit of a play though on GitHub Copilot and the Apple inclination to call things kits.

Robin Heinze:

Things kit. Yeah. Yeah. I love it. So is front-end stack for agents, is it this new category of thing that didn't exist before and now needs to?

Mike Ryan:

I would say so. I mean, it's not going to look too dissimilar from some of the UI patterns we saw beforehand. I think what trips up front-end developers about doing agent-to-user interfaces is that for a lot of front-end-oriented folks, it's their first time encountering things like streaming or persistence of that streaming state, reconnection, and then also setting up this pipeline that allows these agents to use the set of components that you're developing in its responses. And so there's a lot of unique challenges around both of those two concepts that I think

Jed Bartausky:

Really

Mike Ryan:

Help define the need for this category to exist and why we'd want a lot of common code in open source libraries to be authored that we can build on top of as we're building these agentic applications.

Robin Heinze:

So what would it look like without a kit like this, if you were just to bolt a chatbot into an existing regular app? What sort of problems would you see that something like CopilotKit is going to come in and solve?

Mike Ryan:

Yeah, so the first challenge you're going to have is how do you actually connect that agent protocol layer to your front end? I think you might intuitively reach for an HTTP call. I'm going to make an HTTP call with my user input. I'm going to send it to this REST endpoint. In the handler of that REST endpoint, it's going to call my agent, produce a response, and I'm just going to throw that back to the user. The problem that'll arise for you though is that when agents are taking their turn, they go through a number of steps to arrive at their final output. They think, they reason, they call tools. They might respond multiple times as they're going through that showing text back to the user. And so the first part that's done the lock is like, oh, how do I actually stream this? How do I show every single step of this agent's turn in my application and give all of that information to my front end and then handle the state management complexities that come in with handling those streaming responses?

So I'd say it's big problem number one, that CopilotKit, and actually specifically we have this open protocol called AG-UI, that protocol really tries to solve first and foremost.

Mazen Chami:

What is AG-UI, but what does AG stand for?

Mike Ryan:

Yeah, it's the Agent-User Interaction Protocol. So it is a completely open source, openly governed protocol for streaming agent responses to any user interface. And this protocol really exists because regardless of your agent framework or regardless of your front end, this streaming problem appears or manifests. So whether you are building with LangChain and React to the web or Microsoft Agent Framework to Microsoft Teams or Google ADK to your React Native application, you always have this streaming problem. And AG-UI as a protocol serves to standardize and formalize how to do that streaming between these different agent frameworks to these different front ends.

Mazen Chami:

Got it. Okay. So let's just talk about the building blocks. We talked about what breaks with the current chatbot that Robin mentioned. Walk us through CopilotKit and AG-UI, how they all kind of come together. What are the core building blocks from your agentic connection to the front end? Any shared tools within there? How does that look end to end?

Mike Ryan:

Yeah, sure. So I'll start with just AG-UI and then I'll tell you how CopilotKit sits on top of that. So let's go back to that example where you're trying to build a chatbot and you decided, okay, I don't want to go roll my own. I don't want to build a REST API myself to go do this. I want to stream these responses back and forth. So AG-UI as a protocol also comes with SDKs for various agent frameworks and various front ends. And you would then pick the relevant AG-UI SDKs for your particular makeup. So I'm going to just keep using LangChain as an example. I think a lot of folks are familiar with LangChain as an agent framework and it's quite popular right now. So let's say I've got my LangChain agent, I'm going to pick the AG-UI adapter for LangChain. And what that will allow me to do is out of the box, stream all of those agent steps from my LangChain agent to my front end over SSE.

So it's still an HTTP request, but it's going to be able to stream those steps via SSE to my front end. I'll then pick an AG-UI client SDK, and that will let me consume that using RxJS observables. So I get this nice reactive interface that allows me to consume all those agentic turns or steps. And so end-to-end now from agent to my front end, AG-UI just by itself is going to handle a lot of that streaming for me. So then where CopilotKit comes in is it sits on top of this request structure. If AG-UI is how to get agents to communicate with clients and back and forth, CopilotKit is, let's make this a really rich experience. So CopilotKit gives you framework specific bindings. Whereas AG-UI gives you RxJS observables, CopilotKit's SDKs will give you React Native specific SDK with React hooks that work in React Native and let you consume those responses and consume that state in a way that'd be really familiar for a React developer.

And then similarly, it lets you expose your React components over this protocol so that the agent can choose to render those components as part of its responses.

Robin Heinze:

I see. Okay. It's sort of like a layer between the developer and the internals of the protocol and stuff, sounds

Mike Ryan:

Like. Exactly. Yeah. And it's all developer focused. You as a developer, you're using these SDKs to power your application. It's funny, I'm AI related, but there's no AI actually in the loop in terms of CopilotKit. We're just handling this communication step or piece between these two parts of your stack.

Robin Heinze:

If people are familiar with other protocol, like MCP is a protocol that you hear about a lot in relation to AI and agents. If they're familiar with that, how does that relate to AG? Is it sort of an alternative? Are they doing different things? Where do they fit together?

Mike Ryan:

Yeah, that's a great question. So as I mentioned, when an agent takes its turn, it can think, it can call tools, and it can respond.

MCP servers out of the box, their primary capability is to allow an arbitrary system to expose tools to agents that they can call. So I use coding agents daily. And as an example, I do a lot of design work inside of Figma. Figma exposes an MCP server that lets my coding agent actually ask Figma for information about the designs that I've created as it's implementing its workflow. But what I would highlight there is that it wasn't like for MCP, it wasn't about how do I get the agents to the user? It's how do I give new tools to the agent that it can call while it's doing its workflow? So AG-UI is very complementary to MCP. And in fact, if you were to go use AG-UI today in CopilotKit, we have MCP support out of the box. And so you can go connect via CopilotKit to any MCP server and do that.

Robin Heinze:

So not competing protocols, complementary protocol.

Mike Ryan:

Very complementary.

Robin Heinze:

And

Mike Ryan:

Then on top of that, there's this additive thing to MCP called MCP Apps. And this one is probably the most analogous perhaps to CopilotKit, which is where an MCP server, its tools, instead of just providing context to the agent, can actually render iframes and embedded applications inside of a chat context. And for MCP Apps, that extra layer on top of MCP CopilotKit also supports that as well. So if you wanted to build a React Native application that consumed an MCP server that supported MCP Apps, CopilotKit would make it really easy to connect over that protocol and stream that to React Native application.

Robin Heinze:

That's super cool. Go ahead, Mazen.

Mazen Chami:

Do I need to use both of them together or can I use one over the other? Because I do know looking at your docs, CopilotKit has some UI to it. So the chat input, for example, was one of them. And I also saw there was a bar chart rendering, which I assume is consumed by AI. Can I use CopilotKit as my UI and then not use AG-UI or even vice versa? Are people adopting AG-UI and not CopilotKit?

Mike Ryan:

Yeah. So I'll answer that in reverse, which is first off, we created AG-UI at CopilotKit to go solve this problem, and it has completely outgrown CopilotKit in terms of the number of people that are using it and experiencing it because that -

Robin Heinze:

So many things end up working like that

Mike Ryan:

In software.

Robin Heinze:

Yeah.

Mike Ryan:

Which we're happy to do. We saw that this needed to be solved and it really deserved an open source treatment and that a lot of different ecosystems would benefit from that solution. So I would say you can absolutely use AG-UI without CopilotKit. There's plenty of folks that do take that approach. And the community around AG-UI is really big. I mean, there's client implementations now for a variety of languages and a variety of agent frameworks that we don't even control or manage. It's all community contributed. Now CopilotKit, you cannot use CopilotKit without AG-UI. CopilotKit is really built on top of this protocol. And I'd say that CopilotKit has two layers to it. It has a headless set of APIs for consuming or making it easy to consume AG-UI and enrich that communication. For React Native developers, that's actually all we have today is a set of headless APIs.

But then if you're building for React on the web, we also do have built-in chat components, chat input, thread drawer, a lot of those basic elements you would want to create just your own ChatGPT or Claude on the web. And you can of course use those if you want to, or you can keep it headless, but you can't use those components independently of CopilotKit. They are really meant for CopilotKit specifically.

Robin Heinze:

Is the lack of pre-built chat UI on React Native specifically just haven't gotten to it yet? Or is there some technical limitation that you're working around? I

Mike Ryan:

Would say mostly haven't gotten to it yet. We're an engineering team of 12 people. And so while we are all certainly very - I totally

Robin Heinze:

Understand.

Mike Ryan:

It's just the amount of surface that this company tackles, it's overwhelming at times.

Robin Heinze:

We understand that

Mike Ryan:

Well

Robin Heinze:

As a small team trying to do -

Mike Ryan:

Trying to do a lot.

Robin Heinze:

Consulting and a podcast and other things. So totally understand that. We usually ask this question at the end, but is it set up in a way that there's open source opportunities for anybody? If there's people in the React Native community that want to build help, is that a possibility?

Mike Ryan:

Yeah, so everything that we've talked about at this point is open source with an MIT license on it. So if you're like, "Man, I'd love to go contribute some built-in components for React Native and help flesh out that SDK," I think it would really benefit from some community love. It's one of our newer SDKs. I think we just released our React Native support maybe a month ago, and it was driven by some of our own users that really wanted to bring the same AG-UI agents that they're using on their web application into their React Native application. We said, "Of course, we'd love to go support that. Let's go do the work that it would take to make that work in that environment." So the more interest that we see in that SDK, the more we're going to want to go level on it ourselves too.

Robin Heinze:

Have there been any rough edges in the React Native side? I mean, React Native has some things that can go deeply complex really quickly, like Hermes and polyfills and the new architecture and all this stuff. Were there some challenges when it came to the React Native SDK or that people implementing it in React Native can expect to face when they try to use CopilotKit?

Mike Ryan:

I mean, certainly. And for me, it was a good learning experience. Like I said, I'm classically an Angular developer, new-ish to React over the last five years. And this was my first real project getting into the internals of React Native and making this work. And so thankfully, by not having built-in components, I think we got to avoid a lot of the common pitfalls that a React developer might have when building for React Native for the first time. But we did hit a few snags. I mean, I think one that comes to mind is that we mint a lot of UUIDs and the random code that we would use underneath the hood, some of the math libraries we use underneath the hood for that didn't work out of the box in React Native and needed some custom support. I think there's also some complexity around handling the SSE streaming that I mentioned on React Native that needed some thought.

Obviously, Fetch is quite different in React Native versus on the web. But the good news is that if you are a React Native developer using CopilotKit, we've solved all of that. You don't have to go think about those things in particular.

That's what's great about these libraries is that they solve those common challenges and you can just enjoy consuming an AG-UI stream without too much complexity on your part. I

Jed Bartausky:

Love

Mazen Chami:

It. I assume y'all are compatible with Expo also.

Mike Ryan:

Yes, absolutely. So I know that Expo's basically become the default way for a lot of new React Native projects.

Robin Heinze:

It really has. It really has.

Mazen Chami:

I think people not using Expo are older applications that have probably adopted Expo, but adopted it in one package here or there that just for a new feature that works for them. And I would be surprised if it's not like tech debt to eventually evolve to full Expo stack.

Mike Ryan:

Yeah, for us, we're greenfield in this way, and so it just was the natural, yeah, this is

Mazen Chami:

How we're going to

Mike Ryan:

Go about it for the way we're going to build.

Mazen Chami:

Yeah. I want to come back to, I think this is a question that I always have, and I think a lot of people now with AI taking over a lot of things. When we're talking about generative UI, who should render the UI? Who should control the UI?

Robin Heinze:

What does generative UI actually

Mazen Chami:

Mean? Yes. Just explain these in terms of a chat.

Mike Ryan:

Yeah. So generative UI as a concept is the idea that the agent can render UI as part of its agent's turn to the user. And I must admit, I think it's quite skeptical for a while that chat would just be the way that we'd be talking to AI. But the further we've gotten into this, more I'm like, actually, you know what? If I think about what computers are meant to do and what a user interface is meant to do, it's meant to bridge the gap between the systems we're building on top of and the way that humans communicate, which is we talk, we type, we text. And so chat has become this dominant way for interacting with agents because they can talk back. They have the ability to understand natural language and they can synthesize new language back at you. And so what we want to do with generative UI is we basically want to say, okay, well, it can't just all be text characters.

At some point I'm going to want to see information. I'm going to want to make choices. Let me as a developer enable this agent to show UI back as part of its response.

Robin Heinze:

So it's basically like an app where you, the developer, previously controlled everything that was ever rendered. Now you're letting an agent basically write code that's immediately displayed in your app and shown to users.

Mike Ryan:

That's right. And we really break down generative UI as being a spectrum from being fully controlled to fully open-ended. So an example of controlled generative UI in CopilotKit, we have this hook called useComponent. And it allows you to expose a React component that you fully authored, you control the styling for, you control the logic, the implementation. You give the LLM some schema on how to bind the props, and you don't even have to do that if you don't want it to let it bind to the props. And that component becomes exposed to the LLM, and then the agent can choose to show that component on demand. And we call this fully controlled generative UI, because while the choice to show the component lives with the agent in terms of what that component looks like, how it behaves, you as the developer have complete control over that experience.

They're picking

Robin Heinze:

From a selection

Mike Ryan:

That

Robin Heinze:

You have curated.

Mike Ryan:

Exactly.

Robin Heinze:

And you're like, "I build the components. You choose which ones you would want to render and what information to give them, but I control the components of that. Okay.

Mazen Chami:

I love that idea of giving it the guardrails. Almost like bowling with the sides up kind of thing like," Hey, this is our UI. This is how we can do here access to props and all that. Now we've defined our UI. How do we sync our state? How does the communication happen back and forth now to that UI to adapt and change based on the agent's communication?

Mike Ryan:

Yeah. So AG-UI as a protocol has this concept of shared state. So if you build with an agent framework, as you define an agent, it has this concept of agent state and it can read from that state and make changes to that state object internally. And what AG-UI lets you do is it actually lets you stream that state object to your front end and then consume it almost the same way you would with any React state. We get essentially a useState hook that lets you read from that state, but also write to it. And so let's say that you wanted to let your LLM or your agent render a menu. You're doing a online ordering for a restaurant. And so your React component is food menu and your menu has a couple of different food options on it. As the user interact with it, you can actually use these hooks to write to that agent state.

And then the agent can read what those selections were and complete its turn based on that information. So controlled generative UI, the far left side of the spectrum. It gives you this ability to let the agent show your components and then you can have those components directly read to and write from that agent state as part of its flow.

Robin Heinze:

So then there's the other side of the spectrum.

Mike Ryan:

So the other side of the spectrum then would be fully open-ended generative UI. And I would say this one, I don't see a lot in production yet, though actually I do see in the ChatGPT app or the Claude app.

Robin Heinze:

It's the YOLO mode.

Mike Ryan:

It is. It's YOLO mode. It's here's an iframe. You the agent, you can generate whatever HTML, CSS and JavaScript you want to within this iframe.

Robin Heinze:

Yikes.

Mike Ryan:

It's fully sandboxed. Have fun. Whatever the users ask for, you go for it. It's one of those things that demos really well, I think, especially for reporting. I actually see a lot of my teammates, not just on engineering, but outside of engineering here at CopilotKit, lean on Claude Code artifacts, which is this ability for Claude Code to generate some HTML and publish it on the web in one of these iframes. And this style of fully open-ended generative UI gives your users a lot of freedom and flexibility to generate whatever they want to, but it's still not super clear to me what the actual maybe business applications of this will be or how much people will embrace it. So then -

Robin Heinze:

It feels like a party trick.

Mike Ryan:

It's a bit of a party trick.

Robin Heinze:

But not maybe worth a liability for enterprise situations.

Mike Ryan:

Yeah. If you're an enterprise and you care a lot about accessibility and performance, this is not going to be the style of generative UI for you. But in the middle of this is what we'd call declarative generative UI. This is my personal favorite approach. So it's kind of like that useComponent piece where I expose a React component, except in this situation, I'm actually going to expose a catalog of React components to the LLM. And what the LLM can do is it can assemble a React tree of those components on the fly. So instead of just picking one and binding props to one, it can make lists of components. It can make a full tree out of those components. Now I still control the catalog. I still have guardrails in place of what all it can pick, but it has a lot more freedom in its ability to assemble a response.

And this is really powerful for.

Let's go back to the restaurant menu ordering example. If I wanted to ask about a couple of different menu options, like I wanted to ask this bot, "Hey, I'm vegetarian. Which menu options do you have that are vegetarian friendly?" It could assemble a response of all the vegetarian-friendly menu items and show them to me in a really nice UI and guide me through them. We actually have collaborated on this with our friends at Google. On a protocol that is similarly named. All these protocols have the most. It's called A2UI, and it is the agent to user interface.

Robin Heinze:

I was just looking at your docs and the fact that you have to have a whole page that's like, okay, their names are similar, but they're not the same.

Mike Ryan:

They're not the same. Yeah. AG-UI carries A2UI. We

Robin Heinze:

Can use both. They work well together. It's not one or the other.

Mike Ryan:

Yeah. Yeah. And again, different use cases. One's handling the streaming, the connectivity. This one is just about, I want to let my agent assemble a UI from a catalog of components. And I see that one having a lot more adoption as developers continue to build out these agentic chat applications.

Robin Heinze:

Yeah. A happy medium is almost always where people land.

Jed Bartausky:

For sure.

Robin Heinze:

You've talked a bit about use cases like restaurant ordering and stuff. What are other sort of mobile, specifically mobile use cases where this CopilotKit is really going to shine in the idea of generative UI and your protocol?

Mike Ryan:

Yeah, I think the mobile use cases are likely to be most dominant. I think it's where we actually want to reach for an assistant that we can quickly talk to or ask a quick question of is when we're on the go. And so some of the use cases that come to mind just from the folks or the developers I get to engage with, there's a medical company that allows you to quickly get questions and answers about some medical reports about you.

Jed Bartausky:

And

Mike Ryan:

So it's handy just to query, "Hey, what was in this report? What did it look like?" And then they're using generative UI to show well-cited charts and visualizations. And that's really important here because if you set the LLM free form of text response, it could make up a bunch of nonsense. And that's not - Which you don't want. You don't want our medical setting. In a medical center. But if you're leaning on generative UI and it's picking authoritative components that you have control over, then when it shows a chart of medical information to a user of the mobile application, you know that that data is well-cited, well-sourced, and you're protecting almost your user from some of those LLM hallucinations. And so that's been one of my favorite use cases where there's on the go, users want to ask about this results that they've received because you were never at your computer when you receive a result.

Let's give them the ability to ask questions, but still give them good authoritative answers as part of that flow.

Mazen Chami:

I love that. I think we're in the age of, okay, we're in the age of AI now, or someone might say we're getting there. We're getting there, let's just say. But we're also in the stage where your app has a lot of information. We've built apps with a lot of information, a lot of text, images, and stuff like that. And this to me sounds like the perfect use case of showing you targeted information, getting your users to be a little more better informed by your app. So going back to the mobile ordering app that you mentioned,

If you've told the LLM or the agent that you are vegan, next time we come into the app, I as a vegan would anticipate not to see your triple stack beef cheeseburger on there again. So now you're winning trust from that user back. It's like, okay, this app knows me, understands me. So my lists are always filtered down to my specialty or my specifics. And maybe there's a button here like, our menu has changed. Do you want to see our full menu? And you get to reset that context for you. We have a question here and I kind of want to ask it still, but I feel like the answer should be like no. But any situations that people should steer away from using something like this? Because I know medical has the whole asterisk of like, okay, we don't want to be recommending something wrong.

We could get sued and this and that. But at the same time, you -

Robin Heinze:

It sounds like it's actually a pretty

Mazen Chami:

Good fit for something like that. I would use it. I'd love to go into MyChart now and be like, because it's a pain to pull up MyChart and look at my test results over the past years. When I could just say, "Hey, I want to look at my, I don't know, cholesterol for all my cholesterol results. Can you plot it on a chart for me?" And I got one pretty chart with all that and a distillation of all the doctor's notes there. That's exactly what I want instead of having to click through every single thing and remember like, oh, two years ago I was at this, now I'm fluctuating. Stuff like that. So yeah, my question is, is there a use case where you say people should steer away from it?

Mike Ryan:

Absolutely. I mean, every company that I get to work with at CopilotKit will tell me the same thing, which is that in terms of accuracy, actually getting your agent to be accurate for your users, you can maybe hit 90% success rate with the state of the technology today. And I think it's still wildly useful. I think we wouldn't all be using these apps all the time like ChatGPT and Claude if that wasn't useful to us. But that 10% of not so accurate or not quite right means that you want to be very careful about putting agents or using this technology in places where you have to make a decision. So it's one thing for the agent to pull up parts of your chart in MyChart. It's a whole nother thing if the agent was going to start prescribing medicine for you based on those results.

It's the difference being here where it's starting to choose to take real world action on your behalf, where I think my alarm bells start to go off in this.

Robin Heinze:

No, too far, too far.

Mike Ryan:

But on the other hand, it is possible and you can make this good over time. And one of the ways that we facilitate this is with human in the loop. So when agents take their turn, like I said, it can call tools. Some tools that actually take real world action, you might want to get your human to approve that tool call before it actually takes place. And so CopilotKit and AG-UI also enable this human in the loop workflow where you can put that guardrail around certain tool calls and actually show a really nice form or confirmation dialog inside of your application saying, "Hey, the agent wants to go do this. Are you sure you want to let the agent to do it before the agent goes off and does it?"

Robin Heinze:

Yeah.

Mike Ryan:

And so we're trying to be mindful of that too, that people are going to go start exploring with putting agents in situations where they get to make these decisions. And we want to make it easy for developers to keep humans in the loop when agents go about that

Mazen Chami:

Flow. Awesome. Well, I think we're kind of getting towards the end of our episode, but what's next for CopilotKit? Any upcoming releases for us to keep our eyes on? What's coming soon?

Mike Ryan:

Yeah. So I liked the way that you extended that analogy about the restaurant menu ordering bot and how the bot can remember what it is that my dietary preferences were. And we basically see this emerging all over the place for all sorts of companies. This ability to personalize themselves based on their past interactions with you. And so at CopilotKit, for us, we used to think of ourselves as just being SDKs for the web. And now we're seeing more and more companies want to bring those agents, that personalization to wherever their user is. So obviously React Native was a natural place to go next, but in upcoming releases, we're going to let you bring those same AG-UI agents to Slack or Teams or Discord or WhatsApp or Telegram. And I think that's really handy because whether I'm in the car and I've got an app using Android Auto, or I'm on my phone or I'm back at my computer or I'm texting from my device, that same agent can be there and remember who I am and sort of pick up and show me generative UI wherever I am.

I'm really excited about those sets of releases that are going to help developers bring those out. And then also we're going to make it a lot easier for developers to build agents that can learn from your interactions. So we're going to really empower developers with a great learning SDK to help make memory and learning a core part of the user experience.

Mazen Chami:

That's pretty cool. That's awesome.

Robin Heinze:

Yeah. I'm super excited for that.

Mazen Chami:

Where can our listeners follow CopilotKit's progress, listen for updates? Where are you all posting all this information?

Mike Ryan:

Yeah, we're most active on X and LinkedIn, so you can check us out at CopilotKit on both of those platforms. And then I am personally active and write and tweet a lot about it. You can follow me on Bluesky, X and LinkedIn @MikeRyanDev. I'm also highly available on GitHub. If you ever have any questions, I'm always happy to help folks out as they're building generative user interfaces.

Mazen Chami:

That's awesome. Yeah. Well, Mike, we really appreciate you coming onto the episode and sharing all this knowledge about CopilotKit. It's exciting. We're living in an exciting time and you all are building something very cool. And I think I can see myself using some of these products to put together a delightful experience for users.

Mike Ryan:

Yeah, thanks for having me on. And as mentioned, if you want to get involved in open source, this is all open source. We'd love to have some outside contribution to it.

Robin Heinze:

Love that.

Mazen Chami:

Well, as we always do, Robin, do you have a mom joke to send us off?

Robin Heinze:

I do. I have a good one. It's really topical for the World Cup. Why was Cinderella so bad at soccer? Because she kept running away from the ball.

Mazen Chami:

Nice. Yes. Timely one. Love it. Awesome. Well, thank you all, and we'll see you all next time.

Jed Bartausky:

Bye.

Robin Heinze:

All right. Bye.

Jed Bartausky:

As always, thanks to our editors, Tyler Williams and Jed Bartausky, our marketing and episode release coordinator, Justin Huskey, and our guest coordinator, Mazen Chami. Our producers and hosts are Jamon Holmgren, Robin Heinze, and Mazen Chami. Thanks to our sponsor, Infinite Red. Check us out at infinite.red/radio. A special thanks to all of you listening today. Make sure to subscribe to React Native Radio on all the major podcasting platforms.