[Chad Davis] 13:01:58 Hey, everybody. Thanks for joining us for our May 2026 webinar. I'm here with Amber Samdahl. [Chad Davis] 13:02:09 Partner in crime here at Public Media Innovators, and we are very excited that this one came together thanks to the Connecticut folks for being willing to kind of do this. When Amber and I started the month of May, we were kind of still [Chad Davis] 13:02:23 figure out what we wanted to do. And then we were at the PBS Technology Summit, and we were in this particular session that you are going to see a reprise of today with the folks from Connecticut. And I leaned over and I was like, this should be a webinar [Chad Davis] 13:02:38 And she agreed. And then I went up to Lauren, and Lauren agreed, and got everybody together. And so it all just came together, and I really appreciate the folks from Connecticut doing this. This really was one of [Chad Davis] 13:02:50 best sessions from the annual meeting, so if you weren't there, and you had a little FOMO, this should assuage some of that. [Chad Davis] 13:02:59 Before we dive in, special thanks to NETA, who always kind of helps support us by helping to facilitate the public media innovators group that we run. And I'll do a quick plug for June. We'll mention it again at the end. Registration will be ready by next week, but we also pulled another session from the annual meeting for June [Chad Davis] 13:03:25 A couple folks from the NewsHour are going to be with us to talk about how they are making use of Reddit. So more on that in the next couple of weeks, but that'll be on June 18th. So I think that covers it for housekeeping. Amber, did I miss anything? [Chad Davis] 13:03:39 Nope, Amber, thumbs up. All right. Well, with that, then I will introduce Lauren Komrosky, and I will just let her introduce her team, and they're going to take it all the way through the Q&A. So please, everybody, just start playing in the chat. Lauren, take it away. [Lauren Komrosky] 13:03:55 Thanks, Chad. Thanks, Amber. We're really excited to be here today. This is one of our favorite subjects these days. So glad to hear that everyone's interested in learning more. So Susan's going to pop up our team here, our names. Basically, you're looking at a cross-functional group across Connecticut public [Lauren Komrosky] 13:04:13 And we like to call ourselves the AI Tiger Team, which basically means kind of like I said, multiple people from multiple departments kind of coming together, focusing on one strategy, goal, or project. So you're going to hear from not me that much, which is [Lauren Komrosky] 13:04:28 preferred on my end. But you're going to hear from Susan Bell, who leads our digital team or Digital Services Bureau, we call it, or the DSB. You're going to hear from Jim Haddadin, who leads our investigative reporting unit, the tap team. You're going to hear from Brendan Foley, who's our Senior Director of Membership and leads the membership team [Lauren Komrosky] 13:04:46 And then you're also going to hear from Rob Gabaree, who is our developer extraordinaire that's on the digital team as well. So I will stop talking because the good stuff is about to come. [Susan Bell] 13:04:58 Good afternoon, everybody. I am Susan. Lauren, thank you for the introduction. We're going to start today with just a couple of Q&A and feel free to give us a thumbs up or a thumbs down as I kind of ask these questions in the chat. [Susan Bell] 13:05:14 who here ever fills your team has enough time? [Susan Bell] 13:05:18 Everybody's got plenty of time [Susan Bell] 13:05:21 Who here has answered the same questions? [Susan Bell] 13:05:25 Over and over and over again, like a lot of times [Susan Bell] 13:05:31 Who here wishes they had more reporters? [Susan Bell] 13:05:37 I know we do [Susan Bell] 13:05:39 We're all really dealing with the same challenges across our environments. And what we want to show you today is how we've started using AI for practical ways not to replace the work that we're doing, but to support it. So today, we were going to concentrate on four different areas. One is strengthening local journalism [Susan Bell] 13:05:59 Reducing repetitive work, better serving our audience, and ultimately growing our impact. So I'm going to throw this over to Jim, who's going to run us through how Connecticut Public is using AI to strengthen local journalism [Jim Haddadin] 13:06:17 Hey, good afternoon, everyone. So as Susan mentioned, one of our pursuits is figuring out how we can incorporate AI into the news gathering process and do that in a deliberate way where we feel comfortable with how it works and our audience would kind of support our use of AI in this regard [Jim Haddadin] 13:06:36 So, I'm going to talk about a problem that we know all too well, that local news is disappearing in many of the markets where we operate. But I'm hoping to leave you with a sense that there's also a real opportunity here for us, particularly as public media stations, to step in and fill that void [Jim Haddadin] 13:06:53 And the hope that AI might help us do that. If you look at this map here, this is a map of news deserts that was put together by MUCRAC and Rebuild local news. They estimate that at this moment, we've lost something like [Jim Haddadin] 13:07:07 you know, three-quarters of the local news jobs that we had even 25 years ago on this map, the areas where it's the darkest orange are really red. They calculated that something like 1 in 3 counties in the Us. Right now does not have the equivalent of a single full-time reporter covering what happens there [Jim Haddadin] 13:07:28 In Connecticut, you know, we're a small state, but even on the scale that we operate at, this is a big challenge for us. We have 169 different cities and towns, and our mission is to provide coverage for that full scope of communities. And they're very different in character. On one side of the state, you might have folks who like the Red Sox on another side, you might have folks that like the Yankees. So it requires a lot of kind of special expertise of what's going on in these different communities [Jim Haddadin] 13:07:53 And they all have their own common councils. They all have their own kind of rules and regulations. So it's a lot for us to get our hands around. The way that news organizations used to do this is by sending folks out to cover government meetings to get a handle on what's going on [Jim Haddadin] 13:08:07 But as many of you know, it can be really time consuming and inefficient. If you have a big state actually physically sending people out to cover stuff is tough. You can look at records like meeting minutes after the fact, but they really don't tell the whole story of what happened at a meeting. Oftentimes, governments will kind of have an incentive to leave out some of the juicy bits or some of the tension of what arose. So it's hard to kind of replace that experience of actually hearing the live conversation of what takes place. [Jim Haddadin] 13:08:36 So why am I so passionate about this? I've been a journalist for a long time. This is my press pass from where I worked at a newspaper in New Hampshire, like, 15 years ago. Some of you have heard this already, but my joke is I look even older there than I do today [Jim Haddadin] 13:08:51 And I really enjoyed covering these meetings and got to love them because it really reinforced to me that it's not just merely about the kind of news that comes out of it or something like a budget that gets passed, but you really get to know like the soul of the community, people's values [Jim Haddadin] 13:09:06 what they want the place to transform into in the future. That's another reason I feel really passionate about finding a way to still, you know, keep abreast of what happens in these conversations. [Jim Haddadin] 13:09:17 So, we put together a project about a year ago called the Meeting Monitor. The idea is to use artificial generative artificial intelligence to help us find news leads within these government meetings. And the way it works is [Jim Haddadin] 13:09:33 When new meetings get posted on YouTube, we're monitoring a big collection of YouTube channels. They're automatically pulled down, transcribed, and then we take the text of that transcription and pass it to a large language model [Jim Haddadin] 13:09:50 With a very directed prompt. We're not asking it to put together, like, a big catalog of everything that happened at the meeting. We've asked it to kind of think with a news reporter cap on about what are three things that might be the most newsworthy developments that came out of this [Jim Haddadin] 13:10:06 This conversation [Jim Haddadin] 13:10:08 So the output from the meeting monitor system flows currently into an internal newsroom slack channel, and what you're looking at here is the output from a single meeting that got transcribed. This was January 2025. This was a meeting of the Hartford City Council [Jim Haddadin] 13:10:25 And you can see it gives us back a brief headline that you'll see in bold here, and then it gives us a very brief description of what it's pitching as the potential news lead that came out of it. [Jim Haddadin] 13:10:39 Reporters and editors can pop into that channel every morning, take a look at what surfaced from the night before. Usually it takes about an hour after a meeting posts on YouTube for us to pull it in and then adjust it into our system. You also get a link to go watch the source material on YouTube so you can verify the details if you're interested in picking up on that story to report it out [Jim Haddadin] 13:10:59 And as you can see, we've discovered this is a really inexpensive process. This entire meeting that we're looking at was a couple of hours long, and doing the transcription, passing it to the LLM for analysis took about 9 cents all in. On average, we're finding it's about 5 to 10 cents to run this process for a typical government meeting. [Jim Haddadin] 13:11:20 So we have it up and running currently in 23 communities across Connecticut. You can see them with the little dots on the map here. It's the type of thing that could scale very easily. As we add more YouTube channels, it's really just a matter of kind of adding the link to the list of stuff that we're taking a look at [Jim Haddadin] 13:11:37 And since we developed this system a year ago, we've also started looking at other types of records that we could pull in. So we're exploring ways to work with other video platforms in addition to YouTube, and also starting to look at how can we run the same process with things like government records or meeting minutes [Jim Haddadin] 13:11:54 And we've developed a pipeline to start working with a much bigger collection of communities with some of those records. So some important safeguards. There is no public facing generative AI content. Everything that the AI system produces is sticking into an internal channel so that we can consider it [Jim Haddadin] 13:12:12 And we can… we maintain editorial control over what the story is that we're going to report and how we would present that to our audience. And importantly, we're not replacing the work of any of our journalists. These are meetings that, by and large, not only were we not covering as Connecticut public, but really virtually no Connecticut reporter is going to be at a lot of these rooms to hear what happens. [Jim Haddadin] 13:12:36 A quick example of some of the work that's come out of it since we piloted this a year ago. On the left, you'll see a brief snippet of one of the summaries that surfaced in our system. This was talking about a very local controversy in a community called Fairfield [Jim Haddadin] 13:12:52 Where, as you can see, some concern had arisen at this meeting about the usage of something called the pink voucher. This caught my attention. It was, you know, it's a bunch of people on, like, a finance committee talking about something in a little bit of a circumspect way that suggested [Jim Haddadin] 13:13:08 That maybe there are some, you know, there was a usage of these vouchers that wasn't intended. We dug into it and discovered that a local official there who held a kind of a high-ranking position had purchased for his official use and a very expensive SUV and had it upgraded with a lot of bells and whistles [Jim Haddadin] 13:13:27 This proved to be, you know, a high interest story. It's obviously very small in scope in terms of who you would think is impacted here for a town like Fairfield, relatively small. But this wound up being the second most high engagement story on our website last year [Jim Haddadin] 13:13:42 I mention that because I think there is such an audience for this type of local news reporting that I think a lot of places have been starved for for a long time. [Jim Haddadin] 13:13:53 And just another example, you know, as I mentioned, this system is so interesting, not just for kind of understanding when, for example, there's, like, a vote taken on a budget, or they pass, you know, some kind of new school budget or something like that. There's so much serendipity in these meetings about what you're not going to expect coming out of them [Jim Haddadin] 13:14:10 So this was from a meeting of a harbor management commission on Long Island Sound, where it came out that a pod of bottlenose dolphins had turned up kind of unexpectedly. Why is that interesting? They used to be very, very rare in this part of Long Island Sound. So we did a story kind of exploring the reemergence of this type of dolphin. Our reporter dug into it and found it could be that you know fish stocks are rebounding, or the water quality is improving [Jim Haddadin] 13:14:36 But not only did we do the story, we talked to the folks who had seen these dolphins and managed to procure the video of them cavorting in the water, having a good time. So this was something we could repurpose for social media, something our audience really loved [Jim Haddadin] 13:14:51 So it's just to underscore, you know, this type of system, even though it's about dry government meetings, can lead you on to a range of really interesting coverage areas that really resonate with the audience. [Brendan Foley] 13:15:06 Alright, so, in addition, as we just heard, Jim talk about how we're using AI tools to kind of help gather potential news stories [Brendan Foley] 13:15:17 We're also looking at ways across the organization of using AI to help us be more efficient, and in this way, some of the tools I'm about to go through [Brendan Foley] 13:15:28 are actually public-facing, unlike the news monitor. So our challenge really is that day after day, our customer service team gets the same questions. If, you know, I'm sure there are a lot of people on this call who have explained how to activate Passport until they're blue in the face [Brendan Foley] 13:15:47 And if you feel like you're gonna get one more question about that, then your head's just gonna pop. [Brendan Foley] 13:15:53 So, these are the type of questions our team deals with, too, and when we're answering these questions, it means that we're not working on other projects or different tasks that could be more impactful for our audiences and our donors [Brendan Foley] 13:16:08 And it means our audiences are really restricted to getting answers to some of their questions during business hours. [Brendan Foley] 13:16:15 So, our solution, our approach is a private AI chatbot built by and for public media. So we call it Curio, which is a play on our tagline, Media for the curious [Brendan Foley] 13:16:30 Unlike some AI tools and other models, it's only using information that we have fed it our existing content. That is what it's using as its knowledge base, and we can embed it on as many web pages across our website [Brendan Foley] 13:16:48 as we would like [Brendan Foley] 13:16:49 So Curio provides instant answers. It feels like ChatGPT or Claude as you engage with it. But again, it's only focused on our content [Brendan Foley] 13:17:00 We know trust is really important, especially given how much of our organization is dedicated to news reporting, and so we make sure to really tightly control what Curio is trained on so it can provide trusted answers. [Brendan Foley] 13:17:15 We've built very strong content filters to try to keep answers on topic and consistent. And our goal, kind of big picture, is not to get rid of questions, it's to make answering the easier questions faster [Brendan Foley] 13:17:32 And to end answering the same questions over and over and over. [Brendan Foley] 13:17:36 So, what is Curio like to use? So we'll have a little bit of show and tell later, because Rob is an absolute genius. But the experience is really simple. For any page that Curio is embedded on, there is a floating chat button that can open a branded interface [Brendan Foley] 13:17:56 Answers stream in in real time. Curio remembers the conversation, so it can provide kind of contextually accurate and relevant answers as there's a conversation going on [Brendan Foley] 13:18:10 And it always errs on the side of referring users to staff, our audience care team if it doesn't know the answer. Because again, we don't want it giving people the wrong types of answers. [Brendan Foley] 13:18:25 So. [Brendan Foley] 13:18:26 Again, we get to control what curio knows. That is kind of building that in as a point of repetition because it bears repeating. We control what Curio knows. It pulls [Brendan Foley] 13:18:41 from our website URLs, it pulls from some PBS URLs when it comes to passport instructions, since those are updated typically faster on the national level than the local level. We can upload documents, PDFs, CSVs, otherwise, and we have several API integrations [Brendan Foley] 13:19:01 This means since it is specifically checking on URLs on our website, that we don't have to update information in two places. We don't have to update a web page and then update Curio. We can just update the page and it automatically carries over to Curio the next time the page is indexed [Brendan Foley] 13:19:20 So this is another example of a question we ask Curio, and based on the incredible work of Rob, again, you'll hear from him in a little bit. This is actually a complex two-part question. We asked, what was the last audacious, it's one of our shows [Brendan Foley] 13:19:38 What was the last audacious podcast episode in October 2025, and what's on CPTV Kids Sunday at 11.45 AM? So we really kind of embedded two questions in this one question. So Curio was able to dynamically look up and display information from our schedules [Brendan Foley] 13:19:56 In the case of Audacious, it actually can embed the audio player right in the chat. So if somebody wants to start listening then and there to the content, they can. There are also links to our little schedule cards so people can find out more about what is on [Brendan Foley] 13:20:16 On what's on, what's available to watch. [Brendan Foley] 13:20:18 So big picture, again, safety is absolutely critical for us. Curio is screening every interaction for kind of harmful or insensitive or abrasive content and can reply with fallback messages if [Brendan Foley] 13:20:35 needed. It can also block harmful content, so one of the things we have set up with Curio is a sentiment analysis, so we get email alerts when it kind of flags different questions, coming in as something we might want to pay attention to, but there are also filters to make sure that [Brendan Foley] 13:20:52 those emails that show up in our inbox aren't something that are, you know, upsetting or that we'd rather not read. It provides custom responses, so we have built-in ways that if somebody expresses ideas of self-harm, it can provide resources and phone numbers and [Brendan Foley] 13:21:09 information to try to redirect someone to the right place. And we have sensitive topic filtering, topic controls. One of the things that I loved the most in a really, like, morbidly hilarious way is when we were testing it, every time we asked it a question about donation, it just assumed we were talking about organ donation. So it took some tweaking to get there, but it's in a really [Brendan Foley] 13:21:34 good place right now. [Brendan Foley] 13:21:37 So, a bird's eye view of the assembly line. Again, we are controlling the knowledge sources, we are then kind of around those knowledge sources, building a guardrails to make sure that we kind of keep Curio on track [Brendan Foley] 13:21:54 focused and on topic, it can search and contextualize this information in its uploaded materials and on our website and be ready that when someone asks [Brendan Foley] 13:22:05 A question it can provide a clear, trustworthy answer, and we certainly put this through our paces during the training process, trying to think of all of the convoluted questions we might get from audience members or a donor. [Brendan Foley] 13:22:21 So we are using this data, then, to also get important feedback about what our audiences and donors and members are asking. And we're using that to try to inform choices we make for the membership experience [Brendan Foley] 13:22:37 Overall [Brendan Foley] 13:22:38 So we can see and analyze user activity. We can see what topics are popular, we can see how long people are spending with Curio, we can find points of frustration. And as I mentioned, we're not only getting kind of [Brendan Foley] 13:22:54 real-time alerts via email about what people are saying over the… expressing frustration or otherwise, but we're also getting really comprehensive monthly reports [Brendan Foley] 13:23:04 These monthly reports are giving us an opportunity to put human eyes on this flood of questions from people, and we are getting summaries, AI-generated summaries of usage and some suggestions. So again, there are human eyes on all of this. We're not taking anything Curio says at face value. We're always kind of probing a little bit deeper [Brendan Foley] 13:23:29 But it is a source of ideas for new support pages, or updating our FAQs, or kind of reinforcing some of our training documents on the Curio side. [Brendan Foley] 13:23:42 These knowledge base gaps that we get to take a look at, have made some really great suggestions. So, we can see how there is a topic about people wanting to know how they can update their billing information. Well, okay, well, we have a page for that, so let's steer them into this page [Brendan Foley] 13:24:00 It can kind of provide a summary of the issue, and it's providing some of its own recommendations. Some of these recommendations are, really helpful. Some of them, again, need human review, so it's a very typical rather than someone [Brendan Foley] 13:24:16 find on their own instructions, how to activate passport that are already on our website. They go straight to Curio in the chatbot. So sometimes Curio will say, people are asking how to activate Passport. You might want to create a webpage for that. Well, we do have a webpage for that, but [Brendan Foley] 13:24:31 Again, it kind of raises a flag of, well, maybe that page isn't as easy to find as we think it is, so let's spend some time to tweak and use this information to improve the member experience overall. [Brendan Foley] 13:24:44 So again, kind of ultimately Curio is helping us improve the audience experience. It's providing audiences and donors and members 24/7 support. It is not replacing staff [Brendan Foley] 13:24:59 That is… we are very upfront about that. This is about reducing staff load of kind of really low-level questions and giving them an opportunity to work on bigger, more impactful projects. [Brendan Foley] 13:25:11 And it's super easy to scale, since we can point it to any URL or upload any training document we would like, it really gives us a ton of flexibility to kind of build in features very quickly. [Brendan Foley] 13:25:25 Susan. [Susan Bell] 13:25:26 So how are we using these tools to grow our impact? And some of this will be review of what Jim and Brendan just went over, but important to say again and also to build into the tech stack, which I know [Susan Bell] 13:25:42 Many of you are probably interested in. So [Susan Bell] 13:25:47 This is kind of where things got really fun for us because when we launched Curio, when it went live, it started teaching us. We can see what people are asking for that we're not answering yet. And that is what Brendan just went over, those content gaps [Susan Bell] 13:26:02 We can track trends, sentiment, and engagement, not just what people are asking for, but what they care about. And we can continue to optimize improving answers while also managing cost and performance. This turns Curio from a tool into a feedback loop for our organization [Susan Bell] 13:26:19 And we're also not limited to just one chat bot. We can create multiple chatbots, each designed for specific audience. For example, we can have one for our public website, another for membership, another for newsroom support. And think of this too [Susan Bell] 13:26:36 as an internal assistant. We can launch one for HR if we wanted to, for staff to ask questions to. And we can have it in its own each chatbot can have its own role, its own tone, and its own purpose. [Susan Bell] 13:26:49 For the public meeting monitor, we can expand to more sources, more towns, more meetings, and more coverage. And I do want to make a little note here that say a lot of the things that we're going to look at from this point of the presentation forward are not things that we have live yet, but they are [Susan Bell] 13:27:06 They are future facing. [Susan Bell] 13:27:09 So, but it's just as we can it's just as important to get this into the hands of our journalist, to grow our impact is newsroom adoption, which we're working on and we're working on creating tools that fit into the newsroom's workflow [Susan Bell] 13:27:25 in a way that everybody's used to working. Are people used to getting new news tips off of Slack and their email? How can we incorporate it so this is not net new for the newsroom? [Susan Bell] 13:27:39 This means customization alerts tailored to specific beats, and continuously improving signal versus noise, so the information is easier to sort through for our newsroom, making sure that what surfaces is actually useful. [Susan Bell] 13:27:53 So this isn't just about building tools. It's about scaling its impact. So we're going to look a little bit of how it works, and then we're actually going to look at a live demo of some of this. [Susan Bell] 13:28:07 What the public meeting monitor is really doing is acting like a fast, very consistent reporter. It watches for new meetings, pulls them in automatically, turns them into transcripts, and then uses AI to identify what actually matters. And instead of someone having to sit through hours of footage, you get highlights, you get summaries, you get alerts delivered right to where your team already has worked [Susan Bell] 13:28:29 This is a layered system. So here is a little peek at some of the tech stack that we're using, but I'd like to say that this is flexible. [Susan Bell] 13:28:38 There are a lot of different AI tools that you can use out there. So kind of pick your poison. But it watches [Susan Bell] 13:28:48 At the top is the dashboard. The dashboard is coming. It is not actually live in our environment yet. But this would be what your team sees. Simple, real-time and searchable. Underneath that is where the processing pipeline is, where it's pulling in meetings, running transcription [Susan Bell] 13:29:04 and organizing everything. And then there's the AI layer, which is actually doing the summarization, tagging, and making sure that the content is actually usable. [Susan Bell] 13:29:14 And finally, it's all powered by our public data sources like YouTube, Civic Clerk, and others. And so it's turning all of these hours of manual work literally reduced down into minutes. Curio works a little bit different than the public meeting monitor, but the goal is the same, and that's saving time and making information usable [Susan Bell] 13:29:35 Curio isn't just generating answers, it's pulling from our actual content that we've curated and we've provided the knowledge sources for. So we are calling that Retrieval augmented generation, where it finds most relevant information first, and then builds a response around it [Susan Bell] 13:29:51 So instead of guessing, it's grounding in what your organization has already published. And then it delivers that answer instantly, whether it's on your website, in a dashboard, in a Slack channel, however we customize that information to flow. [Susan Bell] 13:30:07 The tech stack is similar to the public meeting monitor. You'll see many of the same AI tools here, but it's just applied to an audience experience instead of a newsroom workflow. At the top is what users see [Susan Bell] 13:30:21 It's a simple chat interface that Brendan reviewed with us. Underneath that is a back-end system handling requests, managing conversations, and streaming responses in real time. The AI layer is where the magic and efficiencies happening [Susan Bell] 13:30:37 are happening, but it's not guessing. It's searching our content first, and then generated an answer. And finally, it's connecting to all of our real data sources, your website, your documents, your external systems, like NPR, PBS, Omni [Susan Bell] 13:30:54 And it's connecting through APIs. This is what turns your content into something that people can actually use. [Susan Bell] 13:31:02 But I am, we thought it would be fun today to have our developer actually take us through some tools. Some of these are in use and some of these are future facing. But Rob, if you will, he's going to actually give us a quick tutorial of [Susan Bell] 13:31:18 Both the Public Media Monitor and Curio backend, and then we'll open it up for Q&A after that. [Rob Gabaree] 13:31:25 Hello? Alright, hold on. [Rob Gabaree] 13:31:30 Can everybody see my screen? [Rob Gabaree] 13:31:33 Nice. All right. Hello, I'm Rob, our Director of Technology slash developer, so today I'm going to show you two tools that we're developing, specifically the admin dashboards, for everything that we talked about today. [Rob Gabaree] 13:31:47 So this is basically Curio, the chatbot interface. This is the admin dashboard. And obviously, we only have one bot right now, but if you click it [Rob Gabaree] 13:31:58 You're able to select all of the important settings like the model that you want to choose from. Knowledge sources and tools, which I'll get into, which is basically like the heart of it. And then you could adjust what's kind of like the personality with system prompt [Rob Gabaree] 13:32:15 Which is actually pretty important because [Rob Gabaree] 13:32:18 We found early on testing, just, like, without, like, a good prompt, it was interpreting things like I need a passport, or I need passport help with like the government issued ID. So it's nice to have like a pretty [Rob Gabaree] 13:32:33 good and like in-depth assisted prompt. So there's even things like membership tiers and everything kind of hard coded. And then you could also adjust everything that you see around it, like the system greeting above it. Disclaimer, any response fallback, if there's like an issue with the API [Rob Gabaree] 13:32:50 And it has issues connecting or something safety. There's tons of guardrails around it. So, as mentioned, there's the open AI moderation Api we use, which has a bunch of different [Rob Gabaree] 13:33:03 category specific things that come back that get filtered, and there's, like, thresholds for each one. So you could adjust them if there needs to be higher or lower. There's also a default [Rob Gabaree] 13:33:16 fallback. There's also another API for guardrails that also does things like block social security numbers and other sensitive info, but I haven't integrated that yet. There's also some preliminary like jailbreak detection down here, where if somebody tries to get your system prompt [Rob Gabaree] 13:33:32 Hopefully [Rob Gabaree] 13:33:33 doesn't let you, but I mean, the prompt's not really anything crazy, and we're not hooked up to, like, financial systems or anything like that, so I'm not really sure. [Rob Gabaree] 13:33:43 what you're gonna do by jailbreaking, but we have built-in prompts to, like, send a message, like, I can't share internal instructions [Rob Gabaree] 13:33:50 And then appearance is just [Rob Gabaree] 13:33:52 Kind of just, like, default, so you could kind of go crazy like if you want to [Rob Gabaree] 13:33:57 Change the colors in real time and everything. [Rob Gabaree] 13:34:01 Customize, like, the fonts and sizes and everything like that. [Rob Gabaree] 13:34:06 Let's see here. [Rob Gabaree] 13:34:09 So now, as I mentioned before, each bot is connected to knowledge sources and tools. And knowledge sources are basically like the information that's actually able to use and learn from. [Rob Gabaree] 13:34:22 So, in our case, we have one source which contains static files, like just a membership FAQ PDF here. [Rob Gabaree] 13:34:31 But then we also have another knowledge source with a bunch of websites on our web, on our bunch of pages on our website [Rob Gabaree] 13:34:39 And these get automatically indexed [Rob Gabaree] 13:34:41 Once a week [Rob Gabaree] 13:34:44 And you can see the status here. Last one was 3 days ago. And it's also intelligent enough where I have it so it kind of blocks out the headers and footers and all the repetitive content, so the index resources are a bit more accurate in terms of content [Rob Gabaree] 13:34:59 And all of these sources also allow you to apply boosts or like weights to them [Rob Gabaree] 13:35:04 So we also have a PBS one with a bunch of PBS support articles, but we assign that a lower boost. So it still uses it, but it tries to look at our own content first and our system prompt and kind of only use the PBS stuff as like a fallback [Rob Gabaree] 13:35:20 And then tools, which is what I kind of am excited about, is custom. So these tools are basically allow you to look up dynamic information via third-party APIs and respond with it. So we made a radio schedule tool that talks to the NPR Composer API [Rob Gabaree] 13:35:39 Which is changing. We also made a TV schedule tool that talks to the PBS TV SS API, I think. And then we have a podcast tool that talks to the Omni studio API, which Brendan showed in that screenshot. They had the built-in embed inside the window [Rob Gabaree] 13:35:57 And there's another tool not listed here, but I made it. It connects to the PBS Media Manager API. So you could ask questions about our local mini docs and documentaries and everything that's in Media Manager [Rob Gabaree] 13:36:13 And then lastly, we have the reports [Rob Gabaree] 13:36:16 This seems like this is one of you. Somebody asked for a banana bread recipe where you could look at all your public chats. So [Lauren Komrosky] 13:36:22 That was Ray. [Rob Gabaree] 13:36:27 Somebody was, somebody tried our jailbreak, but at least it kind of works, gives you the message. But yeah, each report also has a cost breakdown of like the embeddings, the moderation API [Rob Gabaree] 13:36:38 Anything it costs for that total [Rob Gabaree] 13:36:41 total chat message, and it's good for, like, reviewing it afterwards, and, like, kind of replaying everything. [Rob Gabaree] 13:36:49 And the scheduled reports is more like an executive, like, senior leadership thing that gets generated every month based on that month's activity. And I actually have like a sample. Now, again, not everything here is going to be accurate, because this is like when we were testing. But the report gets generated like this automatically every month [Rob Gabaree] 13:37:08 That tells you, like, the chats, the messages, the cost [Rob Gabaree] 13:37:11 it generates an executive summary with key findings, usage [Rob Gabaree] 13:37:18 Messages by day a week [Rob Gabaree] 13:37:22 it kind of… it uses AI to figure out a lot of the topics and, like, figure out topics and, like, example questions for each one. [Rob Gabaree] 13:37:30 I spent a good amount of time on here because it actually hallucinates a lot between pages. So you had to really make sure it had all the information accurate [Rob Gabaree] 13:37:38 Page 3 mentions 312 and 95, which is [Rob Gabaree] 13:37:42 Like what it mentions on the cover page. And before, sometimes it would start to hallucinate as the report got bigger. So [Rob Gabaree] 13:37:49 You had to really work on it and, like, pass it the correct information on every turn when it would generate everything. [Rob Gabaree] 13:37:56 And then lastly, we talked about the sentiment monitoring, which is [Rob Gabaree] 13:38:01 Basically [Rob Gabaree] 13:38:03 It runs on a set interval that you could choose, like 1 hours, 2 hours, 3 hours. [Rob Gabaree] 13:38:08 You can send it to Slack or email, you can choose the bot and your sensitivity. And basically, if there's anything that comes up that here, anything negative or neutral, it sends it to us in Slack or via email immediately, so it's not just [Rob Gabaree] 13:38:25 go into a black hole. That makes sense [Rob Gabaree] 13:38:30 Yeah, aside from that, we have full support in the background as well for organizations. So I set us up as a Connecticut public org so we could easily add other orgs with members and they could all have their own bots. [Rob Gabaree] 13:38:43 And I forgot to also mention that these knowledge sources and tools could be attached to multiple bots. So if you have a knowledge source with like 37 websites, but then you wanted an HR bot or another bot, you could just attach it to both of them so you don't have to [Rob Gabaree] 13:38:58 redo each knowledge source for each bot, or even the tools so you can share them. [Rob Gabaree] 13:39:04 And that's pretty much it for the [Rob Gabaree] 13:39:07 The admin interface for the curio demo. Is there any questions in the chat or anything? [Lauren Komrosky] 13:39:17 I think there was some questions about the tech behind both. So maybe go through the monitor and then we'll come around and see if there's still open questions. [Rob Gabaree] 13:39:26 All right. Let's see, include this [Rob Gabaree] 13:39:29 So I'll switch to the public meeting monitor, which as mentioned, this is like a next generation, like a preview. It's kind of like an internal thing we're testing. It's not actually being used yet. The one in production right now is the one that Jim talked about where [Rob Gabaree] 13:39:45 It looks at just YouTube videos and transcribes them and picks out three summaries and sends it to our private Slack channel. This version is essentially kind of the same thing, the same features, but evolved with the actual admin interface and [Rob Gabaree] 13:40:00 more sources. So basically sources are like the heart of everything, and the first version obviously just had YouTube, but we started adding support for all these other agenda websites, like Civic Plus and Civic Clerk, and [Rob Gabaree] 13:40:15 some Connecticut-specific ones, and I'm sure other states even have other ones, but [Rob Gabaree] 13:40:22 All these sources kind of get added here the same way and everything generates a meeting. And [Rob Gabaree] 13:40:29 The non-YouTube sources, like Civic Clerk and Civic Web and everything kind of look like this just to give you an idea. So a lot of them are these things here. Then when you click through to them, each one has agendas and minutes and multiple PDF packets and stuff like this [Rob Gabaree] 13:40:47 So we kind of ingest all of them and all the sources and they get added to a queue run in the background as background processes and go through a bunch of process and steps. [Rob Gabaree] 13:40:57 And everything basically gets translated into a meeting [Rob Gabaree] 13:41:04 And everything is kind of set up the same way, so you could check the source types if you wanted to, or towns, a specific town or channel. And each one here, let's see, public safety. [Rob Gabaree] 13:41:19 Each one follows the same format. So every meeting, whether it's a YouTube source or one of these ones here, it all has the same thing where it [Rob Gabaree] 13:41:28 Well, first ingests all the stuff from the meeting, so you can see from this one, it has, like, the agenda packet. The agenda and the agenda packet. This one, I think, is kind of new, because usually there's minutes, but each one extracts the text and gives you a Pdf in the right where you can download the original stuff [Rob Gabaree] 13:41:47 It generates a summary [Rob Gabaree] 13:41:48 And then the processing tab actually shows you the status for everything. So you kind of see what's happening. And ultimately, it turns into a summary based on any of the stuff it has [Rob Gabaree] 13:42:01 So, once it scans it again, if somebody adds the minutes, it knows to automatically download the minutes and reprocess it, and then I'll regenerate these summaries again based on the minutes, which is probably more accurate than just the agenda. [Rob Gabaree] 13:42:15 But the goal is like these highlights here are the 2 to 3 blurbs that originally would get sent to Slack in our private Slack channel from YouTube. So we kind of have the same feature here for each meeting. So if you want to send it to slack, you can. Then it generates what it thinks is newsworthy summaries [Rob Gabaree] 13:42:33 Key topics, motions and votes action. It kind of pulls out everything it thinks is in the meeting. [Rob Gabaree] 13:42:41 And there's also a chat tab, which is cool. So this basically allows you to [Rob Gabaree] 13:42:46 Chat directly with that meeting, ask questions about it. So you can see here, I already asked one, and it saves it, too, so if you refresh the page, the meetings are… the chats are still going to be there [Rob Gabaree] 13:42:59 Let's see [Rob Gabaree] 13:43:01 So you can see another, actually [Rob Gabaree] 13:43:06 Go to a YouTube example. So you can see the YouTube ones still have the highlights. So basically everything kind of follows [Rob Gabaree] 13:43:12 The same format for all them [Rob Gabaree] 13:43:15 You could have chats and everything [Rob Gabaree] 13:43:18 And then on the left, there's also a chat here, which I actually was playing with. So this looks across all of your meetings. I think we have 300 plus right now. And it has a bunch of tools similar to the Curio chatbot, like tools to search just the meeting summaries [Rob Gabaree] 13:43:36 the titles [Rob Gabaree] 13:43:38 all sorts of different, like, ways to find information, and so I asked this question, like, any towns on their budget, and [Rob Gabaree] 13:43:46 It gave some information and links to like the meetings. This one I asked anything about affordable housing and was able to pick out a bunch of different meetings that kind of mentioned it and links to them down here at the bottom [Rob Gabaree] 13:44:01 So these chats here kind of just [Rob Gabaree] 13:44:05 Cover all the meetings [Rob Gabaree] 13:44:08 And now alerts is basically the original functionality that we set up where it just goes from YouTube to Slack via the new meeting alert where you could just choose the sources you want to monitor, your integration [Rob Gabaree] 13:44:23 Which right now, we just have email or Slack, and then how often you actually want to receive it. But we also added a feature that's like a smart alert where you could just kind of ask it [Rob Gabaree] 13:44:34 Like, what you're looking for via LLM prompts, and every time a meeting's processed, it kind of uses what you said to look it up and see if it's relevant, and then it would send it to you the same way. And there's also the more primitive, like, just basic, like, keywords and stuff if you wanted to target [Rob Gabaree] 13:44:50 Instead of like a [Rob Gabaree] 13:44:53 So you can see here, there's nothing yet for this one, but you can see like [Rob Gabaree] 13:44:57 You can tell it to, like, notify me about anything related to affordable housing, so it would run this after every meeting gets processed, and then, like, send you the alert via [Rob Gabaree] 13:45:05 However you want it to come in, so you have, like, ones for Slack here. You can see where they get here, they get delivered [Rob Gabaree] 13:45:12 And now the dashboard is basically [Rob Gabaree] 13:45:15 Again, this is all works in progress, so, like, everything, like, the prompts probably needs to be adjusted and optimized. It might not necessarily pull out what you think is important, but it kind of generates a top story dashboard, which [Rob Gabaree] 13:45:29 uses AI to kind of rank them what criteria here that you could see [Rob Gabaree] 13:45:35 And then when you click through to it, it will also take you to that meeting [Rob Gabaree] 13:45:38 Oops [Rob Gabaree] 13:45:40 It also takes you down here to a trending section where [Rob Gabaree] 13:45:44 kind of looks at everything and tries to look for up to 5 different trends that are kind of relevant a lot and link you to their sources to help you. [Rob Gabaree] 13:45:52 And then on the right here, up here is just recently completed, just literally just the meetings that are getting finished, so there's nothing special about that section. [Rob Gabaree] 13:46:01 And then Newswire is still, it's kind of basically just [Rob Gabaree] 13:46:07 The top stories, but expanded a little bit more at this point. So it's still like the same like criteria, how it rates it. It does add in like decay. So like older stories, even though they're rated higher, aren't going to be like number one just because they're rated higher because it gives more preference to like newer stories [Rob Gabaree] 13:46:26 Again, not saying like all this is accurate either because it generates potential angles and stuff, but I do want to add another feature to actually adjust the prompts for a lot of these things and how they get called, because right now it's kind of like built into the system back end like you're a journalist [Rob Gabaree] 13:46:41 Things like that. So that would give more control. [Rob Gabaree] 13:46:44 And let's see. [Rob Gabaree] 13:46:49 Integrations, as I said, is like, so you could right now, there's just two like Slack or email, but [Rob Gabaree] 13:46:56 There's I can't see why you can't add like Discord or like Telegram or any other sources in the future. [Rob Gabaree] 13:47:03 The AI providers is, again, we just have a couple right now, but I do have a set so you could choose, like, what model you want to use for, like, summaries and, like, text extraction, or validate titles [Rob Gabaree] 13:47:15 The validate title thing is used for YouTube. So a lot of YouTube channels sometimes don't have just government meeting videos or just like random videos. So this uses the nano model, which is very cheap and lightweight. So look at the title really quick and just say, hey, is this a government [Rob Gabaree] 13:47:30 Like, meeting, and if it isn't, then it doesn't process it and waste more time. [Rob Gabaree] 13:47:36 But it's pretty much it. I said, like, none of this is actually being used yet. This is more like an internal tool. Like all this is accurate. It's actually ingesting and working. All the numbers and like the queue, like they get updated in real time like as things get processed and everything, you don't have to refresh [Rob Gabaree] 13:47:53 There's also little buttons up here if you need to refresh it, but [Rob Gabaree] 13:47:56 Yeah, that's basically the demo. There's [Rob Gabaree] 13:48:00 really not too much. I don't understand more questions about this [Lauren Komrosky] 13:48:03 Yeah, let me pop in and see because I was trying to keep track, but we were also trying to answer them as we went. [Lauren Komrosky] 13:48:10 I do… there was a question about how many meetings were actively tracking [Lauren Komrosky] 13:48:15 Are we putting meeting notes on the website for hyperlocal coverage? No. How are we using all this great info in your reporting? So I'll let you guys talk to the number of meetings we're actively tracking. And then Jim did mention a couple use cases of how we're using these [Lauren Komrosky] 13:48:32 discoveries to then have our journalists go and actually do a little bit more research and write stories, but Jamie, you can talk to that more too. [Jim Haddadin] 13:48:41 Yeah, I'll jump in quickly. Rob, this is so cool. I'm getting up to speed on some of the features as we talk. I'm seeing stories that I want to jump in and do. So the way that it's working now, as we mentioned, is what you're looking at is a tool that we're building out and we'll be rolling into our newsroom, but at the moment, what the newsroom is working with is the version of it that is posting to Slack in an internal Slack channel [Jim Haddadin] 13:49:03 And so, right now, we have about 20 editors and reporters in the newsroom who have access to that channel and can take a look at stuff. When we bring people in, we have an onboarding process to let them know, you know, what the system is, that it's generative AI, that it can hallucinate and make errors, and so we've got to verify everything that's there [Jim Haddadin] 13:49:21 And we're also encouraging people not to use that material verbatim to, you know, figure out how they want to phrase it. But basically it's open to folks to take a look at everything that pops up as it comes along to see if there's something that they think they want to jump into. Maybe if it's something related to environmental reporting or environmental reporter would pick it up [Jim Haddadin] 13:49:40 Our midday newscaster saw something about a town that's rolled out new speed cameras and generated, like, an insane amount of fine revenue, and that became just kind of a short item that was in the midday newscast to say that that had come along. I'm I'm looking at stuff that maybe we could dig into that would take, like, a couple of records requests or something like that that would have a little more investigative half [Jim Haddadin] 13:50:00 So people are finding different use cases for what's in there, but generally it's kind of seeing something that you think is interesting and doing the reporting to flesh it out. In terms of the number of meetings that we're doing, the YouTube-only system that's posting to Slack is 23 communities. [Jim Haddadin] 13:50:17 it would be hard for me to ballpark what the number of meetings is, but it's probably, like, you know, a couple hundred a month, something along those lines. What we have expanded to with these additional platforms and sources has opened it up to with written records, something like 60 or 70 additional communities [Jim Haddadin] 13:50:34 And we have the unique position of all of the state boards and commissions. So this is like state agencies use the same backend to host their materials. So we have something like 150 different state boards and commissions that we're also running this pipeline on [Jim Haddadin] 13:50:50 And that's been a really interesting source of information, too, and it, you know, as a statewide news organization, it can kind of turn us on to stories that have a little bit of a wider breadth [Lauren Komrosky] 13:51:02 Thanks, Jim. And then I think there was one specifically about the cost calculator. And maybe, Rob, if you could talk a little bit more about how that was set up, what variables go into the calculation, and then our annual token budget, which I can answer is [Lauren Komrosky] 13:51:16 We don't really have a budget. This has been a little bit of figure it out as we go. We've been kind of using our own internal labor, as you can tell, for a lot of this, but we did want to know and make sure that this wasn't an extreme out-of-pocket cost. We have started to build in kind of TBD AI tools into the digital budget, I will say [Lauren Komrosky] 13:51:35 But Rob, you want to talk about the cost calculator? [Rob Gabaree] 13:51:38 I would say, yeah, for costs, really, especially with the YouTube ones right now, it's mostly the cost to transcribe it to text, like the audio, which I think was a question on here. For that, we actually use [Rob Gabaree] 13:51:54 The whisper model from OpenAI, but we do it via Grok, and I don't mean like Elon Musk Grok, I mean G-R-O-Q, which is like a fast inference provider. [Rob Gabaree] 13:52:05 But yeah, there's also Deepgram, which has, I think, Nova is the model [Rob Gabaree] 13:52:10 But yeah, I actually have some… I built in support so it has a backup, so if one audio transcription provider is down, it tries to back up one. [Rob Gabaree] 13:52:21 But then besides that, the only other cost would be like sending the actual summaries and like the prompt to the LLM, which we use the OpenAI SDK. [Rob Gabaree] 13:52:30 Directly for that [Rob Gabaree] 13:52:34 What else? We see all these other questions. [Lauren Komrosky] 13:52:37 There's [Lauren Komrosky] 13:52:38 Sorry, Rob, I didn't want to cut you off. I was going to move to another question. [Rob Gabaree] 13:52:41 Oh, I was trying to look for more questions. [Lauren Komrosky] 13:52:42 Okay, so the most recent question that came in was pushback. Was there or are we experiencing pushback from the news staff either because of ethics, environmental concerns, and just kind of change management. So I can talk a little bit about that and then Jim, you can add on [Lauren Komrosky] 13:53:00 So, we have approached AI kind of holistically before we even started in on this very specific project. So kind of at the highest of kind of governance levels, we created a AI committee at our board level [Lauren Komrosky] 13:53:14 So we have some folks that are very skilled in this. They have legal backgrounds in AI. They have digital tech backgrounds in AI. And so that has been kind of a constant support system and guidance along the way. We have also created guidelines [Lauren Komrosky] 13:53:30 for the organization, and Chad asked for those, and we can share them. But that is something that we did kind of a demystification webinar with our staff, too, to talk about, you know, where we are, and it's going to evolve. We're going to continue to revisit it, but kind of as things evolve, what are the expectations at the organizational level with AI? And I will say that one of the messages is we do encourage our employees [Lauren Komrosky] 13:53:53 to use AI, just use it responsibly, right? And there are some guardrails around that in terms of, like, sensitive information, and obviously not ever publishing anything that comes out of generative AI. We don't do any generative AI in terms of publishing video or images and things like that. So we can share those more broadly, but we kind of started with a foundation, right? A general understanding [Lauren Komrosky] 13:54:16 We've also created, like I said, a little bit of budget for some tools, and we've done some paid ChatGPT accounts for those that are interested, just so we can kind of block our prompts and our conversations in that instance with any of the training materials [Lauren Komrosky] 13:54:32 And then one of the big ways that we've kind of been bringing the organization along is through this little AI tiger group. So, as we started to talk about these things and do some research on what's happening in news and media and public media, there was definitely some folks that were kind of naturally inclined to be curious and want to play in that space, and you can see their faces right here. And so I said, you and you, you, you come here, we're going to work on this together and kind of help lead the charge [Lauren Komrosky] 13:54:58 We don't want anyone in our organization to feel forced into this, but we do want them to understand the opportunities, because there are opportunities here, and this is happening around us. And so how can we use this for good? And so it's this team that's kind of the tip of the spear in kind of showing those ways and showing those use cases to our team. But in terms of some concerns, yes, there are some concerns and I think there wouldn't be good journalists if they weren't [Lauren Komrosky] 13:55:25 So, I don't know, Jim, if you want to add on, talk a little bit about what you're hearing from the newsroom too, and how we're kind of working through some adoption stuff. [Jim Haddadin] 13:55:32 Sure. I mean, I think these are good questions and we are all going to have to work through them in our organizations. And I think we'll have an evolving set of answers. I think to reinforce something that Lauren said, you know, one way that I've been approaching it with people is to really stress that it's not a requirement to use these AI tools. I kind of start by telling everyone, you know, we're all really strong journalists in this newsroom and you're going to do good work whether you have AI there to help you or not [Jim Haddadin] 13:55:57 So, you know, you can engage with this to the extent that you do feel it's going to, you know, help you with your work. And, you know, I've heard of some other organizations kind of like in more commercial news where there's already some tension and some folks in the newsroom who kind of feel like they're getting pressure from management to use these AI tools that people are rolling out [Jim Haddadin] 13:56:15 And so I really wouldn't ever want to design it in that way or have people feel like they're in the position where they're getting pressure from the top to incorporate AI into what they're doing. [Jim Haddadin] 13:56:25 You know, I think [Jim Haddadin] 13:56:27 The other thing I would say about it is, as Lauren said, try to be judicious about when we're using it. It's kind of like we don't want to just use it for the heck of it. It's kind of like we want to have a reason that we're doing it and we want to do it in a way where it's additive versus kind of replacing what we're doing or trying to just turn out more content. And that, I think the kind of environmental concern that you see around AI as well. [Lauren Komrosky] 13:56:51 Yeah, and I would say, just like we have the AI Tiger team, we're also recognizing folks as they are exposed to this or we are sharing some of those wins, right? Some of those stories that the journalists did pick up that resulted from one of these, you know, from this discovery, from these tips [Lauren Komrosky] 13:57:08 We're noticing those that are leaning in, and, you know, I think that's the lowest hanging fruit, so to speak, when you're trying to do change management. So, work with those people and try to figure out what their needs are, what their concerns are, and kind of, you'll slowly kind of [Lauren Komrosky] 13:57:24 grow that that interest and and usage. But we are being mindful of it, and Jim is very intentionally describing how to use this and how not to use this as well. [Chad Davis] 13:57:39 I love the leaning in. We use that kind of approach here quite a bit where when someone's interested in something, you try to facilitate it as much as possible because that kind of internal drive will just get you so much further if you're trying to innovate [Chad Davis] 13:57:55 Thank you, guys, for doing this. I know we want to wrap at the top of the hour because some folks have some things. And thanks everybody else for joining us. We did not have the benefit of Rob when we were in Austin. So this is sort of the director's cut of this presentation and [Chad Davis] 13:58:10 And that bit, Lauren will tell you, like, I came up afterwards and like 90% of my questions, she was like, you need to talk to Rob about that. So [Lauren Komrosky] 13:58:17 It's like, yep, I am not that smart. Nope, that is Rob. [Chad Davis] 13:58:20 Yeah, yeah. So everyone who got a bit of a bonus, and if you want to share this with anyone, it will be live. We'll have it up on YouTube. Total credit to Amber for making all of that happen. And that will be ready [Chad Davis] 13:58:36 by middle of next week, probably, and hopefully they'll also have a newsletter out, and I'll be putting the links in there as well. So, [Chad Davis] 13:58:46 Thanks again to the folks for Connecticut Public for joining us. Thanks to NETA. Next webinar, June 18th [Chad Davis] 13:58:55 Again, it's gonna be a couple folks from the NewsHour talking about how they are kind of optimizing content and making use of Reddit in a very authentic way. Again, it was another highlight of the annual meeting and something we just immediately felt like we wanted to elevate and [Chad Davis] 13:59:11 get that conversation in front of folks here. So look for registration for that also in the next week or so, and until then, everyone else, just hang in there, and maybe start enjoying summer a little bit. [Chad Davis] 13:59:23 We'll see you soon. [Lauren Komrosky] 13:59:25 Thanks everyone.