Unleashing Creativity and Applying Standards to Generative AI

[Linda Wei] 12:33:28
Hi, I’m Linda Wei. I’m the Chief Content Officer for Alaska Public Media and co-founder

[Linda Wei] 12:33:36
of executive content managers or ECM

[Linda Wei] 12:33:39
I’m joined by, as you met earlier, Amy Shumaker, my co-founder for ECM and associate

[Linda Wei] 12:33:45
General Manager of Content for WGCU in Fort Myers, Florida.

[Linda Wei] 12:33:51
ECM is a professional development group for public media professionals that engage in content leadership.

[Linda Wei] 12:33:57
We provide education and direction to not only lead staff, but to promote the curation, production, distribution, and engagement of high quality content on all platforms.

[Linda Wei] 12:34:06
to elevate local voices regionally and nationally.

[Linda Wei] 12:34:11
And we’re so pleased to co-host this session on generative AI with Public Media Innovators, or PMI.

[Linda Wei] 12:34:18
PMI is a NETA PLC dedicated to exploring and leveraging emerging media

[Linda Wei] 12:34:24
and technologies like generative AI, virtual reality, games, and interactive media to enhance

[Linda Wei] 12:34:30
public media storytelling and audience engagement.

[Linda Wei] 12:34:34
I’m joined here today with Chad Davis, Chief Innovation Officer of Nebraska Public Media.

[Linda Wei] 12:34:40
and Amber Samdahl, Director of Design and Innovation for PBS Wisconsin.

[Linda Wei] 12:34:45
And David Huppert with the Director of Media Innovation Lab for PBS North Carolina.

[Linda Wei] 12:34:51
These three are the leads for PMI and will be monitoring the chat for today’s.

[Linda Wei] 12:34:56
event. If you are at the NETA Thought Leader Conference this past September, you know that Chad, Amber, and David produced two sessions on AI.

[Linda Wei] 12:35:06
getting the AI conversation started at your station and everything that’s wrong with AI. I highly recommend going back and watching recordings of both of those

[Linda Wei] 12:35:16
If you were unable to attend or if you want to revisit.

[Linda Wei] 12:35:19
those presentations. And it’s relevant because today’s webinar was developed as a continuation

[Linda Wei] 12:35:26
of those presentations.

[Linda Wei] 12:35:29
And I’m so happy to also share that our presenter today is Talia Rosen.

[Linda Wei] 12:35:34
Talia is the VP of Standards and Practices and Associate General Counsel at PBS.

[Linda Wei] 12:35:39
Talia oversees the team that is responsible for ensuring that PBS content complies with applicable

[Linda Wei] 12:35:46
editorial and funding standards. She’s also responsible for oversight of regulatory affairs, including developing PBS positions on relevant public

[Linda Wei] 12:35:56
policy matters. Talia, thank you so much for all you do at PBS and for the system.

[Linda Wei] 12:36:01
And really fun fact, if you were wondering where our icebreaker was coming from.

[Linda Wei] 12:36:06
Talia is also a board game designer.

[Linda Wei] 12:36:10
Happy to have you here. And we have a lot to cover. We’re going to have a full 90 minutes.

[Linda Wei] 12:36:16
With time for questions, we hope.

[Linda Wei] 12:36:18
So I’m going to go ahead and toss it over to you. Thanks for being here.

[Talia Rosen] 12:36:21
Thank you so much for having me and thank you to everyone in the audience for joining us this afternoon or this morning, depending on

[Talia Rosen] 12:36:30
where you are. I’m going to share my slides now and talk through them, but

[Talia Rosen] 12:36:35
But yeah, I’m really excited to talk today about generative AI and how we apply the

[Talia Rosen] 12:36:40
PBS editorial standards and practices, this nice blue book, to generative AI.

[Talia Rosen] 12:36:47
So let me go ahead and pull this up. I’m going to start by talking about kind of

[Talia Rosen] 12:36:53
core principles. And then I’m going to switch to talking about use cases.

[Talia Rosen] 12:36:59
And I’m going to be monitoring the chat if I can get this to work. So now you should see that. I’m going to slide this over.

[Talia Rosen] 12:37:07
I’m going to pull up the chat just so I can monitor it.

[Talia Rosen] 12:37:14
pull that over as well.

[Talia Rosen] 12:37:16
Nope. Okay, there we go.

[Talia Rosen] 12:37:21
Let me see if I can get that chat to come off of there. Yep. All right. I’ll put that over here.

[Talia Rosen] 12:37:26
So now I have the chat on my monitor so I can see that because I’m going to pause at a bunch of different places along the way to answer questions.

[Talia Rosen] 12:37:36
First question, yes, the session is being recorded.

[Talia Rosen] 12:37:40
So it’ll be on the um

[Talia Rosen] 12:37:42
on the site, the public media innovator site for you to check out afterwards.

[Talia Rosen] 12:37:47
So thank you so much to ECM and PMI for inviting me to give this presentation.

[Talia Rosen] 12:37:55
We’re not going to be able to cover it all even in 90 minutes, which I asked for that extra 30 minutes just to have even more time.

[Talia Rosen] 12:38:03
And I love talking about this stuff, so I’m happy to spend more time. But to jump right in here.

[Talia Rosen] 12:38:10
Okay, when I do that.

[Talia Rosen] 12:38:13
chat hides from me.

[Talia Rosen] 12:38:15
All right. Well, I may not always see the chat, but

[Talia Rosen] 12:38:18
Chad or Amberwell?

[Talia Rosen] 12:38:20
Tell me. Okay. All right. So we’re going to start by talking about the core values. Then I’m going to talk about a case study, particularly American Historia, which many of you, if not all of you, aired in September.

[Talia Rosen] 12:38:31
Then I’m going to talk for a minute about generative fill, which is, I think, the most worrying use of generative AI from a standards and practices perspective.

[Talia Rosen] 12:38:42
Then I’m going to go through the other nine use cases that I’ve seen to date.

[Talia Rosen] 12:38:47
And then questions and discussion. But again, I’m going to try to pause along the way for questions and discussion.

[Talia Rosen] 12:38:52
These nine use cases that I’m going to talk through in a little bit, those are derived directly from

[Talia Rosen] 12:39:00
the submissions that we’ve received here from producers and filmmakers that have proposed to use generative AI

[Talia Rosen] 12:39:08
And I’ve tried to abstract those into different sort of

[Talia Rosen] 12:39:12
types of uses.

[Talia Rosen] 12:39:14
You should definitely feel free to reach out to me and the whole standards and practices team there at standards at pbs.org.

[Talia Rosen] 12:39:22
with follow-up questions, because I think that every individual use case is

[Talia Rosen] 12:39:27
unique and warrants more discussion.

[Talia Rosen] 12:39:31
So my favorite website on the internet, hopefully everyone knows is pbs.org slash standards.

[Talia Rosen] 12:39:36
It’s where you can find the actual policies themselves, but also

[Talia Rosen] 12:39:41
whole wealth of resources that bring those standards to life, including our two page

[Talia Rosen] 12:39:46
guidance memo on generative AI, but also like tons of other stuff. And I feel like I can just brag about it because

[Talia Rosen] 12:39:52
I didn’t make it. The team made it. And I think they just did such a phenomenal job. And it’s crazy to think that a site like this didn’t exist.

[Talia Rosen] 12:40:02
three years ago. It launched a couple years ago.

[Talia Rosen] 12:40:07
really recommend checking that out and just browsing that for the case studies and everything that it has.

[Talia Rosen] 12:40:14
So let’s start with the core values.

[Talia Rosen] 12:40:19
The question that I started getting last year was.

[Talia Rosen] 12:40:21
how could PBS standards possibly apply to this totally unprecedented development in media? Your standards were in years ago.

[Talia Rosen] 12:40:29
before there was any generative AI, so they can’t apply, right?

[Talia Rosen] 12:40:34
And I think the answer to that is no, wrong. They can apply.

[Talia Rosen] 12:40:39
If you read the 12 pages of the 12 pages of

[Talia Rosen] 12:40:42
our collective public medias, editorial standards

[Talia Rosen] 12:40:45
closely, you actually find a lot of language that is, I think, remarkably prescient

[Talia Rosen] 12:40:51
And remarkably on point.

[Talia Rosen] 12:40:54
even though it didn’t anticipate this particular

[Talia Rosen] 12:40:58
technological development, but the values

[Talia Rosen] 12:41:01
damage apply. So we live in this really complicated world where there seems to be a new tool every day.

[Talia Rosen] 12:41:09
And you really have to try them to even be able to conceive of what they are or what they do. If you haven’t used any of these, I think start with cloud.ai.

[Talia Rosen] 12:41:21
just have a quote unquote conversation with it. And it’ll open your eyes to

[Talia Rosen] 12:41:28
the world we now inhabit. It’s not really a conversation because it’s a predictive text model.

[Talia Rosen] 12:41:33
that has no true understanding, but it predicts text in a way that I think passes the Turing test with flying colors.

[Talia Rosen] 12:41:42
start there and then go over to Notebook LM and upload a PDF, not a confidential or proprietary one by any means, but one that you grab off of

[Talia Rosen] 12:41:52
internet somewhere. And it’ll make you a 10 minute podcast that is indistinguishable from humans, but is

[Talia Rosen] 12:42:01
entirely synthetic.

[Talia Rosen] 12:42:04
You know, these tools are

[Talia Rosen] 12:42:06
remarkable, but they’re also overwhelming. And so I want to make it simple.

[Talia Rosen] 12:42:11
And I think it can be simple. I think that if you’re thinking about

[Talia Rosen] 12:42:16
Accuracy, transparency.

[Talia Rosen] 12:42:19
and inclusiveness.

[Talia Rosen] 12:42:21
then you’re on the right path here. So those are the three words I want you to walk away

[Talia Rosen] 12:42:27
from this session.

[Talia Rosen] 12:42:29
Remember, if you remember no other words accuracy

[Talia Rosen] 12:42:33
transparency and inclusiveness. These are the three core values, the three lenses through which

[Talia Rosen] 12:42:39
I and the team here looks at every potential generative AI use. And that applies whether it’s a voice

[Talia Rosen] 12:42:47
AI, text, images, video, songs.

[Talia Rosen] 12:42:51
Whatever it is, these are the values that have undergirded PBS and public media.

[Talia Rosen] 12:42:56
for decades and they’re, I think, more relevant than ever. And they actually give us an opportunity

[Talia Rosen] 12:43:02
to, I think, become more trusted as other media outlets, whether that’s CNET or Sports Illustrated, make missteps

[Talia Rosen] 12:43:10
with generative AI.

[Talia Rosen] 12:43:13
Our use, if it’s thoughtful, if it’s done with intentionality.

[Talia Rosen] 12:43:16
has the opportunity to actually elevate us even further above the competition as

[Talia Rosen] 12:43:22
trusted, thoughtful, and really content of consequence.

[Talia Rosen] 12:43:27
So let me dive into those three words a little bit more because I’m never one to stop with just three words.

[Talia Rosen] 12:43:33
when I could use 75 of them.

[Talia Rosen] 12:43:36
So accuracy

[Talia Rosen] 12:43:38
You know, a lot of this is common sense, but we live in a complicated world. So I think it’s good to remind ourselves

[Talia Rosen] 12:43:45
What we mean by accuracy, we mean that we have to fact check everything that we say.

[Talia Rosen] 12:43:51
And that AI output is not a reliable source of fact.

[Talia Rosen] 12:43:56
And you’d be forgiven for thinking when having this quote unquote conversation with Claude.ai.

[Talia Rosen] 12:44:03
or any of these tools.

[Talia Rosen] 12:44:06
for thinking that it’s

[Talia Rosen] 12:44:08
believable because if nothing else these tools

[Talia Rosen] 12:44:12
have the appearance of credibility that is hard to overstate.

[Talia Rosen] 12:44:17
When they say things, when they generate text or images, they do it with a level of

[Talia Rosen] 12:44:26
seeming authority that is really impressive. They say things, it types things.

[Talia Rosen] 12:44:33
that that just

[Talia Rosen] 12:44:35
sounds so believable.

[Talia Rosen] 12:44:39
are reasonably frequently entirely fictional and entirely not based in fact. But it’s easy to fall for it, right? It’s easy to think that you’ve researched something, that you’ve looked something up.

[Talia Rosen] 12:44:49
When in fact, you’ve generated synthetic material, it’s easy to think that you’ve sort of Google image searched up this image of whatever that you’ve just created.

[Talia Rosen] 12:44:58
or you’ve just Googled up this text. But what you have to do is sort

[Talia Rosen] 12:45:02
Constantly remind yourself

[Talia Rosen] 12:45:04
that this is less reliable than Wikipedia, certainly. It’s less reliable than like a good intern. And so this is sort of like a

[Talia Rosen] 12:45:16
read it

[Talia Rosen] 12:45:17
mediocre Wikipedia kind of mediocre intern

[Talia Rosen] 12:45:21
That doesn’t mean it’s useless. It means that you just can’t believe what you’re seeing

[Talia Rosen] 12:45:28
or reading. And so you could use it as a starting point for sure, but you need to know your sources. And this isn’t a source, just like Wikipedia is not a source, just like your intern isn’t a reliable source of fact.

[Talia Rosen] 12:45:41
You need to vet that content. You need to find actual sources that underlie

[Talia Rosen] 12:45:46
what it’s saying or depicting.

[Talia Rosen] 12:45:50
Transparency.

[Talia Rosen] 12:45:52
This is where we can set ourselves apart. There are a number of media companies that have been accused of using generative AI and they almost never cop to it.

[Talia Rosen] 12:46:03
Because they’re very secretive and their standards and practices are actually very secretive. When I share our standard site with other major media companies, they sort of

[Talia Rosen] 12:46:15
are amazed that we have such a thing and that we’re so public about it. But I remind them that

[Talia Rosen] 12:46:19
public media it’s it’s in the name

[Talia Rosen] 12:46:22
that were public. And we’re going to lean in on our standards and we’re going to lean in to transparency on our use of generative AI.

[Talia Rosen] 12:46:32
This has been one of the things that has been

[Talia Rosen] 12:46:34
producers has made producers decide

[Talia Rosen] 12:46:37
a handful to not use generative AI. They’ve come with, hey, I want to use it for this depiction or

[Talia Rosen] 12:46:42
or for this part of the program. And we say okay

[Talia Rosen] 12:46:47
you know, we’re going to make sure you have your experts to help you fact check.

[Talia Rosen] 12:46:50
And we’re also going to do some form of labeling.

[Talia Rosen] 12:46:53
That’s going to be customized to the use case.

[Talia Rosen] 12:46:55
that’s where people say, well, wait a second.

[Talia Rosen] 12:46:58
I don’t want to do that. But for me, this is just like the language in the editorial standards that’s been around forever.

[Talia Rosen] 12:47:05
around recreations and simulations. It’s on the page 12

[Talia Rosen] 12:47:10
The very last page of this editorial standards, but it talks about identifying recreations and simulations. And that’s exactly what we have here.

[Talia Rosen] 12:47:19
The standards also have this great section all about using labels

[Talia Rosen] 12:47:25
And disclosures to aid the audience’s understanding. That’s here on page eight of the standards.

[Talia Rosen] 12:47:30
And I think about this transparency

[Talia Rosen] 12:47:34
It’s not a disclaimer, right? It’s not that we have something to be embarrassed about.

[Talia Rosen] 12:47:38
It’s that we want to empower the audience we want to

[Talia Rosen] 12:47:42
foster trust. We want to foster credibility. We want to put them in a position of really understanding

[Talia Rosen] 12:47:49
what they’re being shown so that they’re being shown

[Talia Rosen] 12:47:52
synthesize that material in their own mind and know where it’s coming from.

[Talia Rosen] 12:47:57
So transparency is a huge piece of this. And if you want to use generative AI,

[Talia Rosen] 12:48:02
you first think about how am I going to fact check everything that’s either depicted or said or written

[Talia Rosen] 12:48:09
And then secondly, how am I going to tell the audience about this? What is the means I’m going to use for that? And it’s almost certainly not going to be closing credits.

[Talia Rosen] 12:48:17
Unless it’s an incredibly minor use, it’s going to be an opening card. It’s going to be part of the narration.

[Talia Rosen] 12:48:24
maybe a lower third. Maybe there’ll be a supplemental material on the show page but

[Talia Rosen] 12:48:30
It could be a combination of those things.

[Talia Rosen] 12:48:33
Third is inclusiveness. And this is uh

[Talia Rosen] 12:48:39
Certainly not the least important. This is probably the most important thing, which is that these tools are so

[Talia Rosen] 12:48:47
baked in with bias and prejudice. And if you’re not careful.

[Talia Rosen] 12:48:52
you will perpetuate long lasting stereotypes in your use of generative AI.

[Talia Rosen] 12:48:57
And I am not the only one saying this. Here’s just a small sampling of headlines.

[Talia Rosen] 12:49:03
from MIT Technology Review, from London Interdisciplinary School.

[Talia Rosen] 12:49:08
from Bloomberg, from Trinity College Dublin. There’s so much reporting and research

[Talia Rosen] 12:49:14
into this fact that we have to be very careful.

[Talia Rosen] 12:49:20
It doesn’t mean you can’t use these tools, but it’s more important than ever that we have a wide range of perspectives and voices on your team behind the camera

[Talia Rosen] 12:49:30
to evaluate the output because there’s plenty of output that is

[Talia Rosen] 12:49:33
obviously sexist and racist, but there’s also, I think, a lot of output that may be insidiously prejudiced, have insidious biases that are harder for any one person to spot based on their own personal experiences.

[Talia Rosen] 12:49:46
We have to be really careful about

[Talia Rosen] 12:49:49
what we produce, how we think about using it, and how we mitigate for these biases in the prompts.

[Talia Rosen] 12:49:56
And this is not only due to the training data, although the training data that most of these models have used.

[Talia Rosen] 12:50:01
draws on a lot of systemic biases, but it’s also based on the underlying technological architecture of

[Talia Rosen] 12:50:09
these predictive generative tools.

[Talia Rosen] 12:50:12
that predict the next token based on what is the most likely next thing.

[Talia Rosen] 12:50:17
And based on society, the most likely next token when we’re asking about a CEO

[Talia Rosen] 12:50:24
is a particular person from a particular demographic background.

[Talia Rosen] 12:50:28
And that bakes in all of

[Talia Rosen] 12:50:33
societies prejudices and stereotypes and so

[Talia Rosen] 12:50:38
It’s such a core part of how these tools work that it’s

[Talia Rosen] 12:50:42
If you read the system card for 01 for the new GPT newish

[Talia Rosen] 12:50:48
GPT-4 model, you see them wrestling with this, but you also see that they don’t really have a solution for it.

[Talia Rosen] 12:50:55
And they can pick around the edges with reinforcement learning and

[Talia Rosen] 12:51:01
with chain of thought reasoning techniques, but it’s not necessarily solvable.

[Talia Rosen] 12:51:06
It is not clear what a solution would even look like, but

[Talia Rosen] 12:51:09
I think the solution is probably the solution is probably

[Talia Rosen] 12:51:13
the human first and human last principle that you see in Nebraska public media standards, right?

[Talia Rosen] 12:51:18
You’re going to have to, I think, always have humans

[Talia Rosen] 12:51:21
to help you evaluate this output and think about it critically.

[Talia Rosen] 12:51:25
The good news is that there’s a consensus emerging, right? That standards are coming out from all sorts of organizations all over the world.

[Talia Rosen] 12:51:33
that are really, I think, infused with these same core values.

[Talia Rosen] 12:51:39
And so PBS is far from the only ones doing this. I think we were one of the first, but it’s really nice to see

[Talia Rosen] 12:51:46
that these same values are

[Talia Rosen] 12:51:49
sort of permeating and percolating across

[Talia Rosen] 12:51:52
of media organizations. And I’m seeing here in my notes

[Talia Rosen] 12:51:56
For this slide, I actually have links to all of these, so I’ll give those too.

[Talia Rosen] 12:52:01
ECM and PMI so that they can post

[Talia Rosen] 12:52:03
the actual links to all of these policies with slides on their site.

[Talia Rosen] 12:52:10
Because I think it’s worth seeing. You start to see these commonalities. And I’m really happy about that.

[Talia Rosen] 12:52:15
I’m going to pause there and check the

[Talia Rosen] 12:52:19
Try to check the chat, see if there’s anything. Oh, okay. We do have some questions. Let me scroll through here.

[Talia Rosen] 12:52:27
Okay, wait.

[Talia Rosen] 12:52:28
Okay, I’m scrolling up. Try to get back earlier. Oh, there’s lots going on in the chat here. Let’s see here.

[Chad Davis] 12:52:36
I’ll winnow it down too because I’ve been like.

[Talia Rosen] 12:52:38
Okay, you’ll winnow it down for me. Yep, there’s a lot of things.

[Chad Davis] 12:52:40
Well, half of these posts to me. So David actually asked a question about recreations.

[Chad Davis] 12:52:45
And it stems from the American Revolution of sizzle we are

[Chad Davis] 12:52:50
opening credits we saw at the NETA conference. But he asks, does a film like American Revolution have to disclose when they recreate

[Chad Davis] 12:52:59
those kind of beautiful scenic Winfrey scenes and some old footage you know they were to recreate something like that with that.

[Talia Rosen] 12:53:06
That’s a great question. So our cutoff for like prominent disclosure of recreation is really like the invention of camera technology so

[Talia Rosen] 12:53:16
Because what the standards say is that whenever there’s a possibility that members of the audience could reasonably be confused or misled. And so if we’re depicting

[Talia Rosen] 12:53:28
things that happened sort of from 1860, 1870 onward.

[Talia Rosen] 12:53:33
then we’re going to have a more prominent disclosure of recreations. If it’s something from the 1700s.

[Talia Rosen] 12:53:40
Or something like dinosaurs battling, like in Chad’s example in his newsletter today.

[Talia Rosen] 12:53:46
We’re not going to do it in the same sort of way. With AI, we’re leaning in

[Talia Rosen] 12:53:54
on more transparency at this moment in time.

[Talia Rosen] 12:53:57
So if they were using generative AI for those scenes in the American Revolution to depict the 1700s.

[Talia Rosen] 12:54:03
than we would figure out a way to be more transparent. They’re not, to be clear. They’re not using any generative AI for the American Revolution.

[Talia Rosen] 12:54:12
But so we are leaning in further, even if it’s depicting content that was before the invention of

[Talia Rosen] 12:54:18
cameras in the propagation of camera technology. And that’s what I’ll talk about here with American Historia in a moment.

[Talia Rosen] 12:54:25
But that’s sort of our dividing line. And that comes from that section for D5 on page 12 of the standards.

[Talia Rosen] 12:54:34
And I meant to say this earlier, and I apologize at the top but

[Talia Rosen] 12:54:39
Everything I’m talking about today is rooted in the PBS editorial standards, and I am not

[Talia Rosen] 12:54:44
talking today about the many

[Talia Rosen] 12:54:47
many legal and copyright issues that are inherent in the use of generative AI. There’s a separate team of lawyers

[Talia Rosen] 12:54:55
at PBS that is thinking critically about that. But if you want to use this, I mean, everyone should be experimenting with this tool behind the scenes to understand it, I think. But if you want to use it in some public facing way.

[Talia Rosen] 12:55:07
you need to talk to lawyers, you need to talk to E&O insurers, you need to think really carefully about the infringement risks. There’s a slide that that legal

[Talia Rosen] 12:55:17
that is just like case names and it’s like dozens of active litigation

[Talia Rosen] 12:55:22
on these tools. And personally, like if I were betting, I would say that

[Talia Rosen] 12:55:26
there’s a decent chance many of these tools won’t exist in

[Talia Rosen] 12:55:31
a year and a half because we may be living through sort of a Napster 2.0

[Talia Rosen] 12:55:37
Now, licensing models are emerging and I think that like the technology is not going anywhere

[Talia Rosen] 12:55:43
But the legal community seems split 50 50 about whether

[Talia Rosen] 12:55:47
any of what’s happening right now is lawful.

[Talia Rosen] 12:55:50
And the Supreme Court will probably decide for us in june of

[Talia Rosen] 12:55:54
I’m going to guess 2020.

[Talia Rosen] 12:55:58
And then we’ll find out.

[Chad Davis] 12:56:02
We have one other question. This comes from Nikki Bates, who works with Nancy and i uh one of our outstanding producers at

[Chad Davis] 12:56:08
Nebraska Public Media. She has a question about transparency and as it relates to short form pieces. She says, I wouldn’t hesitate to use the word

[Chad Davis] 12:56:17
with a piece on generative AI footage b-roll in a long form documentary. But then what about like a short, like an image spot or a 30 second promo or

[Chad Davis] 12:56:27
Or something like that. And this, I will say like when Sora dropped earlier this week, Nikki and I were swapping messages and she asked us

[Chad Davis] 12:56:36
I was like, hey, hang on, ask that of Talia later. So what about short form content image spots, things where it’s, you know, maybe illustrative as opposed to something that is driving the editorial?

[Talia Rosen] 12:56:39
Yeah.

[Talia Rosen] 12:56:49
Yeah, I mean, I…

[Talia Rosen] 12:56:51
I think we’re all figuring this out as we go. I think that’s a really good question and there’s no

[Talia Rosen] 12:56:56
definitive answer. I think what we’ve written right is that

[Talia Rosen] 12:57:00
We could use all sorts of different tools, right? Lower third top of show closing credits, supplemental online materials.

[Talia Rosen] 12:57:06
It’s easier with long form. You’re absolutely right.

[Talia Rosen] 12:57:09
short form that’s on broadcast is going to be the most challenging, right? Because there’s really no way, there’s no easy way to contextualize it.

[Talia Rosen] 12:57:18
Short form online is easier, right? If you’re doing it on youtube or on

[Talia Rosen] 12:57:23
TikTok or wherever, you could do something beneath the video, right? Like in the YouTube, like on It’s okay to be smart

[Talia Rosen] 12:57:31
Which I’ll always call that, even though I know it’s now be smart with Johansson. We put it in the show notes underneath.

[Talia Rosen] 12:57:39
With a podcast, I would put it in the podcast notes. For broadcast

[Talia Rosen] 12:57:44
I could see coming up with a rationale for including it in a 30 or 60 promo

[Talia Rosen] 12:57:50
without a label.

[Talia Rosen] 12:57:52
If you don’t think that you’re misleading people, if you don’t think that a reasonable audience member is going to feel like they were

[Talia Rosen] 12:58:00
duped or tricked, right? We want to put ourselves in their shoes and think about that. So if this is

[Talia Rosen] 12:58:06
graphics of trees or graphics of spirals or whatever. And it’s not people and it’s not contemporary and it’s not

[Talia Rosen] 12:58:16
I’m going to mislead people, then maybe there’s an argument for that. The other argument that I could see is if you made a 30 second or 60 second promo

[Talia Rosen] 12:58:24
And its purpose was to deliver people to a full-length show.

[Talia Rosen] 12:58:29
and you wanted to use some piece of generative AI in the short form, maybe when they get to the full length show.

[Talia Rosen] 12:58:35
then they’ll somewhere in there

[Talia Rosen] 12:58:38
learn about the use of generative AI.

[Talia Rosen] 12:58:40
We do this with funders. We figure we don’t need to tell people all of the funders

[Talia Rosen] 12:58:45
when they watch a 30 or 60 second promo because the goal is to get them to watch

[Talia Rosen] 12:58:50
an hour-long show and that’s where they’ll learn about the fact that Lockheed Martin was a funder or whomever was a funder, right?

[Talia Rosen] 12:58:56
there’s different use cases depending on whether you’re talking about broadcast or online, whether you’re talking about something that really is a promo or maybe something that’s more standalone.

[Talia Rosen] 12:59:03
And then I would be really careful if we’re thinking about depicting

[Talia Rosen] 12:59:08
humans or depicting

[Talia Rosen] 12:59:10
some known setting or people in a contemporary setting. If it’s something that’s more stylized, something that’s more background, then

[Talia Rosen] 12:59:18
who may be.

[Talia Rosen] 12:59:20
Maybe it’s okay. And I think that this is an evolving debate, right?

[Talia Rosen] 12:59:24
how much is this like?

[Talia Rosen] 12:59:28
traditional CGI and Photoshop tools that you’ve been using for 10, 15,

[Talia Rosen] 12:59:35
years and how much of this is something new. I think that

[Talia Rosen] 12:59:39
this is going to evolve over the coming couple of years.

[Talia Rosen] 12:59:42
But that at this moment.

[Talia Rosen] 12:59:44
The quickest way for us to lose trust is by using generative AI, having people see that and having us not be open about it. And so we want to

[Talia Rosen] 12:59:53
innovate and experiment, but we want to be conscious of

[Talia Rosen] 12:59:57
how we’re going to make people feel if they

[Talia Rosen] 13:00:00
know all of the facts, right? Like we don’t want to benefit from the fact that maybe they won’t notice or maybe they won’t

[Talia Rosen] 13:00:06
find out. This is the same advice we give people when they’re editing an interview. All interviews have to be edited down right for content.

[Talia Rosen] 13:00:15
Almost all.

[Talia Rosen] 13:00:16
But the two things to be thinking about are, you know, if

[Talia Rosen] 13:00:21
the audience had full access to all of your interview footage where they feel like you did something misleading?

[Talia Rosen] 13:00:25
Or that you tried to sort of pull one over on them, or they feel like you did a fair representation of that. And I think it’s sort of the same mentality here. It’s like, how would people feel if they had

[Talia Rosen] 13:00:36
we’re privy to everything.

[Talia Rosen] 13:00:38
I’m going to jump into this case study because I think it will be useful. So American Astoria

[Talia Rosen] 13:00:43
is what I’m going to talk about. But first.

[Talia Rosen] 13:00:45
You may have seen there have been four uses of generative AI in PBS content in the last

[Talia Rosen] 13:00:51
six to nine months. The first use was on Be Smart on the thumbnail, this image of Johansson. And this is the note that we included in the, there was nothing in video, but it was in the episode description.

[Talia Rosen] 13:01:06
Then in, I think it was April of 2024, we had both A Brief History of the Future and Nova’s AI Revolution, both of which included

[Talia Rosen] 13:01:16
a number of generative AI images. I think both were pretty straightforward from a standards and practices perspective. There were lots of legal and copyright questions that went into the analysis of both of those shows.

[Talia Rosen] 13:01:28
And discussions with E&O insurers and all sorts of things we’re not going to get into here today. But in Brief History of the Future, you saw an architect

[Talia Rosen] 13:01:35
And it talked about how he used generative AI to imagine buildings of the future in tune with nature as inspiration.

[Talia Rosen] 13:01:42
for his architecture. And this is, I mean, I didn’t say this earlier when I was talking about how you might want to play around with Claude or Notebook LM.

[Talia Rosen] 13:01:49
But the weirdest thing about all of this, and maybe it’s obvious to everyone who’s sort of been playing with these things, but the computer programs that we’ve used for the previous

[Talia Rosen] 13:01:57
few decades have been really good at

[Talia Rosen] 13:02:00
facts, accuracy, data.

[Talia Rosen] 13:02:03
and really bad at creativity.

[Talia Rosen] 13:02:05
And this kind of turns everything on its head. These tools are

[Talia Rosen] 13:02:10
particularly bad at facts accuracy data.

[Talia Rosen] 13:02:13
being tethered to like reality in a reliable way but

[Talia Rosen] 13:02:18
I think they’re pretty good at creativity, at least for those of us who aren’t that creative intrinsically. I think that people who have a lot of creativity

[Talia Rosen] 13:02:26
see these image tools and these music tools and say well that’s pretty

[Talia Rosen] 13:02:30
planned. But for those of us who don’t have that gene, the tools are not a bad place to just

[Talia Rosen] 13:02:37
brainstorm and ideate. And that’s what this architect is doing in this show that many, if not all of you broadcast last spring.

[Talia Rosen] 13:02:45
And then the one I really want to focus on is American Astoria.

[Talia Rosen] 13:02:49
And to do that, I’m going to start by, I think everyone’s

[Talia Rosen] 13:02:53
Hopefully familiar with the show, but let’s watch this like 45 minute second intro just so you can see

[Talia Rosen] 13:03:00
You know what? I don’t know if I clicked it.

[Talia Rosen] 13:03:02
play with sound. So tell me if you don’t hear the sound.

[Talia Rosen] 13:03:06
Once we get to the sound.

[Talia Rosen] 13:03:09
You know, when I was growing up in Jackson Heights, Queens, I learned the same history that they taught kids all over America.

[Talia Rosen] 13:03:16
that George Washington cut down his cherry tree, that Ben Franklin

[Talia Rosen] 13:03:20
flew his kite and Abe was honest.

[Talia Rosen] 13:03:23
Guess what was missing?

[Talia Rosen] 13:03:26
The Latino contributions to this country.

[Talia Rosen] 13:03:30
You can’t have a history book about the United States without considering

[Talia Rosen] 13:03:35
histories of all of the Latino communities. Most of the history we’re still taught is from the point of view of the conquistadors who tried to wipe out

[Talia Rosen] 13:03:43
any trace of advanced indigenous cultures

[Talia Rosen] 13:03:47
As a kid, that lack of inclusion in the history textbooks

[Talia Rosen] 13:03:50
made me feel invisible.

[Talia Rosen] 13:03:54
So I’m going to assume the audio worked there.

[Talia Rosen] 13:03:57
So that’s great.

[Talia Rosen] 13:04:00
And right after that opening, we played this advisory.

[Talia Rosen] 13:04:05
Again, not a disclaimer. It’s a disclosure to empower the audience. It’s an advisory.

[Talia Rosen] 13:04:10
And this is what we played for them, which has a voiced part. So hopefully you’re hearing the voiced part as well.

[Talia Rosen] 13:04:17
To bring the history in this series to life, the following program includes images generated by AI tools.

[Talia Rosen] 13:04:22
To learn more about the process of developing these images, visit pbs.org slash historia.

[Talia Rosen] 13:04:31
So there was so much to tell people about how generative AI was used to bring this history from the 15 and 1600s to life.

[Talia Rosen] 13:04:39
in this storybook kind of fashion that

[Talia Rosen] 13:04:44
We prevailed upon the producers to create this Q&A.

[Talia Rosen] 13:04:50
on this custom website that we could tell people about because we really wanted to lean in

[Talia Rosen] 13:04:57
on transparency and talk about why generative AI was used to show this part of history.

[Talia Rosen] 13:05:04
how it was carefully vetted.

[Talia Rosen] 13:05:06
And sort of the thinking both from an artistic perspective and from a historical perspective.

[Talia Rosen] 13:05:11
And I’m really proud of the Q&A that we came up with with

[Talia Rosen] 13:05:16
with John Legazamo and the producing team in partnership with WNET and Latino Public Broadcasting.

[Talia Rosen] 13:05:23
And one of the most important steps of all of this was bringing on these three

[Talia Rosen] 13:05:29
really world-renowned experts in the history to carefully vet

[Talia Rosen] 13:05:35
every single image

[Talia Rosen] 13:05:37
and prompt and piece of generative AI output to make sure that it was as accurate as we could know

[Talia Rosen] 13:05:46
based on the records from this time period and that we were not propagating any stereotypes or biases about the people that were the subject of this really important historical documentary.

[Talia Rosen] 13:06:00
So that was such a vital step. Now, I think that one of the takeaways that I would suggest for everyone from this webinar

[Talia Rosen] 13:06:10
is that this was not a shortcut, right? This use of generative AI was not a way to get to air quicker.

[Talia Rosen] 13:06:19
Or cheaper, it took a long time to vet all of this content.

[Talia Rosen] 13:06:24
And to get these experts on board and to have them looking at all of the images really carefully and to create

[Talia Rosen] 13:06:31
this Q&A. So it’s something that can absolutely be done, but it’s not something that’s going, I think, and I don’t think the producers on this program would suggest that it’s not a way to

[Talia Rosen] 13:06:45
like just get things done more quickly. In fact.

[Talia Rosen] 13:06:49
Other producers have come in and said, hey, I want to use it for this or I want to use it for that. And when we say.

[Talia Rosen] 13:06:56
great. What’s your plan in terms of accuracy?

[Talia Rosen] 13:07:01
you know, vetting

[Talia Rosen] 13:07:03
In terms of transparency and in terms of evaluating for inclusiveness

[Talia Rosen] 13:07:07
And we just ask those questions.

[Talia Rosen] 13:07:10
Nine times out of 10,

[Talia Rosen] 13:07:12
The answer is never mind.

[Talia Rosen] 13:07:14
And it’s not that we don’t want it to happen. We just want to make sure that there’s a plan in place to really address these values through the use.

[Talia Rosen] 13:07:23
I’ll leave you on this example with this clip that shows another disclosure

[Talia Rosen] 13:07:29
One of the examples of a disclosure in show in lower third that uses a web marker to remind people that this is generative AI.

[Talia Rosen] 13:07:39
And to tell them where to go to learn more.

[Talia Rosen] 13:07:43
There was a third important person at this diplomatic meet and greet.

[Talia Rosen] 13:07:48
Cortez’s native interpreter.

[Talia Rosen] 13:07:50
The Spanish called her Doña Marina, and the Maya gave her the title of respect, Malincene. Malinche is an extremely divisive figure in Latin history.

[Talia Rosen] 13:08:02
She gave birth to Hernan Cortez’s son, metaphorically. This is the first Latino, a child with both indigenous

[Talia Rosen] 13:08:10
in European ancestry.

[Talia Rosen] 13:08:12
She’s the mother of all modern Mexico. But as I learned, many also see her as a traitor, a temptress who aligned with the conquistadors over her own people.

[Talia Rosen] 13:08:24
She actually was born into a noble Aztec family.

[Talia Rosen] 13:08:29
She was sold to a Chontal Mayan community.

[Talia Rosen] 13:08:33
She’s basically on the lower rungs of the society in Chontan Maya.

[Talia Rosen] 13:08:39
She learns the language, of course. Hernan Cortez is traveling through the southern part of Mexico.

[Talia Rosen] 13:08:47
He looks scary and dangerous, and so they just like, here, we’re going to give you some gifts, move on.

[Talia Rosen] 13:08:56
All right, I’m going to pause again and check the chat, or I don’t know, chat, if you want to, if there’s any other questions I can address.

[Talia Rosen] 13:09:03
Before I jump into the next section.

[Chad Davis] 13:09:05
I think go ahead and jump in, Legendary

[Talia Rosen] 13:09:07
Okay, great. Well, I won’t be thinking about your questions or, you know, feel free to email the questions.

[Talia Rosen] 13:09:13
to standards at pbs.org if you have questions afterwards.

[Talia Rosen] 13:09:19
Which this reminds me, actually, I gave a similar presentation at the PBS Producers Summit

[Talia Rosen] 13:09:25
Last month or not that long ago, this fall.

[Talia Rosen] 13:09:28
And I was really struck by something and I wanted to share it with all of you, which was that

[Talia Rosen] 13:09:34
Most of the questions in the room with 50 people or I don’t know, 100 people, whatever it was.

[Talia Rosen] 13:09:42
were really about how

[Talia Rosen] 13:09:46
worrying generative AI is and how how problematic

[Talia Rosen] 13:09:51
it is or might be. And then 100% of the questions that I got in the hallway

[Talia Rosen] 13:09:58
throughout the afternoon and the next day were about how

[Talia Rosen] 13:10:03
people wanted to use it and specific questions about their particular use case that they were thinking about.

[Talia Rosen] 13:10:11
And I say that because I think that it’s

[Talia Rosen] 13:10:16
My biggest worry is that there’s a there’s like a

[Talia Rosen] 13:10:20
a shame about using it

[Talia Rosen] 13:10:22
When talking in a group setting. And then there’s a very strong interest in using it

[Talia Rosen] 13:10:29
But without doing it privately. And my job in standards, I think, is to shine a sunlight on that and to like

[Talia Rosen] 13:10:36
try to make those conversations

[Talia Rosen] 13:10:39
open and forthright and open and forthright

[Talia Rosen] 13:10:42
and not kind of behind the scenes, because I think that the vast majority of

[Talia Rosen] 13:10:47
stations and producers that I’ve talked to are very

[Talia Rosen] 13:10:51
interested in experimenting with and potentially using this technology but

[Talia Rosen] 13:10:55
in big group settings, people tend to be very reticent for fear of

[Talia Rosen] 13:11:02
sort of being judged. And a lot of this is wrapped up in the fact that like many of these tools.

[Talia Rosen] 13:11:08
are using unlicensed training data, and there’s a lot of things to think about with that.

[Talia Rosen] 13:11:14
This is not a presentation about that. This is a presentation about sort of the

[Talia Rosen] 13:11:18
standards, the editorial standards. So there’s a lot to think about here.

[Talia Rosen] 13:11:23
So the 10 use cases that I want to talk about in our time remaining are listed here on the screen with my

[Talia Rosen] 13:11:32
very oversimplified traffic light summary.

[Talia Rosen] 13:11:35
of whether I think that they’re pretty good to go, have some concerns or have

[Talia Rosen] 13:11:42
significant concerns, but that’s an oversimplification of something much more complicated. I’ll also say that like this list was probably seven or eight use cases long.

[Talia Rosen] 13:11:52
Two months ago and then some more use cases came to my attention and potentially by the end of this hour

[Talia Rosen] 13:12:00
We’ll have an 11th, a 12th, or 13th use case. I don’t pretend to have all of the

[Talia Rosen] 13:12:06
answers or use cases. I think that there are essentially infinite possible ways that this technology can be used.

[Talia Rosen] 13:12:12
But rather than go through this in order, I’m going to just start with generative fill because I think it’s the most important thing. I keep saying that this thing or that thing is the key takeaway.

[Talia Rosen] 13:12:20
And I think the three core values are a key takeaway. I think whatever the other thing I just said 10 minutes ago was a key takeaway is also a key takeaway.

[Talia Rosen] 13:12:28
But I think that if you remember nothing else, how about this? Please don’t use generative fill.

[Talia Rosen] 13:12:35
without thinking really carefully about what you’re doing. My biggest fear in all of this is that the way that generative fill is being baked into Adobe and other tools.

[Talia Rosen] 13:12:47
will make it seem sort of like spellcheck almost where people will just click it and make the image fit the frame. And that is so anathema.

[Talia Rosen] 13:12:56
to, I think, so much of the work that public media does that I really want people to be really

[Talia Rosen] 13:13:04
careful and thoughtful about how we use this incredibly powerful

[Talia Rosen] 13:13:09
new tool that we have. On top here with the green outline are three real images

[Talia Rosen] 13:13:15
These are all drawn from, they’re not identical to what I’ve received from producers thinking about using this technology but they are

[Talia Rosen] 13:13:26
very closely comparable. On the left here is a real image from the Statue of Liberty, 1930s.

[Talia Rosen] 13:13:35
an actual photograph from Afghanistan in, I think, 2020.

[Talia Rosen] 13:13:40
And then an actual 1500s wood carving.

[Talia Rosen] 13:13:44
Below that is what you might get, although each time you use the tool, you’ll get something a little bit different because of the way that it’s built.

[Talia Rosen] 13:13:53
Below that in the orange box is a generative AI

[Talia Rosen] 13:14:00
output that

[Talia Rosen] 13:14:02
fills the frame, right? It would fill a 16 by nine frame

[Talia Rosen] 13:14:06
And that might seem innocuous enough or innocent enough, but I think that it goes against our editorial standards

[Talia Rosen] 13:14:16
really more clearly than any other use of

[Talia Rosen] 13:14:19
these technologies i think that

[Talia Rosen] 13:14:24
Whether it’s an archival photograph like the Statue of Liberty photograph

[Talia Rosen] 13:14:30
And you’re creating some new elements of reality that simply weren’t there, or it’s a contemporary photograph like the one in Afghanistan.

[Talia Rosen] 13:14:39
that it’s

[Talia Rosen] 13:14:41
It’s so misleading.

[Talia Rosen] 13:14:43
And you might argue

[Talia Rosen] 13:14:45
like a producer might have argued that, hey, this image from Afghanistan, we’re not adding anything.

[Talia Rosen] 13:14:52
I would say that you are adding something. You’re adding this empty tarmac.

[Talia Rosen] 13:14:57
that we don’t know if it was actually empty you’re adding this sort of

[Talia Rosen] 13:15:01
feeling, you’re changing kind of feeling

[Talia Rosen] 13:15:04
what was there. And maybe you’ve talked to people who were there and they say

[Talia Rosen] 13:15:09
Yeah, though, this is what it looked like. It was pretty empty they are um

[Talia Rosen] 13:15:13
this is fine. Okay. Well, then what you’ve done is you’ve created a recreation, right? You’ve created

[Talia Rosen] 13:15:20
a new depiction that appears to be a photograph

[Talia Rosen] 13:15:24
is not a photograph anymore, right? It’s part of it that’s a photograph, but there’s a whole other part of it that isn’t a photograph.

[Talia Rosen] 13:15:31
If you feel it’s essential that you use that recreation.

[Talia Rosen] 13:15:34
Okay, but let’s talk about how we’re going to tell the audience. And that’s going to be really tricky, right? Because it’s it’s

[Talia Rosen] 13:15:41
you know, half of it’s not a recreation, the other half of it is.

[Talia Rosen] 13:15:45
I will have to figure out how we go ahead and communicate that if it’s really essential.

[Talia Rosen] 13:15:51
So the answer isn’t that you can never use generative fill. That’s definitely not.

[Talia Rosen] 13:15:55
what I’m saying, but I think what I’m saying is that if you think it’s essential for editorial reasons that you use generative fill.

[Talia Rosen] 13:16:04
Then let’s talk about how we’re going to fact check the thing that’s filled, how we’re going to communicate to the audience.

[Talia Rosen] 13:16:11
And certainly if people are being added, how are we going to think about inclusiveness and bias?

[Talia Rosen] 13:16:15
And I’ve had a producer say to me, well, isn’t this just like what Ken Burns does with old photos?

[Talia Rosen] 13:16:22
And I would say definitely not.

[Talia Rosen] 13:16:27
His style, of course, is to zoom and pan over images, not to

[Talia Rosen] 13:16:32
create synthetic fills around synthetic around

[Talia Rosen] 13:16:35
historic images. And then the other thing I would say is a lot of our content ends up in PBS Learning Media and is taught in schools.

[Talia Rosen] 13:16:43
And so if we have a famous 1930s image of the Statue of Liberty and we decide

[Talia Rosen] 13:16:48
that we want it to be 16 by 9.

[Talia Rosen] 13:16:51
Well, now that’s going to be the new historic record that’s taught in schools and that is passed down.

[Talia Rosen] 13:16:58
And I don’t think we want to be responsible for

[Talia Rosen] 13:17:02
changing the history of 1930s photography of the Statue of Liberty.

[Talia Rosen] 13:17:09
Please be careful and thoughtful about this.

[Talia Rosen] 13:17:12
And then very recently, Generative Extend was announced.

[Talia Rosen] 13:17:15
makes me very nervous. Extending existing footage to add a few more seconds.

[Talia Rosen] 13:17:22
feels very fraught again not

[Talia Rosen] 13:17:24
impossible to use. I think that there are filmmakers and documentarians who will come up with

[Talia Rosen] 13:17:29
brilliant and creative ways to use

[Talia Rosen] 13:17:32
all of these tools in very like

[Talia Rosen] 13:17:36
in ways where the use is a feature, not a bug. And that’s great. And I can’t even anticipate the ways that they will think to use these things. And I can’t wait to see it.

[Talia Rosen] 13:17:45
But if it’s just…

[Talia Rosen] 13:17:48
to make it fit the frame, which we have millions of other ways that we can all do that.

[Talia Rosen] 13:17:53
Which just depends which

[Talia Rosen] 13:17:54
get a few more seconds out of an image of footage, particularly if it’s of people

[Talia Rosen] 13:18:00
Let’s pause and have a really sort of thoughtful conversation about that use case. There are other uses of generative AI that are much less worrying, I would say.

[Talia Rosen] 13:18:11
And all of what I’m saying really is rooted in the best practices on page 11 of the longstanding editorial standards. And I don’t think any of this really

[Talia Rosen] 13:18:20
rests on this being an AI tool or not. We’ve had editing tools

[Talia Rosen] 13:18:25
for the longest time that have allowed us to do things like franken biting or composite quotes

[Talia Rosen] 13:18:31
that were very slick.

[Talia Rosen] 13:18:33
But our job is to edit in a way that fairly and accurately portrays reality.

[Talia Rosen] 13:18:39
And I would argue that most of the generative fill uses that I’ve seen

[Talia Rosen] 13:18:44
don’t meet that standard that was set up many, many, many years ago for us.

[Talia Rosen] 13:18:49
And the standards acknowledge that our jobs, that the job of producers is to edit material, but to do so in a way

[Talia Rosen] 13:18:57
that.

[Talia Rosen] 13:18:58
collects and orders information that

[Talia Rosen] 13:19:01
accurately portrays reality. So I don’t think this needs to be AI to have this concern. This is really part of the core of what our standards stand for.

[Talia Rosen] 13:19:12
I want to get back to the full list and I’m going to walk through the other nine use cases. I think

[Talia Rosen] 13:19:18
Yeah, I think I have time.

[Talia Rosen] 13:19:21
Maybe, hopefully. But I’m going to pause there for questions.

[Chad Davis] 13:19:27
We have one actually at Davis.

[Talia Rosen] 13:19:29
Yay, I love questions.

[Chad Davis] 13:19:31
No relation.

[Chad Davis] 13:19:35
Specifically about the experience you had at the PBS Producers Academy and um

[Chad Davis] 13:19:40
that the fact that you had the discussions in the hallway that were very different and the questions you were getting from

[Chad Davis] 13:19:46
in a more maybe official, like everybody’s listening capacity. And Kit writes um

[Chad Davis] 13:19:54
Do you feel that represents different aspects of the stations and productions? I think specifically management maybe versus

[Chad Davis] 13:20:04
wine producers or folks who are maybe more in the trenches versus

[Chad Davis] 13:20:07
dictating the strategy. Kit says, because as a station in place seems very familiarly drawn to them across sort of

[Chad Davis] 13:20:16
lines of professional roles or different types of roles. Are you noticing

[Chad Davis] 13:20:21
stratification and the sort of attitudes about the use of generative AI management versus

[Chad Davis] 13:20:27
you know maybe production.

[Talia Rosen] 13:20:29
That’s interesting. No, I had not. I didn’t notice that. And in thinking back to who was speaking in each of these different instances i don’t i don’t

[Talia Rosen] 13:20:38
get the sense that it was roles-based, although my sample size is not large enough to have a definitive answer for you and your experience or your mileage may vary, as they say.

[Talia Rosen] 13:20:49
I think it’s more that there’s a lot

[Talia Rosen] 13:20:54
swirling

[Talia Rosen] 13:20:56
a lot of anti-AI sentiment swirling. You see it with the Coca-Cola ads and people, the backlash there you see it

[Talia Rosen] 13:21:04
When people accuse Disney of using AI and I think the Loki posters, you see it every time there’s a discussion. You saw it in the Olympics with both Al Michaels and with the Google Gemini ad and that ridiculous ad about writing a letter to your childhood hero.

[Talia Rosen] 13:21:22
So you see it every month it feels like there’s another uproar, another backlash. And so I think that the

[Talia Rosen] 13:21:30
the questions and the discussion in the larger group setting

[Talia Rosen] 13:21:34
is really motivated by that. And I didn’t get a sense that it was dependent on roles or anything like that.

[Talia Rosen] 13:21:39
And that sort of

[Talia Rosen] 13:21:42
my sense was discouraged maybe people, or maybe it’s just that people have very particular questions about a particular idea and a particular use case.

[Talia Rosen] 13:21:50
But I sort of got the feeling that it was that there was a worry that like saying that I’m thinking about using it

[Talia Rosen] 13:21:57
in a bigger group setting.

[Talia Rosen] 13:21:59
felt like you were being there was a level of vulnerability there and

[Talia Rosen] 13:22:03
I think it’s interesting because when I give a presentation in a bigger group setting, sometimes I get the pushback that’s like.

[Talia Rosen] 13:22:09
How can you be encouraging or supporting

[Talia Rosen] 13:22:11
the use of generative AI. And I think my answer is that that’s definitely not what I’m doing. I’m just recognizing that

[Talia Rosen] 13:22:17
this is something that’s happening and it’s real. And we have to wrestle with it and like

[Talia Rosen] 13:22:23
that’s just my core part of my job is to wrestle with the fact that it’s

[Talia Rosen] 13:22:28
it’s truly happening. And it’s happening at least with dozens of producers. And so I can’t sort of

[Talia Rosen] 13:22:36
wish it away. And I think that having any hard and fast, like bright line rules that are like never this or always that

[Talia Rosen] 13:22:43
that’s not going to work either. So we’re going to have to have very sort of nuanced and thoughtful

[Talia Rosen] 13:22:48
conversations and rules about this. So I haven’t seen that kit, but it’s an interesting uh

[Talia Rosen] 13:22:54
possibility, certainly.

[Chad Davis] 13:22:56
I think Kit’s question, well, there’s a follow-up. As a designer, I appreciate the point about having to wrestle with the reality and come up with the standards.

[Chad Davis] 13:23:05
Because they are very important. We appreciate your call. I think kids…

[Chad Davis] 13:23:08
kind of question and follow up also give permission to the all attendees like if you have doubts, if you have concerns, if you want to express some of that like this is definitely

[Chad Davis] 13:23:17
an okay space to do that if you don’t feel like

[Chad Davis] 13:23:20
You’ve got to be all cheerleadery about it all the time. And there will be different perspectives from like a management standpoint. There’s just like someone who’s actually got to produce release.

[Talia Rosen] 13:23:30
Yeah, that’s totally fair. I mean, everyone has a different role here to think about.

[Talia Rosen] 13:23:35
And I certainly don’t have all the answers here. And so I am

[Talia Rosen] 13:23:44
Let me jump into these use cases, though, before I run out of time. Okay.

[Talia Rosen] 13:23:48
So the first one I’m going to talk about is B-roll stills because that’s by far and away the most common use case that has been submitted to me by producers in real life.

[Talia Rosen] 13:23:59
And then these other ones that I’ve seen as well.

[Talia Rosen] 13:24:02
And for all of this, again, I’m like a broken record here. And my goal is to be the most predictable person at PBS.

[Talia Rosen] 13:24:10
I’m going to just say, okay, well, have we thought about accuracy? Have we thought about transparency? Have we thought about inclusiveness? I’m not going to surprise anyone with those questions.

[Talia Rosen] 13:24:19
I don’t think. So B-roll stills come in all flavors. Some of these are actual ones that have been submitted. Others are ones that I created inspired by things that have been submitted.

[Talia Rosen] 13:24:29
And I’ve arranged these from left to right in a way to just sort of give you a sense of the spectrum of use cases, whether it’s like a dinosaur flying a spaceship

[Talia Rosen] 13:24:39
that’s going to have far less concerns from a standards perspective, may have concerns from a legal perspective, may be copyright infringement, not for me to say.

[Talia Rosen] 13:24:50
To the images on the right that raise a lot more standards.

[Talia Rosen] 13:24:55
issues. I think for all of these, we want to think about, does there need to be transparency

[Talia Rosen] 13:25:00
You know, for the image of the buildings that are kind of trees that look very like

[Talia Rosen] 13:25:05
Rivendell or Lothlorien for the Lord of the Rings fans out there

[Talia Rosen] 13:25:11
You know, that was easy because the show talked about it being generative AI. And so we didn’t need to think about whether there was going to be a label or not.

[Talia Rosen] 13:25:22
For something like that or the dinosaur.

[Talia Rosen] 13:25:26
we probably don’t need like an on-screen label, right? Because it goes to that question about American Revolution, right? Like if it couldn’t be real.

[Talia Rosen] 13:25:35
then we probably don’t need to label it. Do we want to put something in the end credits?

[Talia Rosen] 13:25:39
Just to err on the side of being open and not being accused of hiding the ball.

[Talia Rosen] 13:25:44
Maybe. Will that change over the next year or two?

[Talia Rosen] 13:25:48
Probably. But then you get to something like that image in the middle, which I love. This was from a poetry program wanting to illustrate a poem.

[Talia Rosen] 13:25:58
from 1922. And I think from a standards perspective, that’s a great use case. Let’s illustrate 1920s poems.

[Talia Rosen] 13:26:07
With generative AI, yeah, that sounds like

[Talia Rosen] 13:26:10
public media’s bag, right? But I would add a lower third there because that’s not a photograph.

[Talia Rosen] 13:26:16
And I think it looks like a photograph. He could be reasonably confused with a photograph. So I would add a lower third when we’re talking about a black and white image of the dresser of Diehl.

[Talia Rosen] 13:26:28
But, you know, it would be something we’d want to discuss, you know, how do we handle that? Or are generative images being used throughout the poetry program?

[Talia Rosen] 13:26:37
In this case, it was just one. But if they’re bringing issue out, maybe it’s something at the top of the show. Maybe it’s a quick card.

[Talia Rosen] 13:26:44
to talk about that? I don’t know. We’ve used cards at the top on various POV programs when they’re full of recreations, like the infiltrators.

[Talia Rosen] 13:26:52
So a card at the top is not a bad way to approach that if it’s something that’s sort of pervasive throughout. But if it’s a one-off like this was.

[Talia Rosen] 13:27:00
Then a lower third. Now, I’ll note that that image

[Talia Rosen] 13:27:03
ended up not working because of copyright concerns. So again, I’m just looking at this from a standards perspective.

[Talia Rosen] 13:27:09
For copyright issues, it’s the terms of use that matter. Luckily for me, I don’t need to worry about what the terms of use say for any given tool.

[Talia Rosen] 13:27:16
But if each of these is created with a different tool.

[Talia Rosen] 13:27:18
it’s going to matter what the terms of use say. And I know no one wants to read the terms of use.

[Talia Rosen] 13:27:23
And you may not in your personal life

[Talia Rosen] 13:27:26
from an organizational perspective, what the terms of use say is

[Talia Rosen] 13:27:29
is really important. And some of the terms of use

[Talia Rosen] 13:27:31
some of these tools are kind of insane and absurd. Like have you indemnifying the tool

[Talia Rosen] 13:27:37
we’re using it rather than the other way around which

[Talia Rosen] 13:27:40
Yeah, you don’t want to do that.

[Talia Rosen] 13:27:43
And then you get to the images on the right where you really need to think about, you know.

[Talia Rosen] 13:27:48
Are we misleading people? Are we misleading?

[Talia Rosen] 13:27:51
accurate? Are there biases here? If we’re going to depict children or we’re going to depict

[Talia Rosen] 13:27:56
You know, late 1800s railroad workers, we’re going to really need to

[Talia Rosen] 13:28:02
go through each of those three lenses

[Talia Rosen] 13:28:05
very purposefully, I think. So B-roll stills are the most common use I’m seeing. People have an interview subject

[Talia Rosen] 13:28:13
the program might as well be a TED Talk, but you want to cut away from this person’s face and show something.

[Talia Rosen] 13:28:20
great generative image tools might be the answer if you don’t have the budget for

[Talia Rosen] 13:28:27
human created artwork.

[Talia Rosen] 13:28:29
or for stock images. Or if you want something really custom.

[Talia Rosen] 13:28:33
Or you want to create a certain style. And from a standards perspective.

[Talia Rosen] 13:28:37
it’s doable.

[Talia Rosen] 13:28:42
But it should be done thoughtfully.

[Talia Rosen] 13:28:45
And you can see, for instance, this boat image in the bottom that’s um

[Talia Rosen] 13:28:49
American historia. And so that goes back to the example we talked about earlier.

[Talia Rosen] 13:28:56
My next one is simpler. Well, no, nothing is simple in this world. The next one example is different, but I’ll pause there to talk about B-roll stills if there are any questions on B-roll stills.

[Talia Rosen] 13:29:12
Anything?

[Chad Davis] 13:29:13
We have questions, but they are not specifically b-roll questions.

[Talia Rosen] 13:29:15
Okay, that’s okay. Let’s go back and let’s do questions generally.

[Chad Davis] 13:29:19
just you can roll through some questions. Okay, cool.

[Talia Rosen] 13:29:21
Yeah, I mean, if there are questions about later use cases, I might punt them, but yeah.

[Chad Davis] 13:29:22
Um.

[Chad Davis] 13:29:24
Totally. Hunting is a valid response here. So, uh.

[Chad Davis] 13:29:30
One, and just for everybody who’s typing in question, very cool.

[Chad Davis] 13:29:36
Maybe check and make sure the dropdown menu has it set to everyone because a lot of stuff’s kind of bouncing around the host and panelists. So I’m pulling from that. Some of you may be like, where did this question come from?

[Chad Davis] 13:29:45
I don’t see it. And that’s because we’re getting it on the host’s panelists like filtered version so

[Chad Davis] 13:29:52
look at that as you’re doing your questions. Amber just posted one for everyone, so you should be able to see that.

[Chad Davis] 13:29:56
And we’ll get to that in a minute.

[Chad Davis] 13:29:59
Let’s see. First question. Joan lights in and says she’s retiring soon

[Chad Davis] 13:30:06
And there are discussions about putting our voice into an AI generator so

[Chad Davis] 13:30:12
She continued to host in the years to come. She is a quote animated host, wants to know, what do you think?

[Talia Rosen] 13:30:20
Yeah, well, we have a whole voice slide coming up, but I will say that um

[Talia Rosen] 13:30:25
This is one where I think the legal issues are particularly prevalent. I’m just going to jump ahead to that slide.

[Talia Rosen] 13:30:30
And that’s why I put legal at an exclamation point there, because this has come up. There’s a couple different voice uses that we’ve heard about.

[Talia Rosen] 13:30:40
First one is interviewee edits. And I’m a pretty, there aren’t a lot of clear yeses or nos in the standards world. We deal with a lot of gray, but I’m a pretty clear no on interviewee edits. This is not what the question was about, to be clear.

[Talia Rosen] 13:30:53
But we did get a submission from a station that asked if they could voice clone their interviewee

[Talia Rosen] 13:31:01
change what they said to make it clearer.

[Talia Rosen] 13:31:04
that’s totally prohibited by the standards. And that’s not because it’s AI. It’s because, again, we need to represent reality.

[Talia Rosen] 13:31:12
And so doing pickups using voice cloning of interviewees, I think, is probably one of the clearest, besides generative fill.

[Talia Rosen] 13:31:19
clearest issues under

[Talia Rosen] 13:31:22
under the standards.

[Talia Rosen] 13:31:24
The flip side of that is what the question is about, which is a narrator voiceover or a host.

[Talia Rosen] 13:31:28
someone who’s paid and employed by the producer or the station. And I think from a standards perspective, that is probably okay.

[Talia Rosen] 13:31:38
with some measure of transparency. And you actually see this if you go to the Washington Post or I just, this isn’t translation we’re talking about, but

[Talia Rosen] 13:31:47
If you go to Washington Post or you go to New York Times, you can get AI to voice, I think, every single article now, but there’s a very clear disclosure

[Talia Rosen] 13:31:59
on the post and the times that it’s

[Talia Rosen] 13:32:01
an AI voice.

[Talia Rosen] 13:32:02
I would be thinking about if this is going to happen kind of

[Talia Rosen] 13:32:08
what actually probably both of the things that I wrote for translation, be thinking about what is the quality

[Talia Rosen] 13:32:14
control in place for the AI voice and also what is the transparency plan

[Talia Rosen] 13:32:22
And we haven’t

[Talia Rosen] 13:32:24
had this happen yet. So I don’t know what the right answer is on transparency.

[Talia Rosen] 13:32:29
exactly how you would do it, but you’d want to think about

[Talia Rosen] 13:32:33
exactly how you tell people.

[Talia Rosen] 13:32:36
That being said.

[Talia Rosen] 13:32:38
licensing your voice for sort of in perpetuity use

[Talia Rosen] 13:32:43
As a host when you’re not

[Talia Rosen] 13:32:46
employed anymore is just so

[Talia Rosen] 13:32:50
fraught with legal considerations. First and foremost, if you’re part of a guild or a union like

[Talia Rosen] 13:32:56
This is part of what SAG, AFTRA, and all of the unions and guilds are thinking about. This is not something that I work on personally, so I can’t

[Talia Rosen] 13:33:04
speak to it with any specific expertise. But that’s sort of step one to be thinking about. And then step two.

[Talia Rosen] 13:33:10
is thinking about what does the contract say? What can they do with your voice?

[Talia Rosen] 13:33:18
And what is the compensation structure look like for voice cloning? I think that the law has

[Talia Rosen] 13:33:24
very much not caught up in this space.

[Talia Rosen] 13:33:29
whether and how people own their voices. And you see this through the controversy around scarlett johansson

[Talia Rosen] 13:33:35
And OpenAI. And so there’s just a lot to work out here. And so you

[Talia Rosen] 13:33:42
you could do what you’re describing in your question.

[Talia Rosen] 13:33:45
And I think it could be aligned with the editorial standards.

[Talia Rosen] 13:33:49
But…

[Talia Rosen] 13:33:51
I’m not sure.

[Talia Rosen] 13:33:56
I don’t know all of the legal implications of setting up an arrangement like that and sort of

[Talia Rosen] 13:34:03
how long it lasts, what it looks like.

[Chad Davis] 13:34:04
Right. Yeah. This is not to be considered legal advice or counsel. It is.

[Talia Rosen] 13:34:09
Not even a little bit.

[Chad Davis] 13:34:10
Well, it’s a hot take. That’s all. Okay. This is a good question. I had said to Jim.

[Talia Rosen] 13:34:12
There you go. This is my hot tape.

[Talia Rosen] 13:34:14
It’s a great question. Yeah.

[Chad Davis] 13:34:16
show them privately one of the producers at Nebraska Public Media, Kelly Rush, had actually done a sort of a blind taste test where she used a voiceover talent to work in

[Chad Davis] 13:34:26
And my department in labs and then uh did it generate the same voice via 11 labs and it was interesting most people could still tell the difference

[Chad Davis] 13:34:36
between AI and real. Some preferred the AI, but they were able to tell

[Chad Davis] 13:34:42
Which says to me, we’re probably not yet at a place where

[Chad Davis] 13:34:46
reproducing like a voice actor’s voice like will be the same quality

[Chad Davis] 13:34:52
Give it a year or two perhaps.

[Talia Rosen] 13:34:54
No, but what’s interesting to me is that like when people see the Coca-Cola AI ads or they hear their voice and they know a little bit of

[Chad Davis] 13:34:55
Mm-hmm.

[Talia Rosen] 13:35:01
it being off. All the discussion seems to be about

[Talia Rosen] 13:35:05
oh, it’s not good. It’s not there yet with Sora, with any of these tools. And from a standards perspective, like that’s

[Talia Rosen] 13:35:11
That’s irrelevant because it’s going to be moot in six months or nine months right like

[Talia Rosen] 13:35:16
People are still talking about, oh, Midjourney can’t get the hands and fingers right. And it’s like, well, that’s not true anymore. It’s pretty great with hands and fingers.

[Talia Rosen] 13:35:23
So from the standard perspective, it’s it’s perspective

[Talia Rosen] 13:35:26
it’s all going to be there in basically the blink of an eye. So we have to wrestle with

[Talia Rosen] 13:35:31
the fact that the voice

[Talia Rosen] 13:35:35
It’s pretty indistinguishable unless you’re paying close attention. And if it’s not indistinguishable yet, it will be.

[Chad Davis] 13:35:36
It’s…

[Talia Rosen] 13:35:41
And then we have to wrestle with the ethics of it.

[Chad Davis] 13:35:41
Right.

[Chad Davis] 13:35:43
Well, the interesting thing, you did this live and they were in the room, people actually preferred the ai

[Chad Davis] 13:35:50
When I did it for PBS producers, it went the other way but

[Chad Davis] 13:35:55
So it’s fascinating that you can be AI and it can actually, there’s a precedence that people have, even if they know it’s AI.

[Chad Davis] 13:36:02
A couple more questions. Justin writes in, and this is about educational materials, usage of AI tools to generate lesson plans and activities. Justin says, what is the perspective of PBS standards and practices on this topic, especially around

[Chad Davis] 13:36:17
Maintaining the integrity of educational content and ensuring alignment with PBS’s values and standards, the lesson plans and activities.

[Talia Rosen] 13:36:24
Oh, okay. Well, now we’re jumping ahead, I think, to use case number eight

[Talia Rosen] 13:36:28
Which I can definitely do.

[Talia Rosen] 13:36:31
Let’s just do that.

[Talia Rosen] 13:36:32
that’s what people, that’s what the people give that’s what we want to do. We’re going to jump ahead here.

[Talia Rosen] 13:36:37
And this is probably

[Talia Rosen] 13:36:40
Well, let me go through an order, actually, because that is probably one of the most gnarly

[Talia Rosen] 13:36:45
use cases. Let me come back to that question. Let me whip through a few of these if that’s okay, and we’ll get to that pretty quickly.

[Chad Davis] 13:36:47
Okay, yeah, nope. We said…

[Chad Davis] 13:36:51
Cool. There’s two more you want to tackle these real quick? One is actually B-roll releases.

[Talia Rosen] 13:36:57
Oh, great. Okay, yes.

[Chad Davis] 13:36:59
I know. Emily writes, how does the generative AI use balance against

[Chad Davis] 13:37:05
contracting with artists slash photographers to create b-roll images to have any sort of scenery on?

[Talia Rosen] 13:37:12
Yeah, I mean, like as a human, I think that like using artists and photographers is

[Talia Rosen] 13:37:19
Great. And that’s a thing that

[Talia Rosen] 13:37:21
people should do. And I’m a little worried about like the

[Talia Rosen] 13:37:26
any sort of growing reliance on generative AI in places where artists and photographers could reasonably be used.

[Talia Rosen] 13:37:37
Yeah, I think that’s very valid.

[Talia Rosen] 13:37:41
But as a standards person.

[Talia Rosen] 13:37:45
I just have to wrestle with the fact that like, that’s not what’s

[Talia Rosen] 13:37:49
always happening anymore, that there are producers that are

[Talia Rosen] 13:37:54
deciding that they want to use

[Talia Rosen] 13:37:57
These other tools, these synthetic tools instead of

[Talia Rosen] 13:38:01
human artists or photographers. And we have to figure out how we align that with the editorial standards so

[Talia Rosen] 13:38:09
I think it’s entirely within anyone’s right to decide.

[Talia Rosen] 13:38:13
hey, we don’t feel like this is right for what we’re doing. And that makes a ton of sense. I don’t think you’re going to find the answers to that sort of

[Talia Rosen] 13:38:22
philosophical question in the standards

[Talia Rosen] 13:38:27
I think that’s…

[Chad Davis] 13:38:28
That’s like a business decision, really. I mean, you’re balancing philosophy what’s your budget

[Talia Rosen] 13:38:30
Yeah, it’s more of a business position. I wouldn’t say it was a philosophical question, like Aristotle probably has something to say about using a mechanical

[Talia Rosen] 13:38:37
machine instead of a human. But

[Talia Rosen] 13:38:40
It’s a great, I mean, you should you your leadership team at your station or production company could absolutely be having that conversation.

[Talia Rosen] 13:38:48
And then if you decide to proceed.

[Talia Rosen] 13:38:50
then reach out to me to talk about the standards of how you proceed maybe.

[Talia Rosen] 13:38:56
If no one wants to use it, then that makes my life way easier.

[Chad Davis] 13:39:01
Cool. That catches us up. So on you go.

[Talia Rosen] 13:39:03
Oh, great. Okay. So let’s talk about inspiration. I’m going to go whip through three, four, and five pretty quickly so I can get, and seven, so I can get to number eight, which you asked about.

[Talia Rosen] 13:39:13
So inspiration and research. So I have a little green check mark here because I think these are two

[Talia Rosen] 13:39:20
probably pretty great use cases of generative AI. First is inspiration. So here is an image from an American Masters on the right there that a human created. On the left is a generative AI image.

[Talia Rosen] 13:39:34
They were trying to sort of get inspiration for what the home of Augusta Savage might have looked like.

[Talia Rosen] 13:39:42
Based on the information available. So generative AI created that image on the left.

[Talia Rosen] 13:39:47
a human artist created the image on the right.

[Talia Rosen] 13:39:49
I looked at that and said, yeah, that’s materially different

[Talia Rosen] 13:39:53
We’re not going to label that as generative AI. You use it for inspiration.

[Talia Rosen] 13:39:58
And you might use it for inspiration in any modality, right? You might use it for text inspiration when brainstorming topics.

[Talia Rosen] 13:40:05
to talk about. You might use it for visual inspiration.

[Talia Rosen] 13:40:11
And if it’s used that way, then I think that it’s fine.

[Talia Rosen] 13:40:17
Of course.

[Talia Rosen] 13:40:18
there’s going to be close calls where you say you use it for inspiration, but the generative AI output and your ultimate creation are very similar.

[Talia Rosen] 13:40:28
then it becomes trickier.

[Talia Rosen] 13:40:30
But this is a pretty clear difference, I would say. And inspiration is kind of

[Talia Rosen] 13:40:37
one of the best use cases for this technology. Another great use case I heard about on a Public Media Innovators webinar

[Talia Rosen] 13:40:43
a few months ago was when an executive producer from Planet Money talked about using Claude

[Talia Rosen] 13:40:49
to sift through academic papers and to upload dozens and dozens of academic papers about economics.

[Talia Rosen] 13:40:56
and then inquire, interrogate.

[Talia Rosen] 13:40:58
Claude to sort of analyze and distill those. And then when they distilled them, going and reading

[Talia Rosen] 13:41:05
the actual article, the handful of articles that they sort of

[Talia Rosen] 13:41:10
sifted to find through that process. I think that’s a great use case.

[Talia Rosen] 13:41:15
I think that you should be aware that there may be biases in Claude, just like we’ve learned about AI tools used in hiring may have biases. It may decide that authors with

[Talia Rosen] 13:41:27
Male sounding names are just more reliable than ones with female sounding names, and you’re not going to know that.

[Talia Rosen] 13:41:34
So be really careful about using it for research but

[Talia Rosen] 13:41:37
That’s just a hypothetical.

[Talia Rosen] 13:41:40
I do think it’s a really interesting use case. I think that the more you prompt these tools with

[Talia Rosen] 13:41:46
the better the output is going to be. So if you just say, you know, what’s the tallest

[Talia Rosen] 13:41:51
tower of toothpicks, there’s a decent chance it’s going to make something up for you. But if you upload 100 pages of PDFs

[Talia Rosen] 13:41:59
and you ask it to distill things down, will you grab a transcript out of

[Talia Rosen] 13:42:05
YouTube from some TED talk from

[Talia Rosen] 13:42:07
it’s going to do a much better job of giving you some output.

[Talia Rosen] 13:42:10
Now, I said a YouTube TED Talk because I would not encourage you to put in transcripts of PBS programs.

[Talia Rosen] 13:42:16
Because when you put in prompts.

[Talia Rosen] 13:42:20
things as prompts, you’re giving like a perpetual worldwide license to this multi-billion dollar

[Talia Rosen] 13:42:26
to that content and you probably don’t have the

[Talia Rosen] 13:42:29
rights to do that. But inspiration and research.

[Talia Rosen] 13:42:33
great use case. Sort of like Wikipedia, I think.

[Talia Rosen] 13:42:36
Anonymizing interviews. This is a phenomenal use case out of the BBC. You can read about it on the Partnership on AI website that I

[Talia Rosen] 13:42:44
put the URL down there at the bottom. But essentially, this is a very mission-driven use of generative AI to generate faces

[Talia Rosen] 13:42:54
For these people here who were interviewed for a show about alcoholism. And instead of using traditional anonymizing techniques like blurring or shadows.

[Talia Rosen] 13:43:05
They went this route so that the audience could kind of emote with and empathize with

[Talia Rosen] 13:43:11
the interviewees in this show about alcoholism. But there was such robust transparency for this program and such a mission driven purpose.

[Talia Rosen] 13:43:20
So one of the things I ask a lot of producers before we even get

[Talia Rosen] 13:43:25
to

[Talia Rosen] 13:43:28
accuracy, transparency, and inclusiveness

[Talia Rosen] 13:43:30
The first question I often ask is, what’s the mission-driven purpose of this use? Is it advancing

[Talia Rosen] 13:43:36
the mission on page two of the editorial standards to serve the audience to educate people.

[Talia Rosen] 13:43:44
You know, I heard from a producer who was thinking about using something similar for a program in the US about uh

[Talia Rosen] 13:43:51
gun violence in schools. And I thought, yeah, that

[Talia Rosen] 13:43:54
who’s mission driven, right? Maybe using AI in a way that avoids re-traumatizing

[Talia Rosen] 13:44:00
You know.

[Talia Rosen] 13:44:02
victims of violence

[Talia Rosen] 13:44:04
Maybe that’s a use case that’s super mission driven. So I love this example.

[Talia Rosen] 13:44:08
highly recommend people read the BBC case study where they really leaned into the, again, the AI being a feature, not a bug, right? It’s something they were proud of, not something that they were sort of using on the sly.

[Talia Rosen] 13:44:24
Very quickly.

[Talia Rosen] 13:44:26
Inadvertent uses are definitely possible. The PBS show Eons accidentally used a generative AI image in this short that was then taken down and then apologized for publicly.

[Talia Rosen] 13:44:37
Because they got it from a stock image library. It then turned out that it was generative AI.

[Talia Rosen] 13:44:43
And so the stack image libraries are

[Talia Rosen] 13:44:48
wrestling with how to not be polluted.

[Talia Rosen] 13:44:51
And that’s watch this space. I don’t know that we know how or whether that will be resolved. But I just put that here because A, 10 is a nice round number for use cases and B,

[Talia Rosen] 13:45:01
I think we should all know that even if

[Talia Rosen] 13:45:04
We have that senior management discussion within our station and we decide from a philosophical perspective we’re never going to use generative AI.

[Talia Rosen] 13:45:11
we still might accidentally use generative AI. So we should probably be aware of that and maybe have some

[Talia Rosen] 13:45:18
general sense of how we would handle that if it came to pass.

[Talia Rosen] 13:45:22
I think we talked about voice pretty well. Don’t use it to edit your interviewees.

[Talia Rosen] 13:45:27
If you want to use it for a host or narrator.

[Talia Rosen] 13:45:31
I don’t have a huge concern from a standard business practices perspective um but uh

[Talia Rosen] 13:45:37
get some lawyers to think about that. Someone in the chat I noticed asked about tools. The one I hear being used most often by producers is Eleven Labs.

[Talia Rosen] 13:45:46
You see the little icon here of 11 labs. Others are Listener and MRF. And then a free one to just kind of

[Talia Rosen] 13:45:54
see how ridiculously realistic these things have become. Is Notebook LM.

[Talia Rosen] 13:45:59
Again, love to upload a document into there. Even a board game rules document, you could download the board game rules off of the publisher’s website.

[Talia Rosen] 13:46:08
throw them into Notebook LM, get a 10-minute podcast about the game.

[Talia Rosen] 13:46:14
Super fun. And incredibly realistic. I also kind of on a personal note, hope that the creators of Notebook LM get sued into oblivion because it definitely feels like they stole

[Talia Rosen] 13:46:28
all of public radio’s work in creating this tool. So hopefully that happens. But in the meantime.

[Talia Rosen] 13:46:35
it’s a very fascinating tool and man, do they sound real.

[Chad Davis] 13:46:42
Taya, can I ask, this is actually a chat question, believe it or not. What about using AI to clean up audio?

[Talia Rosen] 13:46:44
Okay, good question.

[Chad Davis] 13:46:48
So this is an editing, but actually taking audio and making it broadcast

[Talia Rosen] 13:46:49
Yeah, okay.

[Chad Davis] 13:46:54
You know, perfect, we’ll say.

[Talia Rosen] 13:46:55
Great question. Wonderful question. Yes, absolutely. So this is one of, we get two questions that are basically that. One is using AI tools

[Talia Rosen] 13:47:03
to clean up audio. And the other is to clean up images, to up-res images.

[Talia Rosen] 13:47:09
So upresing, color correction, audio cleanup.

[Talia Rosen] 13:47:14
are all things we’ve been doing for many years. They are now infused with machine learning algorithms.

[Talia Rosen] 13:47:20
I would say that that is not generative AI. And so it’s not governed by any of our

[Talia Rosen] 13:47:26
generative AI guidelines, which are all about creating novel and new material. If we’re cleaning up

[Talia Rosen] 13:47:32
existing material to make it more

[Talia Rosen] 13:47:35
legible, more understandable, more educational to our audience, then I don’t have

[Talia Rosen] 13:47:42
particular standards and practices concerns, but I would go back to

[Talia Rosen] 13:47:48
This provision on page 11 of the standards

[Talia Rosen] 13:47:51
is my use of these machine learning tools

[Talia Rosen] 13:47:55
that are infused with sort of an artificial intelligence

[Talia Rosen] 13:47:59
to clean up audio or to up-res images

[Talia Rosen] 13:48:03
Is it still fairly inaccurately portraying reality? I can certainly envision

[Talia Rosen] 13:48:08
hypotheticals, we’re cleaning up audio to remove crowd noise

[Talia Rosen] 13:48:13
would be misleading.

[Talia Rosen] 13:48:15
maybe you have maybe you have

[Talia Rosen] 13:48:17
footage of a footage of

[Talia Rosen] 13:48:20
of a riot and you want to sort of tone down the background noise from the crowd

[Talia Rosen] 13:48:26
maybe you’re really not

[Talia Rosen] 13:48:28
sharing what really happened with people then.

[Talia Rosen] 13:48:31
or you want to up-res something

[Talia Rosen] 13:48:34
so far that it’s like that you’re really sort of misrepresenting

[Talia Rosen] 13:48:41
what was there or what was possible.

[Talia Rosen] 13:48:46
I think the simple answer is that’s fine. Go forth and clean up.

[Talia Rosen] 13:48:51
the more complicated answer is like, maybe don’t do too much that you’re like really warping things. But in the vast majority of cases

[Talia Rosen] 13:48:59
I wouldn’t consider that a generative AI use, and I would consider that

[Talia Rosen] 13:49:02
probably fine.

[Talia Rosen] 13:49:07
And so I do think it’s worth distinguishing generative AI from

[Talia Rosen] 13:49:11
the fact that the word AI has been applied to everything now as a marketing thing and as a hype gimmick, right? It is sort of two different things from my perspective.

[Talia Rosen] 13:49:22
All right, so we talked about voice translation. I think translation is an awesome use case. I’ve heard about producers saying like, what if my show was in 100 languages? I’m like, yeah, that sounds great.

[Talia Rosen] 13:49:31
That sounds like you’re really actually pursuing our inclusiveness principle and you’re really pursuing our educational mission.

[Talia Rosen] 13:49:40
That being said, let’s think about accuracy. Let’s think about transparency. But I think both of those are very solvable. And so if someone wants to translate a PBS show into 100 languages.

[Talia Rosen] 13:49:51
Assuming the legal and rights issues have been figured out by those folks then

[Talia Rosen] 13:49:58
Yes, absolutely. Let’s do it.

[Talia Rosen] 13:50:01
All right. Text creation and editing. And I put a green check mark here because I do think this is a viable use

[Talia Rosen] 13:50:07
For generative AI, but I also think it’s very thorny. And this gets back to the question that was asked about learning media.

[Talia Rosen] 13:50:15
And I actually just had a meeting with some learning media people a couple of days ago and was talking about a potential guide maybe we’ll put out.

[Talia Rosen] 13:50:21
Because I do think that this is a great use case and also

[Talia Rosen] 13:50:26
tricky.

[Talia Rosen] 13:50:28
And the part that’s tricky is that

[Talia Rosen] 13:50:29
So yes, using Claude, using a gas gemini, if you don’t have access to Claude.

[Talia Rosen] 13:50:37
Using ChatGPT.

[Talia Rosen] 13:50:40
ideate, to brainstorm, to create an outline, to help edit text.

[Talia Rosen] 13:50:46
all seem okay.

[Talia Rosen] 13:50:49
But…

[Talia Rosen] 13:50:51
I would worry and think about plagiarism.

[Talia Rosen] 13:50:54
I think that you’re not going to go to Wikipedia and just

[Talia Rosen] 13:50:58
copy and paste text and plop it into your

[Talia Rosen] 13:51:02
document and publish it and say, look, I made a discussion outline. Look, I made discussion questions.

[Talia Rosen] 13:51:09
look at what I wrote. You’re not going to do that. That’s plagiarism.

[Talia Rosen] 13:51:14
And…

[Talia Rosen] 13:51:16
the world is still coming to grips with what

[Talia Rosen] 13:51:20
whether this is plagiarism, but I think my current

[Talia Rosen] 13:51:23
understanding is if you go to

[Talia Rosen] 13:51:26
Claude and you say

[Talia Rosen] 13:51:29
Here’s a document. Here’s a transcript.

[Talia Rosen] 13:51:31
give me three discussion questions for a classroom and it gives you three discussion questions.

[Talia Rosen] 13:51:36
and you copy and paste those discussion questions.

[Talia Rosen] 13:51:38
into a document that you then publish.

[Talia Rosen] 13:51:41
and you don’t credit it

[Talia Rosen] 13:51:44
I think that’s plagiarism.

[Talia Rosen] 13:51:46
That doesn’t mean you can’t use it, right? Like I’ve used it to ideate, to bounce ideas off of.

[Talia Rosen] 13:51:53
to say, here’s a thing.

[Talia Rosen] 13:51:56
I’m thinking about discussion questions.

[Talia Rosen] 13:51:59
let’s come up with a couple. It comes up with some, they’re probably kind of like b minus

[Talia Rosen] 13:52:05
discussion questions. So like copying and pasting them, like you’re not doing like the best work to begin with

[Talia Rosen] 13:52:11
Maybe then you go back and forth with it. You kind of continue to probe it. Maybe you edit it.

[Talia Rosen] 13:52:17
And, you know.

[Talia Rosen] 13:52:19
Just like with quoting from a website or a book.

[Talia Rosen] 13:52:23
there isn’t a bright line for like how much paraphrasing makes it your own words.

[Talia Rosen] 13:52:28
I think there’s not a bright line here.

[Talia Rosen] 13:52:32
If you’ve gotten your inspiration and your research in part from this tool with lots of fact checking.

[Talia Rosen] 13:52:38
outside of this tool and you’ve put it into your own words

[Talia Rosen] 13:52:42
then I think that’s fine. I think that’s a great use case. I would also go and read Ethan Moloch’s latest substack

[Talia Rosen] 13:52:51
on one useful thing, which I’ve linked here, where he talked about

[Talia Rosen] 13:52:55
15 uses for AI and five uses to avoid.

[Talia Rosen] 13:52:59
I thought the five years to avoid were pretty

[Talia Rosen] 13:53:04
instructive and really interesting. Ethan did, I think, a public media innovators webinar

[Talia Rosen] 13:53:10
Maybe a year ago?

[Talia Rosen] 13:53:12
Really interesting speaker and thinker on all of this and also

[Talia Rosen] 13:53:17
You may recall had some really interesting board games on his shelf behind him when he gave that webinar, including Dixit and Mice and Mystics. But anyway.

[Talia Rosen] 13:53:25
So the five uses to avoid, which you’re all going to go and read one useful thing after this, but three of them. One was you avoid using it if high accuracy is required.

[Talia Rosen] 13:53:34
Well, a lot of our work, high accuracy is required like

[Talia Rosen] 13:53:39
guides, discussion guides, right? Avoid using it if you’re trying to synthesize new information to you, information that you don’t have a deep familiarity with.

[Talia Rosen] 13:53:48
And if you’re going to use it to synthesize new information, you’re going to need to do a lot of sort of

[Talia Rosen] 13:53:53
double checking. And then his third use to avoid, which I thought was

[Talia Rosen] 13:53:57
really on point was if on point

[Talia Rosen] 13:54:00
the failure modes for your use are unclear. Essentially, you don’t know enough

[Talia Rosen] 13:54:05
to know if it’s failed. If you’re so new to the subject that

[Talia Rosen] 13:54:12
I hate the word hallucinate. It may…

[Talia Rosen] 13:54:12
it may…

[Talia Rosen] 13:54:16
create tokens that are untethered from reality

[Talia Rosen] 13:54:21
then um

[Talia Rosen] 13:54:24
then it’s dangerous to be using it if the failure is not something that’s going to be clear to you.

[Talia Rosen] 13:54:30
Common Sense Media has created this

[Talia Rosen] 13:54:33
course in partnership with OpenAI about the use of ChatGPT. And while it has its flaws, I do think it’s a good grounding point both for students and educators, but also for all of us.

[Talia Rosen] 13:54:44
to be thinking about ways to use

[Talia Rosen] 13:54:49
text tools that are responsible and that

[Talia Rosen] 13:54:53
or err on the side of using them

[Talia Rosen] 13:54:56
not to just copy and paste but to

[Talia Rosen] 13:54:59
help us sort of avoid the blank page problem and to help us come up with discussion questions, but not

[Talia Rosen] 13:55:07
Just cut and paste. I hope that helps. I think we’re all still figuring this out.

[Talia Rosen] 13:55:12
I guess, oh, yes, I have my fun little image down here in the little red box. And this is from the Nebraska Public Media Policy.

[Talia Rosen] 13:55:18
Human first, human last.

[Talia Rosen] 13:55:21
the robot is in the middle, right? It’s the pickle on the sandwich, as I’ve heard.

[Talia Rosen] 13:55:26
Chad and his colleagues say. So if you’re using it that way.

[Talia Rosen] 13:55:30
then I think you’re on the right path.

[Talia Rosen] 13:55:34
Okay, we just have two more

[Talia Rosen] 13:55:36
Oh, God, we only have five minutes. Oh, no. Okay.

[Talia Rosen] 13:55:40
Wow. Marketing.

[Talia Rosen] 13:55:43
I’ve talked to the marketing folks here at PBS and they say, they literally said to me, there’s a quote, approach it like the work of an intern.

[Talia Rosen] 13:55:50
So your marketing professionals out there, yeah, you should be playing with generative AI. Yeah, you should be thinking about using it.

[Talia Rosen] 13:55:58
But where it’s at right now, do approach it like the work of an intern. Maybe it can compile a sizzle wheel if you have the rights to give it the full show and ask it to

[Talia Rosen] 13:56:07
pull clips to make your 30 or your 60. Maybe it can do that someday.

[Talia Rosen] 13:56:11
Maybe it can help you drafting audience engagement, but I do think we have to still think about those same three core principles.

[Talia Rosen] 13:56:20
And how accuracy, transparency, and inclusiveness are going to apply here. And if you have a marketing use case and you want to use it for sizzle wheels or social media posts or audience engagement or newsletters.

[Talia Rosen] 13:56:31
I’d love to talk about it. I think I saw something in the question in the chat, which was like, how do we do that?

[Talia Rosen] 13:56:38
Just email. Yeah, email standards at pbs.org. We’ll schedule time to talk. I can talk

[Talia Rosen] 13:56:43
next week. The way I learn about these things is by hearing about how you

[Talia Rosen] 13:56:48
the creative people of the world are

[Talia Rosen] 13:56:50
actually thinking about using it so like that’s the way that I can kind of

[Talia Rosen] 13:56:55
come up with the framework to try to

[Talia Rosen] 13:56:57
fairly inconsistently develop what we do here but again

[Talia Rosen] 13:57:02
The human is on both ends of the creation of this marketing content.

[Talia Rosen] 13:57:06
And then lastly, video and music.

[Talia Rosen] 13:57:10
It used to say that this was the next frontier, but as of Monday, with Sora.

[Talia Rosen] 13:57:14
It’s here, it’s now.

[Talia Rosen] 13:57:17
If you know nothing about Sora, I would recommend going to mkbhd’s video

[Talia Rosen] 13:57:24
From Monday, KBHD, if you don’t know is like

[Talia Rosen] 13:57:28
one of the most one of the biggest tech reviewers on youtube

[Talia Rosen] 13:57:32
And I think he does a really good job of being level-headed about tech. So I have the YouTube link there, which you’re not going to be able to copy down, but if you just search MKBHD SORA, you should get this 16 minute video that in 16 minutes gives you kind of the good, the bad and the ugly.

[Talia Rosen] 13:57:48
of Sora. But this is such a new frontier that like there are no standards yet around video and music. I think it’s such a legal morass.

[Talia Rosen] 13:57:56
that I probably don’t need to solve for the standards issues quite yet.

[Talia Rosen] 13:58:01
I don’t know that SUNO and UDR are going to exist in a year.

[Talia Rosen] 13:58:05
Given the litigation they faced and

[Talia Rosen] 13:58:09
I expect Sora will be sued by everyone you can imagine, including the video games industry, I think is not

[Talia Rosen] 13:58:16
happy if what I’m reading today is right.

[Talia Rosen] 13:58:18
My key takeaways, the standards

[Talia Rosen] 13:58:22
provide guidance.

[Talia Rosen] 13:58:24
They can’t anticipate the nuanced nature of all of the possible issues. Let’s all just focus on ensuring like our truth, our credibility and our trustworthiness.

[Talia Rosen] 13:58:31
There are a handful of issues that are clear-cut like

[Talia Rosen] 13:58:35
Generative fill is very worrisome.

[Talia Rosen] 13:58:37
using voice cloning to edit what your JVUE says is very worrisome. But 99% of the time.

[Talia Rosen] 13:58:44
The answer is going to be it depends. And thinking about the specific context is going to be essential.

[Talia Rosen] 13:58:49
And you’re not alone. I really want to create a culture of dialogue around these issues.

[Talia Rosen] 13:58:54
please don’t hesitate to reach out standards at pbs.org. And if those are too many key takeaways, I think remember

[Talia Rosen] 13:59:01
Accuracy, transparency.

[Talia Rosen] 13:59:04
inclusiveness, or maybe just that the robot

[Talia Rosen] 13:59:08
is the pickle on the sandwich.

[Talia Rosen] 13:59:12
I also go back to our mission all the time. This is right at the beginning of the standards.

[Talia Rosen] 13:59:17
I’m not going to read it out, but

[Talia Rosen] 13:59:20
I reread this whenever I’m sort of wrestling with one of these thorny things.

[Talia Rosen] 13:59:25
Okay, sorry to rush through that last part and I will stay on late if anyone wants to talk more about these things.

[Chad Davis] 13:59:38
I actually think we caught up with, I mean, we did it as we went and

[Talia Rosen] 13:59:38
All right.

[Talia Rosen] 13:59:42
Really?

[Chad Davis] 13:59:43
I don’t see anything lingering, actually.

[Chad Davis] 13:59:47
If you missed it, oh wait, something just came in. Robot equals pickups.

[Talia Rosen] 13:59:49
Wow. Okay. I’m going to stop the share so i can

[Talia Rosen] 13:59:51
I want to see people. I’m going to stop the share.

[Chad Davis] 13:59:53
Got it.

[Chad Davis] 13:59:55
So yeah, we did put some, for those who are scrolling through still, we put a couple of Sora videos up.

[Talia Rosen] 13:59:55
Stop the share.

[Chad Davis] 14:00:03
Earlier in the chat, if you just want to get a quick five second glimpse of what that looks like, it’ll open it up.

[Chad Davis] 14:00:09
in a player window for you, independent of sense.

[Chad Davis] 14:00:14
That means I will go to maze again.

[Talia Rosen] 14:00:19
Wait, what?

[Talia Rosen] 14:00:21
What happening?

[Linda Wei] 14:00:24
Well, I’m going to close out because we’re at the top.

[Chad Davis] 14:00:26
That was a really bad toss to

[Talia Rosen] 14:00:28
Okay. All right.

[Linda Wei] 14:00:31
We’ll close things out. Thank you so much. This was such a jam-packed hour of information. Of course, we know things are changing by the minute at this point.

[Linda Wei] 14:00:39
Thanks everyone for attending. Chad, you did an excellent job on curating all the questions. Recordings of this webinar. I know everyone’s interested in that. They will be posted on

[Linda Wei] 14:00:49
Public Media Learns and the public media innovators.

[Linda Wei] 14:00:53
section, but also the ECM member space as well. So thanks for being here and Talia, of course, for your knowledge and your thoughts on

[Linda Wei] 14:01:02
how this is evolving. So thank you so much.

[Talia Rosen] 14:01:05
Thank you.