Creating structure with generative AI

Once upon a time there was ChatGPT, and people had a lot of fun with it. You could use it to write poems, formal emails, or a pitch for a Bravo reality show about the bats that live underneath Congress Avenue Bridge.

What couldn’t generative AI do? Write this whole presentation for me at the last minute with a Poe bot. (Kidding.) Create an inspiring pop rock ballad about Rosental with Suno. (Find me later, and I'll play it for you.) It can be a blast.

A fundamental misunderstanding

But a year later, it seems clear that introducing the world to generative AI through parlor tricks created some distorted impressions of what the technology is good for—or, at least, what it's best at. How many of you have stared at a powerful LLM tool like this one, the cursor in an empty text box blinking menacingly as you wonder, good question, how can you help me today?

You’re not alone. AI apps, though popular, suffer from lots of users who try a parlor trick—write the first page of my novel, sure—and then never return.

Just 14% of monthly users of AI apps are daily users, according to one analysis, compared to 51% for popular apps like games and social-media platforms.

I’d like to think that’s due, in part, to a fundamental misunderstanding. I mean, generation, it’s right there in the name, so I get the confusion. But generative AI’s most powerful use is not in creating entirely new text or images, it’s creating structure out of messy data that already exist.

The common element

This is building on a point I made briefly at the end of my last public talk, here in Austin at SXSW last month, where I shared some examples of journalists using generative AI to create structure out of unstructured prose:

• The Marshall Project’s banned-books database that Andrew Rodriguez Calderón will talk about in more detail shortly;
• a custom GPT by Jaemark Tordecilla that parses complex government audit reports to help expose corruption in the Philippines; 
• and a product recommendation site generated by intelligently mining links in the archives of the newsletter “Why is this interesting?”

The common element in each of those generative-AI projects is that they aren't really generating something new. They're creating summaries, extracting information, and structuring data in a more usable form.

Today I’d like to build on that argument with a few other examples.

Structuring the city council

On March 11, New York City’s committee on general welfare held a nearly six-hour meeting at which several important issues we’re discussed: funding cuts for homeless shelters and rental-assistance programs, backlogs in applications for food stamps, the city's response to migrants seeking asylum, etc. 

But if you went to the city council's website to find a record of the meeting, you wouldn’t know any of that. You’d be starting at a page like this one, wondering if this really how the nation’s largest city operates in 2024.

Vikram Oberoi, an engineer in New York with an interest in local government, thought there had to be a better way.

The city council does publish transcripts and video recordings of its meetings, if you can find them. Oberoi wrote detailed prompts for GPT-4 to break up the transcript into timestamped question-and-answer pairs he called chapters, give each chapter a helpful title, and summarize the salient points of the discussion.

And so Oberoi had all the structured data he needed to create a far more accessible version of the same meeting: navigable by chapter, with helpful summaries, and the video synced to the transcript.

Not just for that meeting of the committee on general welfare, but every city council meeting since he started this project a few months ago. What a wonderful contribution to civic engagement and local journalism.

Structuring the mayor

Give it some thought, and you will find plenty of ways to use this approach to your advantage as a journalist.

A colleague of mine at The Times covering immigration was frustrated by New York’s habit of releasing important statistics ad hoc in the middle of City Hall press conferences. So he gave the transcripts to an LLM and asked it to spit out all the statistics. It gave him a kind of Harper’s Index of each press conference.

That’s a really simple prompt anyone here could employ without a lick of code. And, yes, you should fact-check these data if you plan to publish them, but in my experience, LLMs are very good at this kind of task and far less likely to invent information than when generating text out of whole cloth. Another reason creating structure is the best LLM use case.

AI hews toward structure

Real life is messy. Journalism, at its best, helps people make sense of that mess. Using AI to give structure to messy data is a pure form of journalism.

The neural networks underlying large language models can seem like enigmas.

What you’re looking at on screen is not just what you might see after wild a night on Sixth Street, it’s a visualization of training a neural network, which even amid mysterious calculations over vast arrays of data, produces gorgeous fractals. AI hews toward structure, it loves structure. It’s not random.

Structuring your receipts

One last example. I’ve been talking mostly about creating structure out of text, but there’s just as much potential in creating structure from images. 

A lot of you are traveling here from out of town at your employer’s expense, which means lots of receipts like this one and a painful data-entry process when you get home. Well, next time, try feeding your receipts to your favorite multimodal LLM and asking for the data you need.

The most beautiful kind of structure out of chaos. Thank you.