Tech Talk Submission

Thanks for offering to give a talk at a Tech Talks meeting! We just need a bit of information from you.

Your Name

James Hochadel

What's your talk title?

LLM no. 1: What does a GPT know? Exploring Generative Pretrained Transformers and the knowledge they may encode

What's your talk about?

Large Language Models like GPT-3 are making headlines for their uncanny ability to chat like humans do. Companies and venture capital firms are betting billions that machine learning is the foundational technology of the future. And the millions of people trying out AI products are wondering: Are these things alive? How do they sound so real? How much better do they have to get before I am obsolete?

LLMs are near the cutting edge of science and engineering and our understanding of them grows by the week. Nobody yet knows the answers for sure, but we can constrain the set of possible answers by careful study of the models' architectures. In this talk, I will walk through the Transformer architecture at a technical level, while assuming little to no audience familiarity with deep neural networks. The explanation will lay the groundwork for a discussion about whether a chat bot like ChatGPT have a "true world model", or "just surface statistics" — in other words, what does a GPT know?

If the talk is well-received, I hope it will be the first in a series of three. My current ideas for future installments: "How Intelligent is a GPT?" and "Do GPTs feel? Intelligence vs Consciousness in Machine Learning Models (and us)".

How long is your talk?

[ ] Lightning talk (5-10 minutes)
[ ] Short tech talk (20-25 minutes)
[x] Long tech talk (40-50 minutes)

Do you have any preferred dates for it?

May 9, 2023 if possible; May 16 or June 6 if not

Todo for the MC:

[ ] Update the TTS Guilds calendar entry for this talk to add the talk details.
[ ] Request captioning for the event date
[ ] Announce the talk(s) in #tech-talks, #dev, #18f, #18f-dev-announce when date is set
[ ] Announce the talk(s) in same channels on Slack in the morning, and follow up with a reminder just before they're about to begin.
[ ] Upload video and transcript to Google Drive.

18F / tech-talks

Tech Talk: LLM #1: What does a GPT know? #75