8. LLMs to Model Agents & Interactions - fundamental

[lkcao]

Post questions here for this week's fundamental readings: Cornell Conversational Analysis Toolkit (ConvoKit) Documentation: Introductory Tutorial; Core Concepts; the Corpus Model.

OpenAI. 2019. “Better Language Models and Their Implications”. Blogpost.

[XiaotongCui]

How does GPT-3 achieve a balance between creativity and factual accuracy in its text generation? Given that the model may sometimes prioritize safety over novelty, resulting in content that lacks innovation, what is OpenAI's perspective on this trade-off and what is the technical detail to train them?

[sborislo]

In the blog post, they mention not releasing the source code, but do not aim to develop tools for determining AI generation, presumably because AI-generated text is harder to differentiate from non-AI ones. With the creation of SORA, OpenAI takes a different approach, developing a method for assessing whether the generated video is AI-generated or not. However, I doubt this would help when the video itself can be detrimental, even when it's known to be fake.

In what ways can OpenAI continue to ensure safety when the tool itself becomes more powerful? At some point, does AI generation not have to throw ethicality to the wind to ensure improved performance?

(Also, for the other readings, how does the conversational analysis tool account for contextual factors, like a person in a conversation changing the topic because they didn't hear the other person?)

[yuzhouw313]

The ConvoKit seems to be very effective in collecting, processing, and analyzing conversational data based on the code demonstration using corpus components of conversations, utterances, and speakers. The Transformer was introduced to conduct preprocessing, feature extraction, and analysis, but I am curious to know its pros and cons compare to other machine learning model or recent LLMs like BERT. Furthermore, I wonder how well its performance will be when apply ConvoKit to less clearly structured corpus which has no distinct conversational flow.

[Audacity88]

It’s interesting to read this paper from 2019, well before the GPT boom took off with GPT3 in 2020. It seems that OpenAI realized their technology could have world-changing effects, and was quite proactive in anticipating the risks. I wonder why GPT3 achieved virality where GPT2 had not? Is it because the high failure rates (~50%) of GPT2 noted in the post puncture the “suspension of disbelief” that makes it feel like you are talking to a human?

[joylin0209]

In the face of concerns about AI-generated content, how can we ensure that digital literacy includes the ability to distinguish between human and AI-generated content?

[YucanLei]

The GPT's creativity was something I was trying to exploit when I was producing my presentation. I was trying to convince GPT to say something that is absurdly wrong, but I failed. However, the blog post again brought this idea up, where GPT can be used to produce misleading and potentially harmful information. How are we suppose to stop this from happening? Is there any measures we can take to prevent anyone from taking advantage of LLMs to produce false information on a mass scale?

Well I guess not, there are already people using AI drawings to produce defaming pictures of celebrities.

[bucketteOfIvy]

OpenAI's 2019 blogpost stresses their fears over the ability of LLMs to be misused to cause social harm. This harm is highly salient in online spaces, especially on social media websites such as Twitter which can have an absurd number of bots. However, pairing this reading with Park et al. (2022) raises a potential option: can we intentionally (mis)use LLMs within simulated online societies to better understand the myriad of ways in which they can derail social media, and use that understanding to design social networks that are more robust in an environment with easy access to powerful language models?

[QIXIN-LIN]

Given ChatGPT's immense popularity in 2023, reviewing a paper from 2019 that expressed concerns about generative AI strikes me as intriguing. It raises the question: was OpenAI aware of the monumental impact their creation would have? Furthermore, I'm curious if they have taken any measures in light of the potential risks.

[icarlous]

Confronted with apprehensions surrounding AI-generated content, how can we guarantee the integration of a robust digital literacy framework that empowers individuals to adeptly differentiate between content created by humans and that generated by artificial intelligence?

[ethanjkoz]

It is extremely interesting to read a paper by OpenAI before the explosion in their popularity due to GPT-3. It is relieving to see the company was forward thinking enough to understand that their models had the potential to have huge policy and even societal implications with further tuning. I am now curious to see how/if OpenAI's stances and tactics (only releasing a small portion of GPT-2 due to concerns of malefactors). Furthermore, though OpenAi clearly has good reasons to not disclose their full training sets for their models, what are the cons of doing so in terms of research transparency. I understand that OpenAi is not a public company and as a business has a right to keep its models and techniques proprietary, but I wonder how much the scientific research community could benefit from more transparency about these models on all fronts?

[donatellafelice]

i would like to know how the convokit treats stemming/lemmitizing and/or tokenizing and what words it removes. how does it treat stop words? as many words that are often removed are important parts of conversation

also I thought this was very interesting in regard to ethan and other's comments: https://defensescoop.com/2024/02/20/scale-ai-pentagon-testing-evaluating-large-language-models/ what sort of ramifications do these kind of contracts with private companies have for Ai governance and oversight?

[erikaz1]

OpenAI has made an effort to publicize both the risks and benefits of GPT, as reflected in this article. I'm surprised that there is no mention of users' privacy as they engage with GPT (2), given that chat history is stored and may be shared with certain external parties. In 2023, OpenAI moved to an opt-out process for deleting chat history and claim to have made their privacy policy more accessible.

[alejandrosarria0296]

It is extremely interesting to see how predictions about the potential harm of LLMs such as GPT-2 have proven to be extremely accurate and, in some cases, tame. Given the ever increasing role that these tools play in society, what would be some concrete ways of regulating the negative consequences of this expansion. Besides calls for "skecpticism", like the footnote on the blog suggest, what are some concrete actions that could be taken?

[naivetoad]

The paper demonstrates a clear correlation between model size and performance across various tasks. However, it also mentions that even their largest model underfits the WebText data. What are the theoretical limits of scaling up model sizes in terms of performance gains?

[anzhichen1999]

The progress from GPT2 to 4.0 is fascinating and has changed our way of perceiving and researching the world. How might the integration of real-time emotional analysis AI improve the accuracy of social simulacra in predicting complex social dynamics and user interactions in virtual environments?

[chenyt16]

From the blog post, it seems that the goal of GPT is to narrow the gap between AI-generated content and human-generated content. However, AI models blur the differences between individuals. It was mentioned in previous classes that in the future, it may be possible to provide AI models with some keywords to mimic the behavioral characteristics of a certain type of people. What I am curious about is whether, without providing these limiting words, there exists an inherent, intrinsic "traits of character" in the responses given by AI model.

[runlinw0525]

The OpenAI Blogpost raises concerns about the potential misuse of large language models like GPT-2. What measures can be taken to ensure their responsible development and deployment while still harnessing their benefits for society?

[Vindmn1234]

How do we ensure the generated social interactions by social simulacra accurately reflect the diversity and complexity of real-world social behaviors, considering the inherent biases and limitations of large language models?

[muhua-h]

Given that GPT-2 was trained on a diverse internet dataset to predict the next word in sentences, how does this training methodology impact the model's ability to distinguish between factually accurate information and fictional or biased content, considering the vast variability and reliability of online sources?

Also, the decision not to release the full GPT-2 model due to ethical concerns marks a significant moment in AI development. But if it is not released, we don't know what is wrong. How could this do any help?

[Twilight233333]

In the GPT blog, the author talked about world modeling failures (e.g., the model sometimes writes about fires happening underwater); I am curious if there is an actual world modeling database or function that GPT will use to improve its learning. How could we do that, like word embedding?

[Caojie2001]

This article reminds me of a previous article in which the initiative to construct open-source large language models with the support of the public sector was proposed. Despite the 'open' in its name, OpenAI's unwillingness to publicize the algorithm and training dataset of GPT has undoubtedly made it more difficult to resolve GPT-related issues, such as texts based on false information. I wonder whether there have been recent efforts made to establish open-source LLMs or normative guides for LLM construction and utilization.

[HamsterradYC]

In light of the article's discussion on the future applications and risks, could we leverage extensive datasets on human behavior and decision-making to train models capable of simulating intricate network behaviors across diverse urban community groups, even capturing spatial behavior patterns?

[volt-1]

How does OpenAI adapt large language models, which are fundamentally sentence completers typically starting and ending with periods, to human conversational format where a dialogue starts and ends with question marks, and ensure the response is coherent and from a first-person perspective?

[Marugannwg]

It feels like the cleaning and tinkering of the model is extremely tedious and costly for LLMs. A look of unexpected outputs are mentioned in the blog, and I wonder how to even resolve all those issues.. On the one hand, how much effort need on data preprocessing to avoid GIGO problem; on the other hand, how to make sure that even with decent data, the model doesn't generative something beyond the expectations?

[yueqil2]

The technical paper written in 2019 reveals that the development of GPT does not seem to happen overnight, but why GPT-2 has not received as much attention as GPT-3.5 or even GPT-4? In the long road of technological iterations and upgrades, how do humans judge the turning point that changes history?

[cty20010831]

The development of OpenAI products over the past few years have been amazing. Looking back on this paper, I am wondering what are the limitations of unsupervised multitask learning in language models and how might they be addressed (or have they been addressed by later language models)?

[ana-yurt]

I wonder if there is any possible tendency for the text generation quality of LLM models to decay over time (due to, for example, the increasing prevalence of machine-generated content on the internet, or the amount of man-made disinformation, etc).

[floriatea]

How will future language models be designed to allow for greater customization and personalization without compromising the integrity of the information they generate, remaining unbiased and factually accurate?

[beilrz]

I was wondering what is the long term landscape of closed source LLM, such as GPTs, and open source LLM? it seems to me, while open source LLM does offer advantages, such as less internal safety constrain or cheaper price, it lacks human feedback and reinforce learning for following instruction, which cause the output to be less structured.

[Brian-W00]

Is it possible for LLMs to have reasoning capabilities? It is not the basic reasoning capabilities, like 1 + 1 -> 2, but the ability to base one's knowledge on existing knowledge and infer something new.

[Carolineyx]

With GPT-4's enhanced abilities to generate human-like text and its potential applications, what are the main ethical considerations and potential risks associated with its deployment in various fields, and how is OpenAI addressing these issues to prevent misuse while promoting beneficial uses of this technology?

[JessicaCaishanghai]

This is very interesting topic. Considering GPT-2's ability to generate high-quality, conditional synthetic text and its performance on various language tasks through unsupervised learning, how does the model's reliance on a diverse internet-sourced dataset influence its ability to understand and generate text across different domains without domain-specific training data?