6. Large Language Models (LLMs) to Predict and Simulate Language - [E4] van Dis, Eva A. M., Johan Bollen, Willem Zuidema, Robert van Rooij & Claudi L. Bockting.

[lkcao]

Post questions here for this week's exemplary readings:

4. van Dis, Eva A. M., Johan Bollen, Willem Zuidema, Robert van Rooij & Claudi L. Bockting. 2023. “ChatGPT: five priorities for research: Conversational AI is a game-changer for science. Here’s how to respond.” Nature.

[XiaotongCui]

Concerning the "human verification" aspect of this article, I appreciate the authors' intentions, but I find the implementation challenging. For research works, especially cutting-edge ones, human verifiers would need extensive expertise and time. Even with these qualifications, it might still be challenging to conclusively determine whether LLM models were employed in generating the text.

[sborislo]

I think the ethical and legal concerns raised by the authors are sensible; however, are the concerns about scientific progress not misguided? As the authors acknowledge, these LLMs are merely drawing on statistical associations between words, so most researchers know not to use them for drawing citations, generating data, and so on (in contrast to what the authors state). And as the authors also note, LLMs can potentially help with creativity. To my knowledge, LLMs are primarily used for generating text in manuscripts that is the "filler" for reporting of the data, which shouldn't affect scientific progress except by making it more efficient. So are LLMs not just generally helpful, at least to an extent where engaging in the nearly impossible task of detecting AI-generated content is not merited?

(I'm more concerned with LLMs increasing prevalence of anti-science and confirmation bias among laypeople).

[chenyt16]

As the author mentioned in this article, ChatGPT's current training set lacks academic papers and books. If ChatGPT were trained on academic articles and books as well, would it significantly improve the bias and errors in research-related issues? However, at that point, it would be more difficult for people to detect AI-generated content. When LLMs can assist in generating research ideas, questions, and hypotheses, it will accelerate the publication of academic achievements, but at the same time, academic research will turn into a race against time. We need to think carefully about how to face this challenge.

[cty20010831]

This short paper is a very stimulating paper and also a paper I am looking for with the increase of using ChatGPT in my own daily life. I strongly agree with the point that ChatGPT can oftentimes fabricate error-prone, yet seemingly convincing, responses, which therefore requires users to pay extra attention.

Building on this paper, I have a question regarding the generation of hypotheses (and also potentially research designs) using ChatGPT. After trying for a few times, I found that they have the same issue of looking seemingly convincing (due to their effective use of language) yet lacking insightful "cores" underlying the surface. Hence, I am wondering what would be the ways to improve how ChatGPT generate insightful/original scientific hypotheses? Also, how can people leverage this hypothetical improvement and what are the ethical implications behind?

[YucanLei]

It is true the paper had raised some concerns about the ethicality and the legality, however, at some point when we dig too deep into this, we would realize the issue gets almost philosophical, like the ship of Thesues. As the author suggested, the paper authored by the AI and then polished by the people and authored by the people then polished by the AI, it is difficult to say. In the meantime, although I do agree that prompting can be a valuable technical skill, it will not be for a long time. The paper agrees the AI could get more developed in the future where ti would be difficult to tell the difference between people and AI, then who is to say that AI will not be good enough to build more with less sophiticated prompts?

In the meantime, the yesman nature of GPT means it will probably be always easy for people to spread false information with GPT. Even if we add more academic sources to train GPT and don't allow people to doubt these academically proven points, then what? Knowledge is constantly evolving. People should always be allowed the rights to question and to doubt. So what should we do here?

[alejandrosarria0296]

One of the main issues that come with a more widespread and uncritical adoption of LLMs in both industry and academia is related to the biases that intrensically exists in the models based on them being trained on biased data. In the context of academic research there is a lot of talk about being up front with these biases and reconizing them. However, few concrete actions are taken to integrate this bias identification into research pipelines. What protocols do you think could be feasibly implemented that would prevent LLM bias from deeping preexisting human biases in CSS research?

[yunfeiavawang]

This paper reminds me of a previous discussion with my friends. We were thinking about using ChatGPT for annotation in ML applications. As students, the possibility is that our article would be rejected if we do so. However, we anticipate that someday a much more renowned scholar will take the initiative and become the first one to apply GPT to data annotation. Then we can use his/her streamline as a reference. I am curious about how others think about the feasibility of the prospect.

[yueqil2]

GPT helps me a lot but I never know that "Information that researchers reveal to ChatGPT and other LLMs might be incorporated into the model, which the chatbot could serve up to others with no acknowledgement of the original source"...Does it mean everyone actually using the same LLM while using GPT? But I used to hear that I could train my GPT with the specific information I feed to it. What's the logic here between GPT and the user?

[Caojie2001]

I appreciate the democratic approach to dealing with LLM-related issues the authors proposed in the article, especially the initiative to construct open-source large language models with the support of the public sector. Considering the copyright controversy surrounding the existing LLMs (both on the construction end and on the application end), do you think that this procedure of democratization would be a feasible solution?

[runlinw0525]

In my opinion, I believe that large language models such as GPT-3.5 and GPT-4 are generally beneficial as they are accessible to everyone through the well-known and free application named Chat-GPT. However, the problem lies in their regulation. Therefore, I think there is an urgent need to develop firm policies to regulate these generative AI tools. I wonder what the scope of such policies would be? Would they be at a national level, state level, school level, or class level?

[naivetoad]

What mechanisms can be implemented to ensure that LLMs do not exacerbate existing biases or introduce new ones in scientific research?

[ethanjkoz]

I agree with the authors' point that the debate around LLMs should not devolve into an arms race between evolving chatbots and its detectors. Furthermore, I agree with their point that these chatbots should not author papers because of their inability to assume accountability. Overall, the paper maintains a positive outlook for science and its acceptance of LLMs, but could this be viewed as a bit too optimistic? Though technology often is touted as a potential equalizer for all, this rarely happens. What specific measures ought we take so that we do not further perpetuate preexisting inequalities? In a more specific example, the authors mention there are potential issues with copyright law as to who owns the generative images. In my mind, the big tech companies and their lawyers have a much better chance to coerce the law in their favor than small artists who don't even realize their art is being used to train LLMs.

[Vindmn1234]

The article mentioned the concerns about the transparency of ChatGPT's training process, including the specifics of its training dataset and the nature of the reinforcement learning from human feedback (RLHF) mechanism, touch on broader issues in AI development and governance. The absence of specific legislation regarding the transparency of AI training processes reflects the nascent state of AI governance. One one hand, the training data, model architecture, and training methodologies constitute valuable intellectual property that gives them a competitive edge in the tech industry. One the other hand, the accuracy of LLMs like ChatGPT can vary, potentially introducing inaccuracies, biases, and plagiarism into specialized research. So, I'm curious how would the policymakers make legislations to actually balanced the intellectual property protection & user privacy while ensuring ethical use and preventing harm.

[Marugannwg]

[Looks like everyone is going for this article directly 0v0]

Regarding accountability, I always believe the person using the GPT should not delegate the responsibility to GPT. I understand with this analogy: Consider hiring a college student to do some task (e.g., paper writing, brainstorming) for you; it is your job to check the work they accomplished and use it wisely, especially when you clearly know that students cannot verify the information by themselves.

I'm actually very fond of the discussion around a truly open LLM. Today, the best model is in the hands of a large company, a lot of concerns arise when we have to pass our information to a third, centralized corporation and rely on it. I'm literally concerned when large models monopolize the industry and light-weight/personalized/transparent models don't have a ground to get researched and developed. [From a chaotic evil perspective, I believe we need some serious legal cases that hinder certain development directions and push the researcher to alternative technologies.]

[muhua-h]

Given the potential for LLMs to transform the scientific research landscape, what are the broader societal implications if these tools become central to knowledge creation and dissemination, particularly in terms of access to and the quality of information?

[QIXIN-LIN]

Reflecting on the issue and importance of human verification, I'm curious about the feasibility of employing another Large Language Model (LLM) for the task of verification, provided that it's given precise instructions to do so. Is such an approach viable, or is this type of verification fundamentally a task that requires human involvement?

[HamsterradYC]

The discussion around the assistance provided by large models, such as GPT, has been widespread, with many finding that a lot of tasks can be completed by GPT without the need for a comprehensive system of learning. However, the one thing tools like GPT cannot do is assume responsibility. For instance, while they can help design blueprints, they cannot take on the responsibility for engineering outcomes; in academic writing, they cannot bear the accountability for the content produced ; and in data analysis and accounting, they similarly cannot be held accountable. Ultimately, all responsibilities must be borne by humans. This raises the question: in the future, is it possible that humans as the subject of responsibility will merely serve as bearers of responsibility.Can LLM etc. take responsibility?

[Dededon]

I quite recognized the hallucination issue of using ChatGPT based information retrieval tasks. It performs better with easier tasks with clear definitions of terms that involve less priori knowledge for ChatGPT to understand. My question is like, what is the best prompt engineering practice in using ChatGPT to finish more complexed NER tasks?

[ana-yurt]

While the use of conversational AIs are poor candidates for accountability for scientific practice, I wonder if LLMs specifically designed to aid in research attempts (such as generate research questions, conduct literature search, etc) may be different?

[erikaz1]

van Dis et al. (2023) is a call to action to discuss how LLMs can be used in a way that maximizes benefits and minimizes risks. The authors seem to err on the side of caution, for instance, writing that “scientific journals should be transparent about their use of LLMs, for example when selecting submitted manuscripts” (225). This statement tangentially echoes concerns raised throughout the paper of the loss of human autonomy and discovery in the research process. How do journals think of LLM use today? Are papers looked upon less favorably if they use LLMs in any way, given the threat to transparency and the supposed threat to individuals’ research capabilities?

[Carolineyx]

Considering these mixed implications, how should we envision the future of research integrity and quality control in a landscape increasingly influenced by AI technologies? Moreover, what specific measures should we propose to ensure that the use of LLMs in scientific research and publication maintains high standards of accuracy, fairness, and transparency?