Technical review Building an AI Research Assistant: Large Language Models as A Versatile Tool for Digital Historians

[eliselavy]

Repo to create with pid JZx9gw7iwGxb. (here https://github.com/jdh-observer/JZx9gw7iwGxb) Wait for last version

[Dr-Hutchinson]

Final version submitted for technical review.

[eliselavy]

Problem to test it without paying an API key: https://platform.openai.com/account/usage

https://openai.com/pricing

Multiple models, each with different capabilities and price points. Prices are per 1,000 tokens. You can think of tokens as pieces of words, where 1,000 tokens is about 750 words. This paragraph is 35 tokens.

[eliselavy]

• regenerate the requirements.txt file to list only the packages usefull

• update path to include media or script folder

• retag image + include label in metadata for image Accuracy of the GPT Series on A.P. History Questions in the MMLU Benchmarks

• Notebook needs to be trusted in order to display output

• keep the output of the author as without OPEN_AI_API_KEY can not reproduce the analysis: article.ipynb

• generate output article-executed.ipynb following problem:

  ModuleNotFoundError: No module named 'langchain.chat_models'
  langchain==0.0.178

  and NameError: name 'SystemMessagePromptTemplate' is not defined

• remove files not mentionned in the notebook

  CHECK RESSOURCES
  file script/benchmarks_tests_euro.py is not called
  file script/more_index_summarized.csv is not called
  file script/benchmarks_tests_us.py is not called
  file script/benchmarks_tests_world.py is not called
  file script/high_school_us_history_test.csv is not called
  file script/high_school_european_history_test.csv is not called
  file script/high_school_world_history_test.csv is not called
  file media/more_history_of_richard_iii.pdf is not called
  file media/ocr_die_lotse-6-30-1945_1.pdf is not called
  file media/GPT-3 Dialogue with Martin Luther.txt.txt is not called

[eliselavy]

• correct one undefined reference
• add label to figure 2 / figure 3 / figure 5 correctly

[eliselavy]

Contact the author for a temporary API key - otherwise buy one

[eliselavy]

Purchase an API key, for using model: https://openai.com/waitlist/gpt-4-api

openai.ChatCompletion.create(
                      model="gpt-4",

[eliselavy]

ModuleNotFoundError: No module named 'langchain.chat_models'

[Dr-Hutchinson]

There seems to be a version error causing this. For cell 24 change get rid of the install for 'openai[embeddings':

# Below are the modules and libraries needed for running the case studies below.
# Requires Python version 3.8

!pip install pandas
!pip install Pillow 
!pip install openai[embeddings] # eliminate this install
!pip install langchain
!pip install matplotlib seaborn

After that I was able to run the code without errors.

Additionally, I can send a temporary API key if needed for accessing GPT-4.

[eliselavy]

@Dr-Hutchinson fixed problem langchain by using specific version as initially provide in the requirements.txt Add only necessary package see here https://github.com/jdh-observer/JZx9gw7iwGxb/blob/gpt-35-turbo/requirements.txt Preference by running the install requirements.txt to be able to use nbconvert otherwise required a restart of the kernel

I run the analysis with gpt-3.5-turbo, see branch here: https://github.com/jdh-observer/JZx9gw7iwGxb/blob/gpt-35-turbo/article.ipynb One error dataframe contains only one element:
Exception: ValueError: 1 is not in range

Linked to the gpt-3.5-turbo model? Still in waiting list for gpt-4 Maybe you can provide me your key ? (email: elisabeth.guerard@uni.lu)

[eliselavy]

From here: https://blog.finxter.com/game-changing-updates-from-openai-a-new-era-in-function-calling/?tl_inbound=1&tl_target_all=1&tl_form_type=1&tl_period_type=3

[eliselavy]

Output will be saved.

[eliselavy]

Generation with appropriate key: ok Notebook needs to be trusted after running nbconvert to display correctly : Nbdiff between author execution (article) - nbconvert (article_executed) saved:nbdiff

[eliselavy]

sent to review @inactinique