How to work with JSON lines database?

openai / gpt-2-output-dataset

Dataset of GPT-2 outputs for research in detection, biases, and more

MIT License

1.93k stars 548 forks source link

How to work with JSON lines database? #43

Open yukiarimo opened 1 year ago

yukiarimo commented 1 year ago

Hello. I downloaded all files. And all of them are just a randomly answers in JSON format. So, I want to train my own tensorflow.js model using this database! But, I don't have a question database here. So, what I need to do?

allosharma commented 1 year ago

Hi @yukiarimo,

You can read the JSON file using the following code, then you can you your model to train the data.

import pandas as pd

#Following line will read json file using pandas
json_data = pd.read_json('json_file_path', lines=True)

#Display the data
json_data.head(5)

I hope this helps.