Add file read utils to read & preprocess the files to upload to database

unna97 / twarc2sql

This converts the output jsonl file twarc2 api to sql.

https://twarc2sql.readthedocs.io

MIT License

0 stars 0 forks source link

Add file read utils to read & preprocess the files to upload to database #3

Open unna97 opened 1 year ago

unna97 commented 1 year ago

Is your feature request related to a problem? Please describe. Add the file utils to read the twarc file to automatically pre-process the data and upload it to the database. Allow for different use cases from twarc2 output.