search

Joeboy / mptweetsgpt

Use chatgpt to analyse UK MPs' tweets
0 stars 0 forks source link

Missing jsons from import_tweets.py #1

Open raineydavid opened 9 months ago

raineydavid commented 9 months ago

Will be unable to pre-process new tweets with current method in import_tweets.py as these files are hardcoded and also not present. Might be worth changing to file names and showing the sample tweets.

DATA_ROOT / "dataset_twitter-scraper_2024-01-20_12-44-03-246.json" ).open() as f: data = json.load(f) with (DATA_ROOT / "dataset_tweet-flash_2024-01-20_16-07-32-791.json").open()

Joeboy commented 9 months ago

Yeah, there isn't currently a good way to update the tweet data. I just worked from the json files Hannah scraped using apify.com, which apparently had issues that prevented us getting complete and reliable data. Figuring out a good way to do this might be another Hack Day project in itself.