The current tweet scraping code has hard-coded data field parameters. It should be made as user input instead of hard-coding. Incase of nested json data, user can enter the child object key, and a deep search should be performed to identify the key-value pair, from which the dataframe would be ideally built.
There should also be a helper function available, so that the user can take a look at the available, full json structure from snscrape (maybe by scraping a single tweet), and decide on which fields are important.
Based on that decision, that can pass a list of key names (child keys incase of nested json), which would be used to scrape and form the dataframe, that would be returned from the scraper function.
The current tweet scraping code has hard-coded data field parameters. It should be made as user input instead of hard-coding. Incase of nested json data, user can enter the child object key, and a deep search should be performed to identify the key-value pair, from which the dataframe would be ideally built.
There should also be a helper function available, so that the user can take a look at the available, full json structure from snscrape (maybe by scraping a single tweet), and decide on which fields are important.
Based on that decision, that can pass a list of key names (child keys incase of nested json), which would be used to scrape and form the dataframe, that would be returned from the scraper function.