Remove tweet-json/, tweet-csv/ etc from the displayed filename,
For example, show tweet-20210730.jsonl.gz instead of tweet-json/tweet-20210730.jsonl.gz
Also look into putting nodes, edges, csv as a prefix on the file to allow user to identify filetype once downloaded, for example:
mentions-nodes-00000-937227a7-46ed-4bcb-95ae-bcfba347357d-c000.csv.gz
Renaming appears to need to happen outside of Spark (e.g., using shutil or os).
Remove
tweet-json/
,tweet-csv/
etc from the displayed filename,For example, show
tweet-20210730.jsonl.gz
instead oftweet-json/tweet-20210730.jsonl.gz
Also look into putting nodes, edges, csv as a prefix on the file to allow user to identify filetype once downloaded, for example:
mentions-nodes-00000-937227a7-46ed-4bcb-95ae-bcfba347357d-c000.csv.gz
Renaming appears to need to happen outside of Spark (e.g., using shutil or os).