Closed lakema17 closed 4 years ago
It's undocumented, but there is a bin/process-records
program which splits the records into multiple files (1 file for every 100,000 records). It's used because GitHub has file size limits, so I workaround that by using multiple files.
Does importing 3 different files merge them into a single record?
Yeah, importing always causes it to merge into existing records.
Seeing that not many bots can do this, I'd suggest adding a way to split the records into a JSON file for each month and . This adds the possibility of sharing certain segments of matches in order to improve the data for the algorithm to use.
P.S.: I was planning to write a nodejs program to do it, but I got too lazy. Might just make a simple js file for splitting and one for merging idk