Open rstarmer opened 1 year ago
@rstarmer do you permit changes in the input format? We generally have far more elaborate user stories. I guess we didn't do that here. Currently, for storage of retrieval information, jsonl
is an easy path forward. Let me see if I can enhance this user story a bit.
Please also note, that processing realnews took quite a bit of time. I have found the wikipedia jsonl files, and it will take about (roughly guessing) the same compute....
If possible extend PyKoi (or a similar toolset) to inference against any/all of the following datasets: