Reprocessing is currently not so fast, given that it's feeding from JSONL files. We should consider using something that's more performant like avro, msgpack or similar.
For historical purposes, here are some benchmarks I ran sometime ago while building OONI Data:
Reprocessing is currently not so fast, given that it's feeding from JSONL files. We should consider using something that's more performant like avro, msgpack or similar.
For historical purposes, here are some benchmarks I ran sometime ago while building OONI Data: