I see that SIMD instructions can be used to speed up reading JSON files in some cases. For example, there is the simd-json crate, although it's not the only choice. Sonic says that it uses SIMD instructions in particular to "parse the fractions of float numbers," which is something that a GeoJSON parser does constantly.
I don't see any discussion of SIMD in this repo at all, so I thought I'd ask whether this a worthwhile avenue to explore for a speedup. I see that we're using serde_json, which simd-json says it is compatible with. I haven't done any work to determine if this would actually speed things up, because I wanted first to ask if there was some reason it wouldn't be a good fit even if it was faster.
michaelkirk
commented
6 months ago
I see that SIMD instructions can be used to speed up reading JSON files in some cases. For example, there is the simd-json crate, although it's not the only choice. Sonic says that it uses SIMD instructions in particular to "parse the fractions of float numbers," which is something that a GeoJSON parser does constantly.
I don't see any discussion of SIMD in this repo at all, so I thought I'd ask whether this a worthwhile avenue to explore for a speedup. I see that we're using serde_json, which simd-json says it is compatible with. I haven't done any work to determine if this would actually speed things up, because I wanted first to ask if there was some reason it wouldn't be a good fit even if it was faster.