search

minio / simdjson-go

Golang port of simdjson: parsing gigabytes of JSON per second
Apache License 2.0
1.8k stars 85 forks source link

Please support arm64 #51

Open lizthegrey opened 2 years ago

lizthegrey commented 2 years ago

Hi! We'd love to use this library, but we're currently all in on arm64. Is there a chance you'll do an arm64 version of this port?

harshavardhana commented 2 years ago

Hi! We'd love to use this library, but we're currently all in on arm64. Is there a chance you'll do an arm64 version of this port?

That is a significant effort @lizthegrey - we are open to pull requests.

lizthegrey commented 2 years ago

Ah, I see, it's not a mechanical translation, it's a hand port? Will take that under consideration. Thanks!

fwessels commented 2 years ago

There is arm64 support in the original simdjson project -- so it is definitely possible to port this over, but it will be a significant effort.

klauspost commented 2 years ago

A more manageable (and possibly also more useful) would be Go fallback. That would probably also make porting easier.

Stage 2 only uses SIMD for string parsing, so that should be rather trivial, but obviously stage 1 is a bit more involved. Though I don't understand enough of it to do it.

I would also like to get rid of the remaining allocs, but I currently don't have the bandwidth for it.

That said, with the latest clean up it looks like a decent v1.

martin-sucha commented 2 years ago

Go fallback would be useful indeed.

I needed a workaround for arm64 machines for simple things like running small tests and debugging, so I hacked together https://github.com/kiwicom/fakesimdjson . By the way, the tape is exported from the simdjson package, is the tape format considered part of the stable API?

Would simdjson-go project consider something like fakesimdjson useful enough to be included or are you interested only in more direct implementations of stages 1 and 2? Obviously something like my hack can't be used to guide ports to new architectures.

klauspost commented 2 years ago

@martin-sucha Not guaranteed, which is why we are still at v0.x. We have no plans to change it, but only API and the serialized format can be considered stable.

You are welcome to send your repo code as a PR, if it makes sense to add.

stokito commented 1 year ago

the original simdjson has some support of the arm64 https://github.com/simdjson/simdjson/tree/master/src/arm64