Open bieli opened 1 year ago
I decided to create the new repository IoText-rs with dedicated CI/CD processes and RUST lang. specific instead of mixing with this (Python lang.) impl.
@przemek1236 As we can see here: https://github.com/bieli/IoText-rs/pull/3 , when comparing the same data in both protocols IoText & JSON, we have very similar speeds of decoding from string to RUST data structure.
This is a good situation, because we have no issues on the performance side with IoText protocol when we have RUST implementation :-)
Based on experience from tiktoken data science Open Source repository and benchmarks for IoText encoder/decoder (compare to JSON), it will be good to have a fast native solution in
IoText data protocol
encoders/decoders.I believe, that RUST programming language is a good way!