search

pepperoni21 / ollama-rs

A Rust library allowing to interact with the Ollama API.
MIT License
367 stars 47 forks source link

null fields in GenerateEmbeddingResponse #11

Closed manglemix closed 5 months ago

manglemix commented 6 months ago

I am using the ollama-rs library with a remote server running the latest mistral. Fairly frequently, I get these errors: An error occurred with ollama-rs: invalid type: null, expected a sequence at line 1 column 17 It is always this error on the same line and column. Most of the responses are fine though. Is it because some values in the embedding are NaN which encodes to null in json? If so, could it be fine for the response to deserialize correctly and just convert these nulls back to NaN?

pepperoni21 commented 6 months ago

Hey, thanks for reporting this issue. It seems to be a rather old issue of serde https://github.com/serde-rs/json/issues/202. I tried to fix that by replacing f64 with Option<f64>, which should cause NaN to be deserialized to None. I was not able to reproduce the issue so feel free to test the change ollama-rs = { git = "https://github.com/pepperoni21/ollama-rs", branch = "issue-embeddings"}

manglemix commented 6 months ago

Funnily enough my mistral server stopped producing NaNs and idk why. There was no change. If there is another person facing this issue in a more predictable way, this branch would probably be more helpful to them. Thank you for doing this though!

manglemix commented 6 months ago

Actually I faced the issue again. This branch does not fix it. The error is exactly the same.

pepperoni21 commented 6 months ago

Actually I faced the issue again. This branch does not fix it. The error is exactly the same.

I think NaN is encoded to "NaN" in json, not "null", so using Option doesn't fix it. I added json5 which handles NaN values in issue-embeddings, let me know if it works.