openai / openai-dotnet

The official .NET library for the OpenAI API
https://www.nuget.org/packages/OpenAI
MIT License
1.13k stars 113 forks source link

Improve Embedding.ConvertToVectorOfFloats #38

Closed stephentoub closed 3 months ago

stephentoub commented 3 months ago

It's currently incorrect on big endian systems (e.g. IBM s390x), as it blits the little-endian bytes into an array of floats. On big endian the values need to be reversed.

It also assumes the data being sent back from the service is always correct. If it's corrupted in certain ways, such as not being as long as was expected, not being quoted, etc., we can silently get bad data.

It also more allocation than is necessary. It first allocates a string for the base64 string data. Then it allocates a new string with the quotes stripped off. Then it allocates a byte[] with the decoded bytes. And finally it allocates the resulting float[]. We can avoid the initial string by getting the raw memory from the BinaryData. We can avoid the substring by just slicing those bytes. And we can generally avoid the temporary byte[] by renting one from the array pool. That leaves just the required float[].