search

ewfian / faiss-node

Node.js bindings for faiss
https://www.npmjs.com/package/faiss-node
MIT License
107 stars 10 forks source link

Ability to encode/decode buffers when FS not available? #37

Closed asilvas closed 11 months ago

asilvas commented 1 year ago

Happy to contribute, just wanted to know if there was a logical way to accomplish this? add is very expensive as it performs an encode, which adds up fast. Basically looking for similar efficiency to that of read/write from disk but from memory/buffers.

ewfian commented 1 year ago

Now it is just wraps the faiss api, if you have a good way, please let me know.

ewfian commented 1 year ago

Suddenly remembered: In langchain Python, the serialize_to_bytes was implemented. If faiss-node can also have a similar solution, the feature could also be ported to langchianjs

Ref: Serializing and De-Serializing to bytes

asilvas commented 1 year ago

My python is terrible, but I'll give it a look thanks.

asilvas commented 11 months ago

Resolved by https://github.com/ewfian/faiss-node/pull/40