search

plasticityai / magnitude

A fast, efficient universal vector embedding utility package.
MIT License
1.63k stars 120 forks source link

Is there a way to read the magnitude files from s3 ? #36

Closed mohammad1234 closed 6 years ago

mohammad1234 commented 6 years ago

Is there a way to read the magnitude files from s3

AjayP13 commented 6 years ago

Yes, it is documented here: https://github.com/plasticityai/magnitude#remote-streaming-over-http

You can see an example Python notebook here: https://colab.research.google.com/drive/1zkPhoNM1NvbTmEk9gr0Jnt8hONrca1Fv

Keep in mind this is still an alpha feature, if you really need performance and speed, download the ".magnitude" file instead of streaming it over an internet connection.

mohammad1234 commented 6 years ago

Thanks ... i will try it out