georgeamccarthy / protein_search

The neural search engine for proteins.
GNU Affero General Public License v3.0
15 stars 6 forks source link

Load csv data #4

Closed georgeamccarthy closed 3 years ago

georgeamccarthy commented 3 years ago

Note: it may be possible to use jina.types.document.generators.from_csv but it is not clear to me how this is possible. Missing documentation on required arguments and lack of examples.

georgeamccarthy commented 3 years ago

I think I need you to accept a collaborator invite before I can add you as a reviewer. @fissoreg

fissoreg commented 3 years ago

The documentation for the Jina from_csv function is here: https://docs.jina.ai/api/jina.types.document.generators/#jina.types.document.generators.from_csv

It is probably a good idea to try to use it.

An example for usage should be similar to this: https://github.com/jina-ai/jina/blob/master/.github/2.0/cookbooks/Document.md#construct-from-json-csv-ndarray-and-files

georgeamccarthy commented 3 years ago

The documentation for the Jina from_csv function is here: https://docs.jina.ai/api/jina.types.document.generators/#jina.types.document.generators.from_csv

It is probably a good idea to try to use it.

An example for usage should be similar to this: https://github.com/jina-ai/jina/blob/master/.github/2.0/cookbooks/Document.md#construct-from-json-csv-ndarray-and-files

I spent around an hour trying to make this work inc: using the function and looking through the source code. I'll post in the support channel for further advice.

georgeamccarthy commented 3 years ago

Figured it out, needed to change the data to include a column title then pass that as a dictionary and map it to the text field.

fissoreg commented 3 years ago

Awesome! I left a little comment but it's a minor thing, feel free to merge when you want.