search

etalab-ia / piaf-ml

PIAF v2.0 repo for ML development. Main purpose of this repo is to automatically find the best configuration for a QA pipeline of a partner organisation.
MIT License
8 stars 0 forks source link

Switch from Finder to Pipeline in Haystack API #33

Closed Rob192 closed 3 years ago

Rob192 commented 3 years ago

Automate the creation of the Haystack API. This implies switching from Finder to Pipeline. See PR :

guillim commented 3 years ago

update of 2 april : finally got this branch working on my local env, replied to the PR thread, and waiting for its fianl approval in the main branch before moving all our projects to it !

https://github.com/deepset-ai/haystack/pull/922

guillim commented 3 years ago

I did come up with a working version : https://github.com/guillim/haystack/tree/revamp-api-dila I wait for Deepset to merge the original one before setting up a more permanent version

Rob192 commented 3 years ago

Nicccce ! Now I think the easiest way to maintain this is to host this code on our piaf-ml repo and not on a haystack fork. This would ease maintenance as we would not need to keep the fork up-to-date.

Also, as in the future, the creation of the knowledge base will be integrated in the docker creation, I think it makes more sense to do everything from the piaf-ml repo. (see #42 )

guillim commented 3 years ago

I created a task for the issue you raised : how do we deal & maintain several Piaf-API "clients". it's here : https://github.com/orgs/etalab-ia/projects/1#card-58680616

It something we will have to do quickly, but not as easy at the moment as only a pipelines.yaml conf. Because to make haystack work with pipelines, several other files when using custom components

guillim commented 3 years ago

Should we consider this issue to be solved once we move also the DILA api to this new way of doing ? If that is the case, it requires some work to do (i would say at least 1 day with the database creation and upgrade of haystack). I should probably create a dedicated card about that. What do you think ?

guillim commented 3 years ago

@Rob192