search

OpenMined / SyferText

A privacy preserving NLP framework
Apache License 2.0
198 stars 49 forks source link

Add pipeline parsing tests #192

Closed bicycleman15 closed 3 years ago

bicycleman15 commented 4 years ago

Fixes #177 Please review. @AlanAboudib I am still not sure how a pipeline is being reused. Can you give a small example in code so that I could improve the tests further.

Currently it is looking pretty similar to distributed_mode/test_pipeline.py

Also if there is anything more you would like me to add, please feel free.

AlanAboudib commented 4 years ago

HI @jatin So the first thing you need to do is to create a pipeline where the SingleLabelClassifier is deployed on bob for instance with access = { 'bob'}. The tokenizer and the simple tagger have access = {'*'}.

Then, after you deploy and load the pipeline, you create two remote strings, one on alice and the other on james for instace and apply the pipeline on both of them.

If you check nlp.pipeline you will see that for alice and james, there is a subpipeline on bob that is reused (same ID)

Am I being clear?

bicycleman15 commented 3 years ago

Please review @AlanAboudib. sorry that it took so long :sweat_smile: