search

elitcloud / elit

🔠 Evolution of Language and Information Technology
https://elit.cloud
Other
47 stars 7 forks source link

move out nlp components #33

Open imgarylai opened 5 years ago

imgarylai commented 5 years ago

This elit project will only be the sdk for deployment to the cloud. Since we have many overlapping with gluon nlp, it might be a better way for us that we contribute nlp model to there. Since then, we don't have to care about having a unified api interface for each nlp component.

In the future, developers don't have to worry about how should implement their component. The only thing they should care about is the decode function.

I have created a snapshot on https://github.com/elitcloud/elit/tree/nlp for this move out process. I'm gonna move out every components after this commit 9e4f1855b8aa06f51b49aa2b62f97e2df8893af8.

@hankcs would you please move your parser to https://github.com/elitcloud/parser?

Please leave your ideas/thoughts here for further discussion.

Best

hankcs commented 5 years ago

Hum, changes in package again. Seems that I need to re-train parser model again.

imgarylai commented 5 years ago

Is it because of the path? Can it be solved by changing the config path?

hankcs commented 5 years ago

Some information is saved via pickle.save. Maybe we've spent lots of resource on adjusting package structure.

hankcs commented 5 years ago

Maybe json would be a better approach than pickle.

imgarylai commented 5 years ago

Yes, that is the part of question we really want to solve. If every component is developed stand alone, the developer can just do whatever they want in their way. Also, the elit api might change, and I don't want the change in this project affects any comp at all. That's why we want to move out again.

@jdchoi77 Please review this issue.

I had this issue with pickle before. This can be solved by read it and replace the config by changing the data object.

Also, json is easier to use, I like json. Either way works fine.