search

snipsco / snips-nlu-rs

Snips NLU rust implementation
https://snips.ai
Other
340 stars 56 forks source link

Loading of the model takes a lot of time #127

Closed cahuja1992 closed 5 years ago

cahuja1992 commented 5 years ago

We have around 40 intents. The model loading takes long time. Is there any way to compress the model so that it can be loaded faster.

adrienball commented 5 years ago

Hi @cahuja1992 , Could you provide the following information, so that we have a better idea of your setup:

Besides, could you share the output of du -h nlu_engine/* (where nlu_engine is the persisted nlu engine directory)?

Thanks

cahuja1992 commented 5 years ago

The model loading takes long time --> 10.0sec what kind of machine do you use?--> android 8.1, nexus 5x average number of utterances per intent?---> 5 average number of slots per intent?--->1 average number of entity values?->5

5.5M nlu-model/builtin_entity_parser/gazetteer_entity_parser/parser_1 1.8M nlu-model/builtin_entity_parser/gazetteer_entity_parser/parser_2 7.2M nlu-model/builtin_entity_parser/gazetteer_entity_parser 7.2M nlu-model/builtin_entity_parser 12K nlu-model/custom_entity_parser/parser/parser_1 12K nlu-model/custom_entity_parser/parser/parser_4 12K nlu-model/custom_entity_parser/parser/parser_3 12K nlu-model/custom_entity_parser/parser/parser_2 56K nlu-model/custom_entity_parser/parser 64K nlu-model/custom_entity_parser 20K nlu-model/deterministic_intent_parser 8.0K nlu-model/nlu_engine.json 12K nlu-model/probabilistic_intent_parser/slot_filler_12 28K nlu-model/probabilistic_intent_parser/slot_filler_19 24K nlu-model/probabilistic_intent_parser/slot_filler_1 32K nlu-model/probabilistic_intent_parser/slot_filler_3 12K nlu-model/probabilistic_intent_parser/slot_filler_17 12K nlu-model/probabilistic_intent_parser/slot_filler_13 12K nlu-model/probabilistic_intent_parser/slot_filler_2 12K nlu-model/probabilistic_intent_parser/slot_filler_16 12K nlu-model/probabilistic_intent_parser/slot_filler_21 36K nlu-model/probabilistic_intent_parser/slot_filler_5 32K nlu-model/probabilistic_intent_parser/slot_filler_7 32K nlu-model/probabilistic_intent_parser/slot_filler_6 12K nlu-model/probabilistic_intent_parser/slot_filler_9 24K nlu-model/probabilistic_intent_parser/slot_filler_20 12K nlu-model/probabilistic_intent_parser/slot_filler_15 28K nlu-model/probabilistic_intent_parser/slot_filler_10 28K nlu-model/probabilistic_intent_parser/slot_filler_11 16K nlu-model/probabilistic_intent_parser/intent_classifier/featurizer/tfidf_vectorizer 28K nlu-model/probabilistic_intent_parser/intent_classifier/featurizer 104K nlu-model/probabilistic_intent_parser/intent_classifier 12K nlu-model/probabilistic_intent_parser/slot_filler_14 12K nlu-model/probabilistic_intent_parser/slot_filler_8 32K nlu-model/probabilistic_intent_parser/slot_filler_4 12K nlu-model/probabilistic_intent_parser/slot_filler_18 12K nlu-model/probabilistic_intent_parser/slot_filler_0 560K nlu-model/probabilistic_intent_parser 64K nlu-model/resources/en/gazetteers 4.5M nlu-model/resources/en/word_clusters 92K nlu-model/resources/en/stemming 5.2M nlu-model/resources/en 5.2M nlu-model/resources

adrienball commented 5 years ago

@cahuja1992 Given the size of nlu-model/builtin_entity_parser, it seems that you are using some built-in music entities. These entities are typically quite big, and it may explain the loading time that you observe.

While it may be fine to have an important loading time in the case where the system is loaded once and then always runnning, I understand that this is problematic when building android/iOS apps.

The only workaround that I can suggest for now, if this is a real blocker, would be to replace the snips/musicXXX entities with custom entities, and adjust the number of values taking into account the tradeoff you have (loading time / NLU accuracy).

One way we could address this problem would be to provide other variants of the music entities, containing less values.

cahuja1992 commented 5 years ago

Thanks Yeah makes sense. Adding bunch of custom entities will increase the loading time. For sure. Closing the issue.

cahuja1992 commented 5 years ago

Closing the issue.