Closed Arjunsankarlal closed 4 years ago
Arjunsankarlal
commented
4 years ago I tried training and loading with the same set of data and code listed about with Python 3.6, now it is loading in 6 seconds! 😍 But I would be nice if I could still minimise the time taken for loading. If there is some way I can do this let me know! Thanks
sara-tagger
commented
4 years ago Arjunsankarlal
commented
4 years ago Hey @sara-tagger, thanks for the quick response. While debugging I found that this is not specific to Spacy Tokenizer! But this happens for every other configuration.
I have trained a RASA NLU model with the following config
language: en
pipeline:
- name: "pretrained_embeddings_convert"This configuration defaults to the list of components,
language: "en"
pipeline:
- name: "SpacyNLP"
- name: "SpacyTokenizer"
- name: "SpacyFeaturizer"
- name: "RegexFeaturizer"
- name: "CRFEntityExtractor"
- name: "EntitySynonymMapper"
- name: "SklearnIntentClassifier"Also I have tried all the other readily available configs like supervised_embeddings and pretrained_embeddings_spacy and custom configs as well. All of it takes 6~9 seconds of load time for instantiating the Trainer object. Similarly when I tried to load the persisted model for inference,
interpreter = Interpreter.load('../path_to_trained_model')again it takes almost of same 6~9 seconds for loading it. Then I tried to find the load time of each component. The first component only took ~ 6-9 seconds rest of them are taking just micro seconds. Then I tried a non time taking component individually with ComponentBuilder which surprisingly took ~ 6-9 seconds, so my guess is that there is something else causing this delay.
Is there anyway that this can be mitigated ? or I am doing something wrong ? Because I want to serve these models on demand, which requires a faster load time.
dakshvar22
commented
4 years ago @Arjunsankarlal SpacyNLP loads the language model provided by spacy for your specified language. I am guessing that should take up most of the time of the first component.
For pretrained_embeddings_convert, ConveRTFeaturizer would take some time because it loads up a large tensorflow based model.
When you say you want to serve these models on demand, do you mean training them or making inferences through them?
Arjunsankarlal
commented
4 years ago Hey @dakshvar22 , I also doubted the same. I am loading spacy en model here. When I load the same model with
spacy.load('en') for normal use, the time taken is ~ 0.6s, it is same model loaded here in rasa which is taking upto 6~9 seconds. So if you could give me any heads up for reducing the time it would be helpful.
Also when I say I want to serve these models on demand, I mean making inferences through them but loading the model when a request / first request is made. When I tried the loading the saved interpreter, the time taken for loading is ~6 secs, which I feel is very high since I will load models dynamically.
I also tried a different pipeline, which is a simple one,
def get_train_config():
return {'language': 'en',
'pipeline': "supervised_embeddings"}
myconfig = config._load_from_dict(get_train_config())
trainer = Trainer(myconfig, skip_validation=True)
interpreter = trainer.train(data)In this case the load time of the config is ~ 8.12 sec which was just micro seconds when I was using the spacy pipeline, not sure about this difference :/ But here the other component loading is ultra fast.
Also one interesting finding is that, after training completed, I loaded the persisted model from the saved path on the same execution, in which the SpacyNLP component load time is just ~ 0.6 seconds. So this is not SpacyNLP specific issue. Is this happening because of Rasa start up ?
I would love to use this NLU feature, but these latency issue is a big hindrance for it. Let me know your thoughts.
stale[bot]
commented
4 years ago This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
stale[bot]
commented
4 years ago This issue has been automatically closed due to inactivity. Please create a new issue if you need more help.
Rasa version: 1.6.0 (latest)
Rasa SDK version (if used & relevant): Not relevant
Rasa X version (if used & relevant): Not relevant
Python version: 3.7.1
Operating system (windows, osx, ...): MacOS
Issue: I am using custom SpacyPipeline with the following components,
name: SpacyTokenizer name: SpacyFeaturizer name: EmbeddingIntentClassifier
after debugging I found that the time taken for loading the specifics components are,
Time taken to load component 0 is 22.385185628000002 Time taken to load component 1 is 0.00010501800000284334 Time taken to load component 2 is 5.866900000128794e-05
Error (including full traceback):
Command or request that led to error:
Content of configuration file (config.yml) (if relevant): Loaded from dict
Content of domain file (domain.yml) (if relevant): Not relevant
For more details please look into this forum post
Is this expected ? Or I am I doing something wrong ? By the way, thanks for the great repo! ❤️