Hi, I encountered a memory issue when using the cc.en.300.bin fasttext when train the core program.
Training Core model...
Processed Story Blocks: 100%|█████| 3/3 [00:00<00:00, 5240.70it/s, # trackers=1]
Processed trackers: 100%|█████████| 3/3 [00:00<00:00, 3503.04it/s, # actions=11]
Processed actions: 11it [00:00, 16331.80it/s, # examples=11]
Processed trackers: 100%|██████████████████████| 3/3 [00:00<00:00, 48770.98it/s]
Epochs: 100%|█| 1/1 [00:00<00:00, 23.03it/s, t_loss=8.110, loss=8.046, acc=0.133
2021-01-19 12:24:49 INFO rasa.utils.tensorflow.models - Finished training.
2021-01-19 12:24:49 INFO rasa.core.agent - Persisted model to '/tmp/tmp1nl0sz0q/core'
Core model training completed.
Training NLU model...
Warning : `load_model` does not return WordVectorModel or SupervisedModel any more, but a `FastText` object which is very similar.
Traceback (most recent call last):
File "/home/zapps/anaconda3/envs/Rasa1/bin/rasa", line 8, in <module>
sys.exit(main())
File "/home/zapps/anaconda3/envs/Rasa1/lib/python3.7/site-packages/rasa/__main__.py", line 91, in main
cmdline_arguments.func(cmdline_arguments)
File "/home/zapps/anaconda3/envs/Rasa1/lib/python3.7/site-packages/rasa/cli/train.py", line 76, in train
additional_arguments=extract_additional_arguments(args),
File "/home/zapps/anaconda3/envs/Rasa1/lib/python3.7/site-packages/rasa/train.py", line 50, in train
additional_arguments=additional_arguments,
File "uvloop/loop.pyx", line 1456, in uvloop.loop.Loop.run_until_complete
File "/home/zapps/anaconda3/envs/Rasa1/lib/python3.7/site-packages/rasa/train.py", line 101, in train_async
additional_arguments,
File "/home/zapps/anaconda3/envs/Rasa1/lib/python3.7/site-packages/rasa/train.py", line 188, in _train_async_internal
additional_arguments=additional_arguments,
File "/home/zapps/anaconda3/envs/Rasa1/lib/python3.7/site-packages/rasa/train.py", line 245, in _do_training
persist_nlu_training_data=persist_nlu_training_data,
File "/home/zapps/anaconda3/envs/Rasa1/lib/python3.7/site-packages/rasa/train.py", line 482, in _train_nlu_with_validated_data
persist_nlu_training_data=persist_nlu_training_data,
File "/home/zapps/anaconda3/envs/Rasa1/lib/python3.7/site-packages/rasa/nlu/train.py", line 75, in train
trainer = Trainer(nlu_config, component_builder)
File "/home/zapps/anaconda3/envs/Rasa1/lib/python3.7/site-packages/rasa/nlu/model.py", line 145, in __init__
self.pipeline = self._build_pipeline(cfg, component_builder)
File "/home/zapps/anaconda3/envs/Rasa1/lib/python3.7/site-packages/rasa/nlu/model.py", line 157, in _build_pipeline
component = component_builder.create_component(component_cfg, cfg)
File "/home/zapps/anaconda3/envs/Rasa1/lib/python3.7/site-packages/rasa/nlu/components.py", line 769, in create_component
component = registry.create_component_by_config(component_config, cfg)
File "/home/zapps/anaconda3/envs/Rasa1/lib/python3.7/site-packages/rasa/nlu/registry.py", line 246, in create_component_by_config
return component_class.create(component_config, config)
File "/home/zapps/anaconda3/envs/Rasa1/lib/python3.7/site-packages/rasa/nlu/components.py", line 483, in create
return cls(component_config)
File "/home/zapps/assistant_LU_1/rasa-custom-fasttext/ftfeat.py", line 36, in __init__
self.model = fasttext.load_model(path)
File "/home/zapps/anaconda3/envs/Rasa1/lib/python3.7/site-packages/fasttext/FastText.py", line 441, in load_model
return _FastText(model_path=path)
File "/home/zapps/anaconda3/envs/Rasa1/lib/python3.7/site-packages/fasttext/FastText.py", line 98, in __init__
self.f.loadModel(model_path)
MemoryError: std::bad_alloc
Is it related to memory limitation? if yes, is there a way to change the batch size or something?
sara-tagger
commented
3 years ago
Hi, I encountered a memory issue when using the
cc.en.300.binfasttext when train the core program.Is it related to memory limitation? if yes, is there a way to change the batch size or something?