Train component HFTransformersNLP MemoryError.

[XiaofeiQian]

Rasa version: 1.8.0 Rasa SDK version (if used & relevant):

Rasa X version (if used & relevant):

Python version: 3.7.5 Operating system (windows, osx, ...): Windows 10 Issue: I use HFTransformersNLP to train a model: that is the data detail:

2020-03-09 21:22:58 INFO     rasa.nlu.training_data.training_data  - Training data stats: 
        - intent examples: 128438 (87 distinct intents)
        - Found intents: .....
        - Number of response examples: 127879 (4910 distinct response)
        - entity examples: 145 (4 distinct entities)
        - found entities: ......

2020-03-09 21:22:58 DEBUG    rasa.nlu.training_data.training_data  - Validating training data...
2020-03-09 21:23:02 INFO     rasa.nlu.model  - Starting to train component DataCleaner
2020-03-09 21:23:02 INFO     rasa.nlu.model  - Finished training component.
2020-03-09 21:23:02 INFO     rasa.nlu.model  - Starting to train component HFTransformersNLP

After 2020-03-09 21:23:02 INFO rasa.nlu.model - Starting to train component HFTransformersNLP Wait a long time then I got a MemoryError. But in fact this component does not need training.

It seems that all the training data is converted into features here and appended to the original data at the same time. This has led to an increase in the size of the data, and finally the memory is not enough.

https://github.com/RasaHQ/rasa/blob/c1e24731ec6bb2751d38eed0e692031e9cef6b07/rasa/nlu/utils/hugging_face/hf_transformers.py#L314-L344

What could we do to fix this? If need more information, please tell me, Thanks. Command or request that led to error:

rasa train nlu

[XiaofeiQian]

In addition, if the amount of data is small, such as less than 100, there will be no errors.

[sara-tagger]

Thanks for raising this issue, @dakshvar22 will get back to you about it soon✨

Please also check out the docs and the forum in case your issue was raised there too 🤗

[dakshvar22]

@HatuneMiku Thanks for raising this. While this is a valid limitation, it is because of the inherent design limitations that current structure of different components exhibit. Although, I wonder if you need so many training examples. You approximately have 1470 examples on an average per intent. Have you tried reducing the training data and evaluate the trained model on a test set in that case?

[XiaofeiQian]

Yes, I have a lot of training data, but most of them are used for FAQ response (ResponseSelector component). - Number of response examples: 127879 (4910 distinct response) In fact, there are about 26 examples on an average per response. I will try to reduce each response example to 4 and try again.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed due to inactivity. Please create a new issue if you need more help.