search

bab2min / tomotopy

Python package of Tomoto, the Topic Modeling Tool
https://bab2min.github.io/tomotopy
MIT License
548 stars 62 forks source link

Bad Allocation using DTModel #146

Open jlevy44 opened 2 years ago

jlevy44 commented 2 years ago

Hi there, I am running into an issue right now when following dtm.py. My jupyter notebook returns:

RuntimeError: bad allocation

when attempting to training the model.

Document loading appears to be okay, but not sure why this is happening. Are there some expensive array initializations? Thanks!

jlevy44 commented 2 years ago

Removed DTModel and replaced with LDAModel and worked just fine.

jlevy44 commented 2 years ago

Can you please assist with DTModel? Is anyone running into this issue?

bab2min commented 2 years ago

Hi @jlevy44 Sorry for late reply. Could you share more detail about your environment including OS, Python version and tomotopy version?

jlevy44 commented 2 years ago

I'll get back to you, as this is work with a collaborator. Do you have any tips of things to try when this happens?

bab2min commented 2 years ago

Hi @jlevy44 bad_alloc exception is thrown usually when the memory allocation is failed. But I think system memory is enough to train DTM in most of use case. So we need to determine if this problem is due to an unusual case that requires a lot of memory, or a case where the system memory is very low, or if it is a memory allocation bug in tomotopy. If you could share a minimal reproducible piece of code or data, it would be helpful in finding out the cause.