"MemoryError: std::bad_alloc" while using compose and intersect function.

facebookresearch / gtn_applications

Applications using the GTN library and code to reproduce experiments in "Differentiable Weighted Finite-State Transducers"

MIT License

80 stars 7 forks source link

"MemoryError: std::bad_alloc" while using compose and intersect function. #15

Open ronitd opened 3 years ago

ronitd commented 3 years ago

I want to compose two gtn, one is for lexicon, and the second is for grammar (LM) which is created from lm_arpa.py file. I am getting "MemoryError: std::bad_alloc" while doing with 250 GB RAM. I am not sure whether this is on the expected lines or not. PFA of both the gtn and code for reproducibility. gu-G.txt gu-L.txt

Code: gtn.savetxt('gu-LG.txt', gtn.compose(gtn.loadtxt('gu-L.txt'), gtn.loadtxt('gu-G.txt')).arc_sort())