search

senarvi / theanolm

TheanoLM is a recurrent neural network language modeling tool implemented using Theano
Apache License 2.0
81 stars 29 forks source link

Decoding strange error #37

Closed amrmalkhatib closed 6 years ago

amrmalkhatib commented 6 years ago

The following error appeared when I tried to decode a lattice, it's very strange because it's the first time to encounter it, and it doesn't appear when I tried to decode on windows O.S, but tried on multiple linux Ubuntu machines and it appears every time. What is the probable cause? how can I solve this issue?

File "/usr/local/lib/python3.5/dist-packages/theanolm/scoring/latticedecoder.py", line 388, in decode node_tokens, link, lm_scale, wi_penalty) File "/usr/local/lib/python3.5/dist-packages/theanolm/scoring/latticedecoder.py", line 465, in _propagate self._linear_interpolation) File "/usr/local/lib/python3.5/dist-packages/theanolm/scoring/latticedecoder.py", line 142, in recompute_total nn_lm_weight) File "/usr/local/lib/python3.5/dist-packages/theanolm/backend/probfunctions.py", line 46, in interpolate_linear d_weight1 = Decimal(weight1) TypeError: conversion from numpy.float32 to Decimal is not supported

senarvi commented 6 years ago

It occurs when using the Decimal library to interpolate very small probabilities. Apparently conversion from NumPy to Decimal type doesn't work. I think the way to fix it would be to convert the number first to normal float (i.e. change the line in question into

d_weight1 = Decimal(float(weight1))

As a workaround you can try log-linear interpolation.

senarvi commented 6 years ago

Can you checkout the develop branch and see if that works?

amrmalkhatib commented 6 years ago

Yeah, that works. Thank you very much. But do you have any idea why this error appeared on linux, but not on windows?

senarvi commented 6 years ago

I can only guess that in Windows you have a different version of Decimal library that accepts NumPy floats, or then in Windows a different data type is used.