search

chokkan / crfsuite

CRFsuite: a fast implementation of Conditional Random Fields (CRFs)
http://www.chokkan.org/software/crfsuite/
Other
648 stars 208 forks source link

Different results across platforms (Windows, Ubuntu, etc...) #117

Closed franklevasseur closed 1 year ago

franklevasseur commented 4 years ago

Hi there,

We talked before, I already submitted few PR's on this project (which is great library btw).

We've been using your library for few months now in the botpress product for slot extraction purposes. Botpress is written in javascript and runs on the nodejs interpreter so I'm a actually using the node-crfsuite binding.

I'm trying to add few benchmark's and regression tests on my slot-extraction engine and I've noticed that results are different accross platforms.

What I mean is that if I run my tests twice on the same machine (ex: my personnal Ubuntu 18.04 Dell XPS computer), I observe the same results for each run, but if I switch OS and try it on Windows, I'll get different results. Again, if I run tests multiple times on Windows, I'll get the same results each time.

For information, I'm always using the "lbfgs" algorithm which uses the liblbfgs library.

Do you have any idea of why this is the case, or rather; do you know where i should start my research to identify the piece of code that makes the results different across platforms?

Much thanks in advance for all the help and information you can provide me,

François

aSmallsheep commented 4 years ago

@franklevasseur when you run in different platforms ,is there a big difference in running speed? There is a big difference in reasoning speed in OS and Windows. For 76000 statements,there is almost 20 seconds difference. The OS is slower. Is the compilation on OS wrong?

franklevasseur commented 4 years ago

Hi @aSmallsheep !

I haven't noticed any change in runtime speed between MacOS and Windows! Both build just fine.

If I notice any speed differences, I will let you know.

François