taku910 / crfpp

CRF++: Yet Another CRF toolkit
Other
505 stars 192 forks source link

why crf_test took long time #36

Open cmathx opened 7 years ago

cmathx commented 7 years ago

I use a machine with 64GB memory and it has only 12GB memory. I run a test with tagger->parse() and it costs long time about 150ms. But when I run a test in machine with enough free memory, it only costs 2-3ms. I find that most of the time spend on featureindex->buildFeatures(this) and buildLattice() operations. image

Naminwang commented 4 years ago

My machine has enough free memory, but parse() cost long time about 200ms. Do you find how to speed up the parse() interface? @cmathx

cmathx commented 4 years ago

You can focus on mmap function in the resp. A little case costs much time is because mmap call method.