search

mauro-idsia / blip

Bayesian network Learning Improved Project
GNU Lesser General Public License v3.0
30 stars 11 forks source link

Measuring performance issues #9

Closed fcole90 closed 6 years ago

fcole90 commented 6 years ago

Hi, I am trying to measure the running time of blip but in my experiments it seems like that if I put a very high time constraint (is it possible to not set one?) , the algorithm runs until the end of such constraint, regardless of the tree-width boundary imposed.

E.g. Using the following parameters: java -jar blip.jar solver.kmax -j data/child-5000.jkl -r data/child-5000.kmax.res -t 9999 -b 0 -w 1 -v 1

Is this expected or should it terminate early instead?

What I am trying to do is to measure how the algorithm scales as the number of features and samples rises. Do you think it would be feasible?

Best regards, Fabio

mauro-idsia commented 6 years ago

This is the expected behaviour of the algorithm - these are approximate methods, that iteratively sample new candidate solution using all the available time.

Measurement in this area is tricky, since one would need the optimal solution in order to do that, and the exact approaches to bounded-treewidth structure learning can handle at most dozens of variables.

fcole90 commented 6 years ago

OK, thanks for pointing this out. So the comparison is done using constant time (60 secs) and comparing the W score among the different algorithms, right?

mauro-idsia commented 6 years ago

That is a possible way (but not necessarily the only one).