Train/test options - Githubissues

celinehocquette / numsynth-aaai23

C. Hocquette and A. Cropper, Relational program synthesis with numerical reasoning, AAAI23.

https://arxiv.org/pdf/2210.00764.pdf

14 stars 0 forks source link

Train/test options #1

Open khrudkova opened 2 months ago

khrudkova commented 2 months ago

Hello,

I would like to ask, whether it is possible to get hypothesis from train data and then test it on test data - therefore, the TP, TN, FP, FN scores would be evaluated on the test set, which would be different, from the train set. Alternatively, if it is possible to insert hypothesis and have it tested on selected dataset.

Thank you.

celinehocquette commented 2 months ago

Yes, you can simply add your hypothesis to a Prolog file, together with the test data and bk. Then you can execute the hypothesis against the positive and negative examples and count how many examples of each it entails.

An example of implementation to count TP / FN / TN / FP is available here: https://github.com/celinehocquette/numsynth-aaai23/blob/main/ilp-experiments/ilpexp/system/test.pl

khrudkova commented 1 month ago

Thank you for the answer, please, could you provide more information, on how to execute this process? Mainly:

where should be the hypothesis that has been previously generated on train set - should it be inserted to the BK file along with the test data?
is there any specific command line argument for this functionality?

Thank you.

celinehocquette commented 1 month ago

Add to the same Prolog file the test data, bk, learned hypothesis, and code of this file. Then query do_test(TP,FN,TN,FP).