ExperimentRunner output unhelpful

[lkq1992yeah]

Hi Matt,

I've downloaded the code, and I found there are many json files in the "examples" folder, so I ran the command: sbt "run ./examples/" , passing this folder to ExperimentRunner as the base directory. The program did recognize all the json files, however, none of experiment specifications is performed, and it seems that the function "runPraFromSpec" has never been executed. In fact I'm a fresh man learning scala, I'm not sure about the meaning of the code "shuffled.map(runPraFromSpec(prabase) )". So is it the bug of program, or I made a mistake somewhere? Thanks so much!

[matt-gardner]

Can you paste in the full console output when you do that?

[lkq1992yeah]

Below is the console information. I added some new println codes.

kangqi@Darkstar:~/workspace/pra$ sbt "run ./examples/" Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0 [info] Loading project definition from /home/kangqi/workspace/pra/project [info] Set current project to pra (in build file:/home/kangqi/workspace/pra/) [warn] Multiple main classes detected. Run 'show discoveredMainClasses' to see the list

Multiple main classes detected, select one to run:

[1] edu.cmu.ml.rtw.pra.experiments.ExperimentRunner [2] edu.cmu.ml.rtw.pra.experiments.ExperimentScorer [3] edu.cmu.ml.rtw.users.matt.one_off.neil_experiments

Enter number: 1

[info] Running edu.cmu.ml.rtw.pra.experiments.ExperimentRunner ./examples/ [info] Kangqi 0: Enter directory ./examples/ ... [info] experiment_specs size = 349 [info] Kangqi 1: Enter directory ./examples/ ... [success] Total time: 4 s, completed Sep 26, 2015 9:05:10 PM kangqi@Darkstar:~/workspace/pra$

I think I've made a stupid mistake: I didn't pass the second param to the program, therefore no specification is kept, causing the previous problem. Then I ran the command sbt "run ./examples/ final_emnlp2015", the program was running successfully (though some new error prompted, maybe related with data files, I'm trying to solve them by myself).

I think the second argument of ExperimentRunner is compulsory, otherwise nothing would be executed.

[matt-gardner]

Thanks for this. I think there are two things here I need to fix: (1) better output when the system can't find an experiment spec to run, and (2) better documentation for the second argument (it's supposed to be optional, so I need to figure out what went wrong there). I haven't ever used the code with the example directory like that, so it's doing some things I didn't expect it to.

[lkq1992yeah]

You can change the function "shouldKeepFile" so that it always return true if the param "filters" is empty, that will do.

BTW, I found some of "/home/mg1/pra/somewhere" hard coding for specifying a file path. Shall I replace them with my local path (like "/home/kangqi/workspace/pra/somewhere") locally in the code?

[matt-gardner]

Yes, that looks like it would work.

As far as hard-coded paths, if you see them anywhere in scala code under edu.cmu.ml.rtw.pra, that's an error that needs to be fixed. If it's under edu.cmu.ml.rtw.users.matt.one_off, those are scripts that you probably shouldn't need to use anyway, so I wouldn't worry about it. And if you're talking about the paths in json files under examples/, yes, you might need to change some of those paths. Using the data download from the SFE paper website should make it so you don't have to change anything, though, because those paths shouldn't ever be actually read (well, maybe there would be an error for stuff related to embeddings or vector space stuff). If you see an error with that, let me know, because that means I need more instructions on the SFE website.

[lkq1992yeah]

Yes, the only things to be replaced are the paths in json files under examples/. Remaining hard paths in the codes only occur under edu.cmu.ml.rtw.users.matt.one_off, which can be ignored by us.

[matt-gardner]

Ok, the issue with filters should now be fixed, so I am closing this.