search

KCL-Planning / VAL

The plan validation system.
BSD 3-Clause "New" or "Revised" License
108 stars 48 forks source link

How can we use "validate" in scripts? #20

Open rpgoldman opened 7 years ago

rpgoldman commented 7 years ago

I was planning to invoke validate in a script, and check to see if my plan validated using the customary check against exit status. But I was disappointed to find that even when validate emitted `Bad plan description!" its exit code was still zero.

Digging further, since the plan argument is optional, and it's possible to check multiple plans in a single run of validate, I can see why the exit code wasn't set. And, indeed, AFAICT if validate fails to parse a plan file, it gets a zero back from getPlan, which means it's just like the plan not being there.

But if validate doesn't set an exit code, how can one use it in a script?

DerekLong101 commented 7 years ago

Hi Rob

Thanks for the comments.

Good question - perhaps it would be useful to set a command line switch to cause a fail-with-exit-code on a bad plan entry?

Cheers

Derek

From: rpgoldman notifications@github.com Sent: 12 September 2017 23:09:45 To: KCL-Planning/VAL Cc: Subscribed Subject: [KCL-Planning/VAL] How can we use "validate" in scripts? (#20)

I was planning to invoke validate in a script, and check to see if my plan validated using the customary check against exit status. But I was disappointed to find that even when validate emitted `Bad plan description!" its exit code was still zero.

Digging further, since the plan argument is optional, and it's possible to check multiple plans in a single run of validate, I can see why the exit code wasn't set. And, indeed, AFAICT if validate fails to parse a plan file, it gets a zero back from getPlan, which means it's just like the plan not being there.

But if validate doesn't set an exit code, how can one use it in a script?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FKCL-Planning%2FVAL%2Fissues%2F20&data=01%7C01%7Cderek.long%40kcl.ac.uk%7C088cb331b5e449b5239908d4fa2afadb%7C8370cf1416f34c16b83c724071654356%7C0&sdata=syCcQp12zfP9m9wJEIsvXqzk6oZzs3oG8xZkm%2Bv9Yhk%3D&reserved=0, or mute the threadhttps://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAHSnVE75cRWGipOmzvkSDqOUq9AcWj27ks5shwEhgaJpZM4PVUqG&data=01%7C01%7Cderek.long%40kcl.ac.uk%7C088cb331b5e449b5239908d4fa2afadb%7C8370cf1416f34c16b83c724071654356%7C0&sdata=yRj1Drsh%2BJXhKcTX9mAZJhP1F3IoUn3GgVEJG0VDBeU%3D&reserved=0.

rpgoldman commented 7 years ago

Yes, I think that would be quite helpful. The alternative would be forcing scripts to parse output strings, which isn't very appealing.

maltehelmert commented 7 years ago

For our work, we have to parse the output anyway because we are also interested in what VAL says about the quality of the validated plans. I don't know if it's helpful for you, but we use the "downward-lab" package by Jendrik Seipp for this, which is available online. I don't know how difficult it would be to adapt Jendrik's code for other planning systems than Fast Downward.