dollabs / pamela

Probabalistic Advanced Modeling and Execution Learning Architecture
Apache License 2.0
233 stars 13 forks source link

Need reference HTN and TPN files for suite of Pamela files #69

Closed dcerys closed 7 years ago

dcerys commented 7 years ago

This will allow us to do regression testing of the HTN/TPN generation process

tmarble commented 7 years ago

First pass at HTN reference files are already in the repo, covering the 4 root-task invocation options:

  1. Plant instance referred in HTN method via field as an instantiated plant in the HTN pclass.
    • pamela -v -v -l trace -i test/pamela/biased-coin.pamela -t "(main.flip-3)" -o biased-coin htn
    • WORKS
  2. Plant instance as an argument to the HTN pclass constructor (plant as a pclass param).
    • pamela -v -v -l trace -i test/pamela/regression/simple-plant.pamela -t "(simple-plant-demo.run.main)" -o simple htn
    • WORKS
  3. Plant instance as an argument to the HTN pclass constructor. (plant as field, initialized by pclass param).
    • pamela -v -v -l trace -i test/pamela/regression/isr-htn.pamela -t "(isr-htn-demo.htn.main \"A\" \"B\" \"C\" \"D\")" -o isr htn
    • WORKS
  4. Root task specifies instantiated plant instances passed to the top level HTN method.
    • pamela -v -v -l trace -i test/pamela/quadcopter.pamela -t "(qc-waypoints.waypoints qc-demo.qc1 qc-demo.qc2 qc-demo.qc3)" -o qc htn
    • WORKS NOTE: visualization of above with:
      planviz -v -i biased-coin.htn.edn=biased-coin.tpn.edn \
      -i simple.htn.edn=simple.tpn.edn \
      -i isr.htn.edn=isr.tpn.edn \
      -i qc.htn.edn=qc.tpn.edn
tmarble commented 7 years ago

NOTE: to complete this issue we should any necessary pamela source files along with TPN and/or HTN output files.

tmarble commented 7 years ago

In addition to the above we now have an additional HTN example:

  1. Root task specifies top level HTN method with choice:
    • pamela -v -v -l trace -i test/simple-choice-parallel/choice.feasible.pamela -t "(choice-feasible.main)" -o choice.feasible htn
    • WORKS

For TPN's we have

  1. From the CLI test 50_tpn.sh
    • test/pamela/parallel-choice.tpn.pamela
  2. From the CLI test 56_tpn-construct.sh
    • test/pamela/{qc,qc-waypoints,qc-demo}.pamela
pmdoll commented 7 years ago

This is not complete yet. We need a list of all pamela files for which we could generate HTN and TPN files by applying htn task.

We also need to do the same for IR format and for all pamela files.

tmarble commented 7 years ago

Based on the work documented in the PR #79 ... We now generate and test IR for all pamela files, and Generate and test HTN generation for one (or more) root-tasks per (HTN compatible) pamela file. And the test framework will test against "gold standards".

@pmdoll please explain what else is necessary to close this issue.

pmdoll commented 7 years ago

Unit tests should pass


prakash@pablo ~/p/g/d/pamela> git branch
  master
* name-harmonization
prakash@pablo ~/p/g/d/pamela> ./bin/jenkins.sh 

HTN sequence-bounds.example.root-task TAG example regression? true
HTN simple-plant.main.root-task TAG main regression? true

Testing testing.pamela.parser

FAIL in (testing-pamela-parser) (parser.clj:73)
testing-pamela-parser
expected: {coverage-demo
           {:type :pclass,
            :args [],
            :fields
            {:sp1
             {:access :private,
tmarble commented 7 years ago

Fixed in 88a8c55

dcerys commented 7 years ago

We now have a flexible framework, populated with a good range of tests