Closed guicho271828 closed 2 years ago
I'd like to see this added to planutils
when it's set, and then an easy transition to the new remote planning service about to be released.
planutil
is just for conveniently calling planners, correct?
Nope. View it as a sort of "apt" for planning-like software. So, yes, easy install of planners, but also libraries, utilities, etc.
Hi Masataro, before you start hacking away: this PR plans to address many issues at the same time (parameter specifications, task filtering, task transformation, batch task creation, uniform output filenames). I think some of these deserve their own PR.
Also, task filtering and task transformation feel like candidates for external postprocessing tools to me. I think it's best to have them as standalone tools.
For batch task creation, I think we can eventually move the code from https://github.com/rleap-project/batch-pddl-generator into this repo. Your __main__,py
file has a "-n" parameter, but how would you choose the n instances?
That leaves mainly the parameter specification issue. Which use case do you envision here? If the goal is to get good error messages for an invalid parameter combination, I think the best approach is to change the generator code and add argument checking and error messages there. If the goal is to present the valid parameter ranges to users, I think the same applies: add or improve the existing help messages in the generator code. This would definitely be helpful for users.
task filtering and task transformation
This is not an immediate task and I do not plan to do it for now. I just made a placeholder.
This is a HUGE reinvention of wheel and I do not think worth pursuing. I would simply use GNU parallel. For example,
parallel pddl-generator blocksworld-3ops --seed {1} -blocks {1}::: {0..4} ::: {0..3}
ah, didn't catch the part that you use the SMAC hyperparamter tuner. The part I am making will be the interface to be called from SMAC then.
That leaves mainly the parameter specification issue. Which use case do you envision here? If the goal is to get good error messages for an invalid parameter combination, I think the best approach is to change the generator code and add argument checking and error messages there. If the goal is to present the valid parameter ranges to users, I think the same applies: add or improve the existing help messages in the generator code. This would definitely be helpful for users.
I don't pursue these non-trivial goals. I simply make this set of generators usable and streamlined. Nothing new, simply refine and improve. Currently I have to manually look into each generator, learn the usage from README, and eventually write code that does the job I want. I did it for 5 domains for my recent paper. The result of this PR will be a generator with a common API that is well documented in a centralized manner.
Alright. In any case, I suggest to start with a minimal solution and to only tackle one domain in the first PR. Ideally, a domain where not all parameter combinations are allowed. Then we can see whether a centralized design is helpful or whether it's better to improve the generators themselves.
I see you're hard at work here Masataro. Let me reiterate though: I won't be able to review the changes for ~60 domains. If you want to eventually have your work merged into this repo, you need to tackle a single domain in its own PR.
it is just about 3 hours of burst work ... It is nothing. btw, I want to show you how the resulting experience would be like. Please take a look at this asciinema. https://asciinema.org/a/0dS4eHPc3D8y4DGybouogrn9j
If you would take time to review changes (thanks!), I would recommend checking the commits tagged as [api] and [implementation].
The rest of the commits (tagged as [domain] ) are largely just copy-and-paste from pddl-generators/__template , using copy-template.sh
script. There is not much to review, and I have not spent significant time in them. They are separated just for the sake of clean commit messages.
So nothing in any specific domain really matters in these commits, I treat them largely as test cases for assessing the utility of the tooling. I left the details as the room for improvement for future users.
By the way, I have not introduced a single line of change in the existing code base --- I have only moved the original files (it was necessary for being compatible with python tooling) and added several new files, no existing files are modified significantly.
Current interface:
usage: pddl-generators [-h] [--output-directory DIRNAME] [--output FILENAME]
[--output-domain FILENAME] [--separator SEPARATOR] [--force]
[--seed SEED] [--helpall] [--list] [--debug]
[{agricola,assembly,barman,blocksworld-3ops,blocksworld-4ops,blocksworld-matching-typed,blocksworld-matching-untyped,briefcaseworld,cavediving,childsnack,citycar,crewplanning,data-network,depots,driverlog,elevators,ferry,floortile,freecell,fridge,goldminer,grid,gripper,grippers,hanoi,hiking,logistics,maintenance,matchcellar,miconic-fulladl,miconic-simpleadl,miconic,movie,mprime,mystery,nomystery,npuzzle,nurikabe,openstacks,parking,pathways,pegsol,rovers,satellite,scanalyzer,schedule,snake,sokoban,spanner,spider,storage,termes,tetris,tidybot,tms,tpp,transport,trucks,tsp,turnandopen,tyreworld,visitall,woodworking,zenotravel}]
[rest ...]
Generates a problem file for a given domain and a set of hyperparameters. If the generator
also produces a domain file (e.g., ADL domain compiled to strips), it is also supported. In
the default behavior, it generates the file(s) to a specified location, then print the
absolute pathname of the problem file and the domain file in two separate lines. The second
output, the domain file, can be a file generated by the generator, or a pddl file that is
shared by problems.
positional arguments:
{agricola,assembly,barman,blocksworld-3ops,blocksworld-4ops,blocksworld-matching-typed,blocksworld-matching-untyped,briefcaseworld,cavediving,childsnack,citycar,crewplanning,data-network,depots,driverlog,elevators,ferry,floortile,freecell,fridge,goldminer,grid,gripper,grippers,hanoi,hiking,logistics,maintenance,matchcellar,miconic-fulladl,miconic-simpleadl,miconic,movie,mprime,mystery,nomystery,npuzzle,nurikabe,openstacks,parking,pathways,pegsol,rovers,satellite,scanalyzer,schedule,snake,sokoban,spanner,spider,storage,termes,tetris,tidybot,tms,tpp,transport,trucks,tsp,turnandopen,tyreworld,visitall,woodworking,zenotravel}
domain name. (default: None)
rest Remaining command line arguments for each domain. (default: None)
optional arguments:
-h, --help show this help message and exit
--output-directory DIRNAME, -d DIRNAME
Directory to output the generated files. If an absolute pathname is
given, it is used as the output directory. Otherwise, the path is
interpreted as a relative pathname to the domain directory. If the
directory does not exist, it creates it first (including the
parents). (default: output)
--output FILENAME, -o FILENAME
If present, override the default name handling and store the output
to the specified file. If the pathname is absolute, it is used as
is.If the pathname is relative, it is relative to the directory
specified by --output-directory.It defaults to a file under the
directory specified by --output-directory, in which casethe file name
defaults to {hash}.pddl , where {hash} is a md5 hash of the
parameters given to each generator. If the generator also generates a
compiled domain file, it is written to a file with a suffix
{basename}-domain.pddl . You can override this behavior with
--output-domain.If the argument is STDOUT, it writes to the standard
output.If the argument is STDOUT for a domain that also generates a
domain file, two outputs are separated by a separator specified by
--separator option. (default: None)
--output-domain FILENAME
See --output. (default: None)
--separator SEPARATOR
See --output. When the generator produces both a problem and a domain
file, and if the --output is specified as STDOUT, two files are
concatenated with a line containing the string specified by this
option. (default: ;; domain file)
--force If the file with the same name exist, overwrite the file. This is
usually not necessary because the file names are determined by the
md5 hash of the generator argument. (default: False)
--seed SEED, -s SEED Random seed. (default: 42)
--helpall Generate help messages for all domains. (default: False)
--list, -l List available domains. (default: False)
--debug Enable a more verbose debugging mode. (default: False)
@jendrikseipp , do you prefer if there is a separate PR just for the API proposal? (no implementations)
Please tackle a single domain in the first PR. Ideally, a domain where not all parameter combinations are allowed. Then we can see whether a centralized design is helpful or whether it's better to improve the generators themselves. Also, I'd like to keep the generator dirs in the top-level.
I am afraid you still don't get the aim and the scope of this PR. Currently this repository is just a collection of generators put in a single place without a common API and that is what I fix in this PR. I make this repository more usable and that is the only purpose of this PR. Besides, this repo is not pip loadable and does not fit in the typical python workflow.
I understand a parameter validation feature is useful, but that is clearly the next step and not this step.
As I said earlier, I can separate the PR into just a API proposal, removing any domain-related commits. The priority of reviewing should be on the API itself.
By the way, I have already started using the PR branch in my project and it is quite useful already.
this is a work-in-progress PR which I made to make it easy to follow the update. The idea is to have a central init file which can generate any domain with any hyperparameter, also controlling for the output pathname naming convention.
todo: