[work in progress] Big main

guicho271828 commented 2 years ago

this is a work-in-progress PR which I made to make it easy to follow the update. The idea is to have a central init file which can generate any domain with any hyperparameter, also controlling for the output pathname naming convention.

todo:

[x] global init file and API for each generator
support for individual domain
[x] agricola
[x] assembly
[x] barman
[x] blocksworld
[x] briefcaseworld
[x] cavediving
[x] childsnack
[x] citycar
[x] crewplanning
[x] data-network
[x] depots
[x] driverlog
[x] elevators
[x] ferry
[x] floortile
[x] freecell
[x] fridge
[x] goldminer
[x] grid
[x] gripper
[x] grippers
[x] hanoi
[x] hiking
[x] logistics
[x] maintenance
[x] matchcellar
[x] miconic
[x] miconic-fulladl
[x] miconic-simpleadl
[x] movie
[x] mprime
[x] mystery
[x] nomystery
[x] npuzzle
[x] nurikabe
[x] openstacks
[x] parking
[x] pathways
[x] pegsol
[x] rovers
[x] satellite
[x] scanalyzer
[x] schedule
[x] snake
[x] sokoban
[x] spanner
[x] spider
[x] storage
[x] termes
[x] tetris
[x] tidybot
[x] tms
[x] tpp
[x] transport
[x] trucks
[x] tsp
[x] turnandopen
[x] tyreworld
[x] visitall
[x] woodworking
[x] zenotravel

haz commented 2 years ago

I'd like to see this added to planutils when it's set, and then an easy transition to the new remote planning service about to be released.

guicho271828 commented 2 years ago

planutil is just for conveniently calling planners, correct?

haz commented 2 years ago

Nope. View it as a sort of "apt" for planning-like software. So, yes, easy install of planners, but also libraries, utilities, etc.

jendrikseipp commented 2 years ago

Hi Masataro, before you start hacking away: this PR plans to address many issues at the same time (parameter specifications, task filtering, task transformation, batch task creation, uniform output filenames). I think some of these deserve their own PR.

Also, task filtering and task transformation feel like candidates for external postprocessing tools to me. I think it's best to have them as standalone tools.

For batch task creation, I think we can eventually move the code from https://github.com/rleap-project/batch-pddl-generator into this repo. Your __main__,py file has a "-n" parameter, but how would you choose the n instances?

That leaves mainly the parameter specification issue. Which use case do you envision here? If the goal is to get good error messages for an invalid parameter combination, I think the best approach is to change the generator code and add argument checking and error messages there. If the goal is to present the valid parameter ranges to users, I think the same applies: add or improve the existing help messages in the generator code. This would definitely be helpful for users.

guicho271828 commented 2 years ago

task filtering and task transformation

This is not an immediate task and I do not plan to do it for now. I just made a placeholder.

https://github.com/rleap-project/batch-pddl-generator

This is a HUGE reinvention of wheel and I do not think worth pursuing. I would simply use GNU parallel. For example,

parallel pddl-generator blocksworld-3ops --seed {1} -blocks {1}::: {0..4} ::: {0..3}

guicho271828 commented 2 years ago

ah, didn't catch the part that you use the SMAC hyperparamter tuner. The part I am making will be the interface to be called from SMAC then.

guicho271828 commented 2 years ago

That leaves mainly the parameter specification issue. Which use case do you envision here? If the goal is to get good error messages for an invalid parameter combination, I think the best approach is to change the generator code and add argument checking and error messages there. If the goal is to present the valid parameter ranges to users, I think the same applies: add or improve the existing help messages in the generator code. This would definitely be helpful for users.

I don't pursue these non-trivial goals. I simply make this set of generators usable and streamlined. Nothing new, simply refine and improve. Currently I have to manually look into each generator, learn the usage from README, and eventually write code that does the job I want. I did it for 5 domains for my recent paper. The result of this PR will be a generator with a common API that is well documented in a centralized manner.

jendrikseipp commented 2 years ago

Alright. In any case, I suggest to start with a minimal solution and to only tackle one domain in the first PR. Ideally, a domain where not all parameter combinations are allowed. Then we can see whether a centralized design is helpful or whether it's better to improve the generators themselves.

jendrikseipp commented 2 years ago

I see you're hard at work here Masataro. Let me reiterate though: I won't be able to review the changes for ~60 domains. If you want to eventually have your work merged into this repo, you need to tackle a single domain in its own PR.

guicho271828 commented 2 years ago

it is just about 3 hours of burst work ... It is nothing. btw, I want to show you how the resulting experience would be like. Please take a look at this asciinema. https://asciinema.org/a/0dS4eHPc3D8y4DGybouogrn9j

guicho271828 commented 2 years ago

If you would take time to review changes (thanks!), I would recommend checking the commits tagged as [api] and [implementation]. The rest of the commits (tagged as [domain] ) are largely just copy-and-paste from pddl-generators/__template , using copy-template.sh script. There is not much to review, and I have not spent significant time in them. They are separated just for the sake of clean commit messages.

So nothing in any specific domain really matters in these commits, I treat them largely as test cases for assessing the utility of the tooling. I left the details as the room for improvement for future users.

By the way, I have not introduced a single line of change in the existing code base --- I have only moved the original files (it was necessary for being compatible with python tooling) and added several new files, no existing files are modified significantly.

guicho271828 commented 2 years ago

Current interface:

usage: pddl-generators [-h] [--output-directory DIRNAME] [--output FILENAME]
                       [--output-domain FILENAME] [--separator SEPARATOR] [--force]
                       [--seed SEED] [--helpall] [--list] [--debug]
                       [{agricola,assembly,barman,blocksworld-3ops,blocksworld-4ops,blocksworld-matching-typed,blocksworld-matching-untyped,briefcaseworld,cavediving,childsnack,citycar,crewplanning,data-network,depots,driverlog,elevators,ferry,floortile,freecell,fridge,goldminer,grid,gripper,grippers,hanoi,hiking,logistics,maintenance,matchcellar,miconic-fulladl,miconic-simpleadl,miconic,movie,mprime,mystery,nomystery,npuzzle,nurikabe,openstacks,parking,pathways,pegsol,rovers,satellite,scanalyzer,schedule,snake,sokoban,spanner,spider,storage,termes,tetris,tidybot,tms,tpp,transport,trucks,tsp,turnandopen,tyreworld,visitall,woodworking,zenotravel}]
                       [rest ...]

Generates a problem file for a given domain and a set of hyperparameters. If the generator
also produces a domain file (e.g., ADL domain compiled to strips), it is also supported. In
the default behavior, it generates the file(s) to a specified location, then print the
absolute pathname of the problem file and the domain file in two separate lines. The second
output, the domain file, can be a file generated by the generator, or a pddl file that is
shared by problems.

positional arguments:
  {agricola,assembly,barman,blocksworld-3ops,blocksworld-4ops,blocksworld-matching-typed,blocksworld-matching-untyped,briefcaseworld,cavediving,childsnack,citycar,crewplanning,data-network,depots,driverlog,elevators,ferry,floortile,freecell,fridge,goldminer,grid,gripper,grippers,hanoi,hiking,logistics,maintenance,matchcellar,miconic-fulladl,miconic-simpleadl,miconic,movie,mprime,mystery,nomystery,npuzzle,nurikabe,openstacks,parking,pathways,pegsol,rovers,satellite,scanalyzer,schedule,snake,sokoban,spanner,spider,storage,termes,tetris,tidybot,tms,tpp,transport,trucks,tsp,turnandopen,tyreworld,visitall,woodworking,zenotravel}
                        domain name. (default: None)
  rest                  Remaining command line arguments for each domain. (default: None)

optional arguments:
  -h, --help            show this help message and exit
  --output-directory DIRNAME, -d DIRNAME
                        Directory to output the generated files. If an absolute pathname is
                        given, it is used as the output directory. Otherwise, the path is
                        interpreted as a relative pathname to the domain directory. If the
                        directory does not exist, it creates it first (including the
                        parents). (default: output)
  --output FILENAME, -o FILENAME
                        If present, override the default name handling and store the output
                        to the specified file. If the pathname is absolute, it is used as
                        is.If the pathname is relative, it is relative to the directory
                        specified by --output-directory.It defaults to a file under the
                        directory specified by --output-directory, in which casethe file name
                        defaults to {hash}.pddl , where {hash} is a md5 hash of the
                        parameters given to each generator. If the generator also generates a
                        compiled domain file, it is written to a file with a suffix
                        {basename}-domain.pddl . You can override this behavior with
                        --output-domain.If the argument is STDOUT, it writes to the standard
                        output.If the argument is STDOUT for a domain that also generates a
                        domain file, two outputs are separated by a separator specified by
                        --separator option. (default: None)
  --output-domain FILENAME
                        See --output. (default: None)
  --separator SEPARATOR
                        See --output. When the generator produces both a problem and a domain
                        file, and if the --output is specified as STDOUT, two files are
                        concatenated with a line containing the string specified by this
                        option. (default: ;; domain file)
  --force               If the file with the same name exist, overwrite the file. This is
                        usually not necessary because the file names are determined by the
                        md5 hash of the generator argument. (default: False)
  --seed SEED, -s SEED  Random seed. (default: 42)
  --helpall             Generate help messages for all domains. (default: False)
  --list, -l            List available domains. (default: False)
  --debug               Enable a more verbose debugging mode. (default: False)

guicho271828 commented 2 years ago

@jendrikseipp , do you prefer if there is a separate PR just for the API proposal? (no implementations)

jendrikseipp commented 2 years ago

Please tackle a single domain in the first PR. Ideally, a domain where not all parameter combinations are allowed. Then we can see whether a centralized design is helpful or whether it's better to improve the generators themselves. Also, I'd like to keep the generator dirs in the top-level.

guicho271828 commented 2 years ago

I am afraid you still don't get the aim and the scope of this PR. Currently this repository is just a collection of generators put in a single place without a common API and that is what I fix in this PR. I make this repository more usable and that is the only purpose of this PR. Besides, this repo is not pip loadable and does not fit in the typical python workflow.

I understand a parameter validation feature is useful, but that is clearly the next step and not this step.

As I said earlier, I can separate the PR into just a API proposal, removing any domain-related commits. The priority of reviewing should be on the API itself.

guicho271828 commented 2 years ago

By the way, I have already started using the PR branch in my project and it is quite useful already.

AI-Planning / pddl-generators

[work in progress] Big main #6