omni-us / jsonargparse

Implement minimal boilerplate CLIs derived from type hints and parse from command line, config files and environment variables
https://jsonargparse.readthedocs.io
MIT License
322 stars 47 forks source link

Using relative config files in dicts and lists raises `Unexpected import path format` #557

Open AlejandroBaron opened 2 months ago

AlejandroBaron commented 2 months ago

Using relative config files in dicts doesn't work. It might be that I'm not using the correct syntax tho

To reproduce

3 files in the same directory

1) cli.py

from jsonargparse import CLI
from typing import Dict, List

class MyObject:

    def __init__(self, x: int) -> None:
        self.x = x

class MyCLI:

    def __init__(self, d: Dict[str, MyObject]) -> None:
        self.d = d

CLI(MyCLI)

(If you use a List instead of Dict for d param, same error happens)

2) main.yaml

d:
  obj1: ./my_obj.yaml

3) my_obj.yaml

class_path: cli.MyObj
init_args:
  x: 3

Command: python cli.py --config main.yaml. Raises

error: Parser key "d":
  Unexpected import path format: ./my_obj.yaml

As I said it might not be a bug. I'm using python 3.8, but in the project that triggered this error I'm using python 3.9.

mauvilsa commented 2 months ago

This is not a bug. Not everything can be parsed as a subconfig. A relative path to a subconfig inside another config only works if the parser has an action that would load it. Subconfigs are only available for top-level classes in a typehint, not nested like in a dict. Generally you can see which subconfigs can be loaded by printing the --help.

If not a bug, could this be added as a new feature? Might be, though most likely it would be technically quite complex and be low priority.

AlejandroBaron commented 2 months ago

This is not a bug. Not everything can be parsed as a subconfig. A relative path to a subconfig inside another config only works if the parser has an action that would load it. Subconfigs are only available for top-level classes in a typehint, not nested like in a dict. Generally you can see which subconfigs can be loaded by printing the --help.

If not a bug, could this be added as a new feature? Might be, though most likely it would be technically quite complex and be low priority.

Yeah I assumed that it was not a bug but just not supported right now. Is there a way/plan to use resolvers like omegaconf does for this kind of situation?

mauvilsa commented 2 months ago

omegaconf resolvers are able to load subconfigs? I don't see that here, but I am not an expert on that. If it is supported, then you can make jsonargparse to use omegaconf as the yaml loader.