[question] How to Use --env-kwargs correctly?

[toksis]

How do you use --env-kwargs correctly?

I have this code in a custom environment.

custom_env = gym.make('forex-v0',
               df = FOREX_EURUSD_1H_ASK,
               window_size = 10,
               frame_bound = (10, 300),
               unit_side = 'right')

let us say I want to use the df = FOREX_EURUSD_1H_ASK

--env-kwargs df = FOREX_EURUSD_1H_ASK post an error.

Traceback (most recent call last):
  File "train.py", line 79, in <module>
    args = parser.parse_args()
  File "C:\anaconda\envs\zootesting\lib\argparse.py", line 1755, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "C:\anaconda\envs\zootesting\lib\argparse.py", line 1787, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "C:\anaconda\envs\zootesting\lib\argparse.py", line 1993, in _parse_known_args
    start_index = consume_optional(start_index)
  File "C:\anaconda\envs\zootesting\lib\argparse.py", line 1933, in consume_optional
    take_action(action, args, option_string)
  File "C:\anaconda\envs\zootesting\lib\argparse.py", line 1861, in take_action
    action(self, namespace, argument_values, option_string)
  File "E:\ML\reinforcementlearning\zoo\rl-baselines-zoo\utils\utils.py", line 394, in __call__
    arg_dict[key] = eval(value)
  File "<string>", line 0

[araffin]

Hello,

Please look at the README example: https://github.com/araffin/rl-baselines-zoo#env-keyword-arguments

it is key_name:value but in your case you want to pass a python variable? If that variable is not defined in the code, this won't work...

[toksis]

"In the code" means train.py? or the environment code? Also can --env-kwargs accomodate multiple arguments?

thanks

[araffin]

"In the code" means train.py?

in train.py otherwise how do you want to retrieve the variable value if it is not in the scope?

Also can --env-kwargs accomodate multiple arguments?

yes, you can use --env-kwargs key1:value1 key2:value2...

[toksis]

I have tried multiple key.

--env-kwargs=df:'FOREX_EURUSD_1H_ASK' frame_bound:(50, 100) window_size:10

I encountered an error here.

in utils.py

values
["df:...w_size=10 ']
special variables
function variables
0:"df:'FOREX_EURUSD_1H_ASK' frame_bound:(50, 100) window_size=10 "
len():1

    def __call__(self, parser, namespace, values, option_string=None):
        arg_dict = {}
        for arguments in values:
            key = arguments.split(":")[0]
            value = ":".join(arguments.split(":")[1:])
            # Evaluate the string as python code
            arg_dict[key] = eval(value)
        setattr(namespace, self.dest, arg_dict)

key will retrieve the value: df

value will retrieve the value: 'FOREX_EURUSD_1H_ASK' frame_bound:(50, 100) window_size:10

    args = parser.parse_args()
  File "C:\anaconda\envs\zootesting\lib\argparse.py", line 1755, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "C:\anaconda\envs\zootesting\lib\argparse.py", line 1787, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "C:\anaconda\envs\zootesting\lib\argparse.py", line 1993, in _parse_known_args
    start_index = consume_optional(start_index)
  File "C:\anaconda\envs\zootesting\lib\argparse.py", line 1933, in consume_optional
    take_action(action, args, option_string)
  File "C:\anaconda\envs\zootesting\lib\argparse.py", line 1861, in take_action
    action(self, namespace, argument_values, option_string)
  File "e:\ML\reinforcementlearning\zoo\rl-baselines-zoo\utils\utils.py", line 394, in __call__
    arg_dict[key] = eval(value)
  File "<string>", line 1
    'FOREX_EURUSD_1H_ASK' frame_bound:(50, 100) window_size=10
                                    ^
SyntaxError: invalid syntax

Is it a bug?

[araffin]

everything you pass to the command line must be a string... so, in your case frame_bound:'(50,100)'

[toksis]

I tried to do it.

key: df value: 'FOREX_EURUSD_1H_ASK' frame_bound:'(50, 100)' window_size:'10'

It fails on

arg_dict[key] = eval(value)

But if the value is just 'FOREX_EURUSD_1H_ASK' It will be an ok.

[caburu]

@toksis could you inform the entire command line you have used?

In a previous comment you have used an unexpected equal sign = after --env-kwargs

I have tried multiple key.

--env-kwargs=df:'FOREX_EURUSD_1H_ASK' frame_bound:(50, 100) window_size:10

That should be:

--env-kwargs df:'FOREX_EURUSD_1H_ASK' frame_bound:'(50, 100)' window_size:10

[toksis]

@caburu sorry for that. I am using Launch.json in Visual Studio Code to add the arguments.

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Current File",
            "justMyCode":false,
            "type": "python",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "args": ["--algo=acktr","--env=forex-v0","-n 50000","-optimize","--n-trials=1000 ","--n-jobs=2",
                     "--sampler=tpe","--pruner=median","--env-kwargs=df:'FOREX_EURUSD_1H_ASK' frame_bound:'(50, 100)' window_size:'10' "]
        }
    ]
}

[araffin]

@toksis Did you solve your issue?

I tried reproducing it but couldn't...

What I did:

python train.py --algo a2c --env CartPole-v1 --env-kwargs a:'TEST' b:'(3,4)' c:5

and I just printed the parsed arguments as CartPole does not accept those:

print(arg_dict)
# outputs: {'a': 1, 'b': (3, 4), 'c': 5} because I defined TEST = 1 in `utils/utils.py`

Note: I defined the variable TEST in utils/utils.py so it is in the right scope.

[caburu]

@toksis I believe the problem is yet with the equal sign.

The other command line parameters has only one value. For example: for --algo, in a terminal, you could use --algo=acktr or --algo acktr, and both it would work. But for --env-kwargs you need to pass more than one value, so you can't use the equal sign.

If you were using a terminal you would only need to replace the equal sign by space character. But this will not work with Visual Studio Code because when there is a space inside the arg, VS code put quotes around the arg. So I believe the solution is to separate the args.

Replace this line:

                     "--sampler=tpe","--pruner=median","--env-kwargs=df:'FOREX_EURUSD_1H_ASK' frame_bound:'(50, 100)' window_size:'10' "]

by this:

                     "--sampler=tpe","--pruner=median","--env-kwargs", "df:'FOREX_EURUSD_1H_ASK'", "frame_bound:'(50, 100)'", "window_size:'10'"]

And tell us if it works.

[toksis]

Thank you for your help @araffin and @caburu cabu. Sorry for the delay, but I did this directly after waking up hehe.

To this:

From this:

The way I put the argument in Launch.json is the culprit. It works now!

[ZdM87]

I have a keyword argument that is an object called net from a class Network. When I import my class and instantiate the object net = Network() in train.py and then I tried with --env-kwargs network:'net' but it does not work. Any idea what is the problem. It says NameError: name 'net' is not defined. It was at rl_zoo3/utils.py in the line arg_dict[key] = eval(value).