Ray.rllib.ModelsV2 problem

[paulkim436]

Hello. It seems like I am experiencing the same problem as the thread above #6 The latest version of Ray has changed the Model.py file to ray.rllib.models.ModelV2.py. In response, I tried changing the code for smac.examples.rllib.model.py as shown in the code attached below.

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import tensorflow as tf

from ray.rllib.models import modelv2 #changed model to modelv2
from ray.rllib.models.tf.misc import normc_initializer

class MaskedActionsModel(modelv2): #also changed model to modelv2
    def _build_layers_v2(self, input_dict, num_outputs, options):
        action_mask = input_dict["obs"]["action_mask"]
        if num_outputs != action_mask.shape[1].value:
            raise ValueError(
                "This model assumes num outputs is equal to max avail actions",
                num_outputs,
                action_mask,
            )

However, this raises another error.

"TypeError: module() takes at most 2 arguments (3 given)". Is there a fix for this?

[benellis3]

Sorry -- we don't really use the ray/rllib interface. It also has not been tested, however if you swap out the Starcraft2Env for the StarCraftCapabilityEnvWrapper you should be able to get it to work as that class has the same API as the original version of SMAC.

If you do manage to get it to work, please submit a pull request with the required changes.

[benellis3]

If this is still an issue let me know -- closing this for now.