[Bug]: Installation difficulties because of `arch==5.3.0` and `envpool` dependencies

[dominikonysz]

🐛 Bug

Hello,

I'm having problems properly installing and using RLLTE. The fixed arch==5.3.0 dependency makes it impossible to install it using Python 3.11. Changing it to arch>=5.3.0 resolves that issue and installs it without errors (Maybe the limitation is not required anymore). I used the reward branch to install with pip install -e ..

However, even after installing correctly no reward module (from rllte.xplore.reward) can be import due to the missing envpool dependency. Simply installing envpool is not possible as neither pip nor conda have it available. Installing rllte with [envs] fails for the same reason.

Is there some workaround for these issues or would it be possible to somehow remove the dependency for envpool if only needing the reward modules? It doesn't seem like it required for that part of the code

To Reproduce

• Clone repository and checkout the reward branch
• Setup environment with Python 3.11: conda create -n rllte python=3.11
• Change arch==5.3.0 to arch>=5.3.0 in pyproject.toml
• Install with pip install -e .
• Run python:

  from rllte.xplore.reward import ICM

Relevant log output / Error message

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\Onyszkiewicz\Documents\workspace\rllte\rllte\xplore\reward\__init__.py", line 27, in <module>
    from .icm import ICM as ICM
  File "C:\Users\Onyszkiewicz\Documents\workspace\rllte\rllte\xplore\reward\icm.py", line 35, in <module>
    from rllte.common.prototype import BaseReward
  File "C:\Users\Onyszkiewicz\Documents\workspace\rllte\rllte\common\prototype\__init__.py", line 26, in <module>
    from .base_agent import BaseAgent as BaseAgent
  File "C:\Users\Onyszkiewicz\Documents\workspace\rllte\rllte\common\prototype\base_agent.py", line 47, in <module>
    from rllte.common.type_alias import VecEnv
  File "C:\Users\Onyszkiewicz\Documents\workspace\rllte\rllte\common\type_alias.py", line 36, in <module>
    from rllte.env.utils import Gymnasium2Torch, GymObs
  File "C:\Users\Onyszkiewicz\Documents\workspace\rllte\rllte\env\__init__.py", line 26, in <module>
    from .testing import make_bitflipping_env as make_bitflipping_env
  File "C:\Users\Onyszkiewicz\Documents\workspace\rllte\rllte\env\testing\__init__.py", line 26, in <module>
    from .box import make_box_env as make_box_env
  File "C:\Users\Onyszkiewicz\Documents\workspace\rllte\rllte\env\testing\box.py", line 33, in <module>
    from rllte.env.utils import Gymnasium2Torch
  File "C:\Users\Onyszkiewicz\Documents\workspace\rllte\rllte\env\utils.py", line 28, in <module>
    import envpool
ModuleNotFoundError: No module named 'envpool'

System Info

No response

Checklist

• [X] I have checked that there is no similar issue in the repo
• [X] I have read the documentation
• [X] I have provided a minimal working example to reproduce the bug
• [X] I've used the markdown code blocks for both code and stack traces.

[yuanmingqi]

We will publish a new version this week. Maybe you can just wait two more days.

[dominikonysz]

Sounds great, looking forward to it!

[yuanmingqi]

We've published the update.

If you have any other questions, please don't hesitate to ask here.

@dominikonysz

[dominikonysz]

Great, I'll have a look at it. Thank for you the notification

[dominikonysz]

Hey, two small questions came up:

1. Is there a specific reason for the fixed version dependency for packages such as matplotlib? Another package for example required matplotlib>=3.7.1 and then the matplotlib==3.6.0 dependency clashed
2. Dict observation spaces are not supported for e.g. ICM yet, correct? Is there any plan to support them in the future?

[yuanmingqi]

• it's totally okay if you don't install the matplotlib, just clone the repo and use it
• yes, actually you can just replace the encoder with the RaffinCombinedEncoder from rllte.xploit.encoder. Also, you need to clone the repo and change the code youself first. We will add the feature later.

[dominikonysz]

Yes alright, thank you! So far it is working with some adjustments so that is perfectly fine for me.

I'll close this issue for now since this isn't related to the original problem anymore. Cheers