ModuleNotFoundError: No module named 'meltingpot.python'

[elliottower]

GitHub workflow with this error: https://github.com/Farama-Foundation/Shimmy/actions/runs/4441939882/jobs/7797630023?pr=39

I've gotten this error a few times when running the code locally in the past few weeks, and am not sure exactly what I did to fix it. It has worked in the past with our github workflows and docker images but now seems to be broken. I am using it to get the list of substrates and call substrate.get_config() and substrate.build(): https://github.com/elliottower/Shimmy/blob/main/tests/test_meltingpot.py#L10

Being outward facing functions intended to be used to initialize environments I would think these should definitely not raise errors. Thinking maybe if the init.py explicitly imported them it may fix the problem?

I am able to import meltingpot itself perfectly fine but when I try to call functions in the submodule meltingpot.python it either gives an error that meltingpot.python is not found or if I try to import it then it will sometimes fail as I've said above. I don't think it's actually stochastic it's just due to me changing the dockerfiles/environment over time to try and work out other unrealted issues, and some of those configurations evidently making this import fail.

[jagapiou]

I don't think I can be much help here I'm afraid.

I suspect that if you put import meltingpot.python ... in meltingpot/__init__.py you'd have the same issue. But I guess try it and see.

My guess is it's something to do with your PYTHONPATH or what directory you are running you are running the tests from. This may help? https://stackoverflow.com/questions/54895002/modulenotfounderror-with-pytest

[elliottower]

I found this previous issue (https://github.com/deepmind/meltingpot/issues/14) and then in the README there's a note about the pythonpath variable, although I think it's best practice to extend it rather than overwrite it: https://askubuntu.com/a/250935

[elliottower]

I think the real problem though is that the python directory doesn't seem to be accessible when you install the package (sometimes?), for example when I use the same exact code from the dockerfile, my /workspaces/meltingpot/ directory only has subfolders named meltingpot and then a folder called saved_models. I guess I can get around this by cloning the repository directly and placing it in the appropriate folder but I think given other people have also had this meltingpot.python issue it's probably something to do with the project structure (maybe needs to be added to init.py somewhere?).

[elliottower]

I was able to get it to work by adding these lines to the dockerfile:

# Download assets
RUN mkdir -p /workspaces/meltingpot/meltingpot && \
  curl -SL https://storage.googleapis.com/dm-meltingpot/meltingpot-assets-2.1.0.tar.gz \
  | tar -xz --directory=/workspaces/meltingpot/meltingpot

# Clone meltingpot repository
RUN git clone https://github.com/deepmind/meltingpot.git
RUN cp -r /meltingpot/ /workspaces/meltingpot/ && rm -R /meltingpot/

# Install meltingpot dependencies
WORKDIR /workspaces/meltingpot/meltingpot/

RUN pip install .

# Set Python path for meltingpot
ENV PYTHONPATH=${pwd}

As a random note, the base requirement for tensorflow should probably be moved to an optional requirement, as the README shows how you can install tensorflow or pettingzoo as optional dependencies, but when you run pip install -e . it still installs tf.

[jagapiou]

Glad you got it working :-)

FYI tensorflow is a required dep for meltingpot as the Bots in the Scenarios are tf.SavedModels.

[elliottower]

Glad you got it working :-)

FYI tensorflow is a required dep for meltingpot as the Bots in the Scenarios are tf.SavedModels.

Ah okay, makes sense. And the shimmy meltingpot integration is now merged, will finish updating the SB3 tutorial PR that I opened here in the next few days. I added a page with a quick summary about meltingpot and the GIF from this repo, and a quick usage example of how to do the conversion and run an environment: https://shimmy.farama.org/contents/meltingpot/