Issue with seeding

[gaarsmu]

Hi there! Thanks for your hints on YouTube I got the game installing, but trying to import the environment I got this error.

I for gym installed as gymansium.

Command:

import retro 
env = retro.make('SonicTheHedgehog-Genesis')

Result:

AttributeError                            Traceback (most recent call last)
Cell In [10], line 1
----> 1 env = retro.make('SonicTheHedgehog-Genesis')

File ~/anaconda3/envs/t_rl/lib/python3.10/site-packages/retro/__init__.py:55, in make(game, state, inttype, **kwargs)
     53     else:
     54         raise FileNotFoundError('Game not found: %s. Did you make sure to import the ROM?' % game)
---> 55 return RetroEnv(game, state, inttype=inttype, **kwargs)

File ~/anaconda3/envs/t_rl/lib/python3.10/site-packages/retro/retro_env.py:131, in RetroEnv.__init__(self, game, state, scenario, info, use_restricted_actions, record, players, inttype, obs_type)
    129 elif record is not False:
    130     self.auto_record(record)
--> 131 self.seed()
    132 if gym_version < (0, 9, 6):
    133     self._seed = self.seed

File ~/anaconda3/envs/t_rl/lib/python3.10/site-packages/retro/retro_env.py:214, in RetroEnv.seed(self, seed)
    210 self.np_random, seed1 = seeding.np_random(seed)
    211 # Derive a random seed. This gets passed as a uint, but gets
    212 # checked as an int elsewhere, so we need to keep it below
    213 # 2**31.
--> 214 seed2 = seeding.hash_seed(seed1 + 1) % 2**31
    215 return [seed1, seed2]

AttributeError: module 'gym.utils.seeding' has no attribute 'hash_seed'

[onaclov2000]

I honestly don't have any clue what the "replacement" for hash_seed was supposed to be in gym, however I just took the old version of gym, and copied the function (and a few other required things) and monkey patched them in you can try that? Though I don't know what the "right" solution is (I'm still running into issues with the newer versions of gym).

import gym
from typing import Optional 
import hashlib
import struct

# TODO: don't hardcode sizeof_int here
def _bigint_from_bytes(bt: bytes) -> int:

    sizeof_int = 4
    padding = sizeof_int - len(bt) % sizeof_int
    bt += b"\0" * padding
    int_count = int(len(bt) / sizeof_int)
    unpacked = struct.unpack(f"{int_count}I", bt)
    accum = 0
    for i, val in enumerate(unpacked):
        accum += 2 ** (sizeof_int * 8 * i) * val
    return accum

def hash_seed(seed: Optional[int] = None, max_bytes: int = 8) -> int:
    """Any given evaluation is likely to have many PRNG's active at once.
    (Most commonly, because the environment is running in multiple processes.)
    There's literature indicating that having linear correlations between seeds of multiple PRNG's can correlate the outputs:
        http://blogs.unity3d.com/2015/01/07/a-primer-on-repeatable-random-numbers/
        http://stackoverflow.com/questions/1554958/how-different-do-random-seeds-need-to-be
        http://dl.acm.org/citation.cfm?id=1276928
    Thus, for sanity we hash the seeds before using them. (This scheme is likely not crypto-strength, but it should be good enough to get rid of simple correlations.)
    Args:
        seed: None seeds from an operating system specific randomness source.
        max_bytes: Maximum number of bytes to use in the hashed seed.
    Returns:
        The hashed seed
    """

    if seed is None:
        seed = create_seed(max_bytes=max_bytes)
    hash = hashlib.sha512(str(seed).encode("utf8")).digest()
    return _bigint_from_bytes(hash[:max_bytes])

gym.utils.seeding.hash_seed = hash_seed

However now I'm personally stuck on

ModuleNotFoundError: No module named 'gym.envs.classic_control.rendering

[onaclov2000]

Ok .... I think I got something working.

I added this to retro_env.py after the imports.

import hashlib
import struct
from typing import Optional 

# TODO: don't hardcode sizeof_int here
def _bigint_from_bytes(bt: bytes) -> int:

    sizeof_int = 4
    padding = sizeof_int - len(bt) % sizeof_int
    bt += b"\0" * padding
    int_count = int(len(bt) / sizeof_int)
    unpacked = struct.unpack(f"{int_count}I", bt)
    accum = 0
    for i, val in enumerate(unpacked):
        accum += 2 ** (sizeof_int * 8 * i) * val
    return accum

def hash_seed(seed: Optional[int] = None, max_bytes: int = 8) -> int:
    """Any given evaluation is likely to have many PRNG's active at once.
    (Most commonly, because the environment is running in multiple processes.)
    There's literature indicating that having linear correlations between seeds of multiple PRNG's can correlate the outputs:
        http://blogs.unity3d.com/2015/01/07/a-primer-on-repeatable-random-numbers/
        http://stackoverflow.com/questions/1554958/how-different-do-random-seeds-need-to-be
        http://dl.acm.org/citation.cfm?id=1276928
    Thus, for sanity we hash the seeds before using them. (This scheme is likely not crypto-strength, but it should be good enough to get rid of simple correlations.)
    Args:
        seed: None seeds from an operating system specific randomness source.
        max_bytes: Maximum number of bytes to use in the hashed seed.
    Returns:
        The hashed seed
    """

    if seed is None:
        seed = create_seed(max_bytes=max_bytes)
    hash = hashlib.sha512(str(seed).encode("utf8")).digest()
    return _bigint_from_bytes(hash[:max_bytes])

gym.utils.seeding.hash_seed = hash_seed

try:
    import pyglet
except ImportError as e:
    raise ImportError(
        """
    Cannot import pyglet.
    HINT: you can install pyglet directly via 'pip install pyglet'.
    But if you really just want to install all Gym dependencies and not have to think about it,
    'pip install -e .[all]' or 'pip install gym[all]' will do it.
    """
    )

try:
    from pyglet.gl import *
except ImportError as e:
    raise ImportError(
        """
    Error occurred while running `from pyglet.gl import *`
    HINT: make sure you have OpenGL installed. On Ubuntu, you can run 'apt-get install python-opengl'.
    If you're running on a server, you may need a virtual frame buffer; something like this should work:
    'xvfb-run -s \"-screen 0 1400x900x24\" python <your_script.py>'
    """
    )
from gym.utils import seeding

gym_version = tuple(int(x) for x in gym.__version__.split('.'))

__all__ = ['RetroEnv']

def get_window(width, height, display, **kwargs):
    """
    Will create a pyglet window from the display specification provided.
    """
    screen = display.get_screens()  # available screens
    config = screen[0].get_best_config()  # selecting the first screen
    context = config.create_context(None)  # create GL context

    return pyglet.window.Window(
        width=width,
        height=height,
        display=display,
        config=config,
        context=context,
        **kwargs
    )

def get_display(spec):
    """Convert a display specification (such as :0) into an actual Display
    object.
    Pyglet only supports multiple Displays on Linux.
    """
    if spec is None:
        return pyglet.canvas.get_display()
        # returns already available pyglet_display,
        # if there is no pyglet display available then it creates one
    elif isinstance(spec, str):
        return pyglet.canvas.Display(spec)
    else:
        raise error.Error(
            "Invalid display specification: {}. (Must be a string like :0 or None.)".format(
                spec
            )
        )

class SimpleImageViewer(object):
    def __init__(self, display=None, maxwidth=500):
        self.window = None
        self.isopen = False
        self.display = get_display(display)
        self.maxwidth = maxwidth

    def imshow(self, arr):
        if self.window is None:
            height, width, _channels = arr.shape
            if width > self.maxwidth:
                scale = self.maxwidth / width
                width = int(scale * width)
                height = int(scale * height)
            self.window = get_window(
                width=width,
                height=height,
                display=self.display,
                vsync=False,
                resizable=True,
            )
            self.width = width
            self.height = height
            self.isopen = True

            @self.window.event
            def on_resize(width, height):
                self.width = width
                self.height = height

            @self.window.event
            def on_close():
                self.isopen = False

        assert len(arr.shape) == 3, "You passed in an image with the wrong number shape"
        image = pyglet.image.ImageData(
            arr.shape[1], arr.shape[0], "RGB", arr.tobytes(), pitch=arr.shape[1] * -3
        )
        texture = image.get_texture()
        gl.glTexParameteri(gl.GL_TEXTURE_2D, gl.GL_TEXTURE_MAG_FILTER, gl.GL_NEAREST)
        texture.width = self.width
        texture.height = self.height
        self.window.clear()
        self.window.switch_to()
        self.window.dispatch_events()
        texture.blit(0, 0)  # draw
        self.window.flip()

Finally, later in the file:

I did the following:

#from gym.envs.classic_control.rendering import SimpleImageViewer

This allows the file to use the SimpleImageViewer from the class we stuffed at the top of retro_env.py

[onaclov2000]

Final comment, I apologize, I haven't tried installing this repo, I'm basing this on using the original "gym-retro" repo, I glanced quickly here, but didn't see an obvious "python package" install, so I just tried to fix the thing I could install using pip3 install gym-retro.

[onaclov2000]

One final comment, sorry, I just noticed you were using Sonic on the Genesis. Please let me know if Sonic works, I ran into issues with gym-retro sometime after their competition, they made changes/updates, and I can't find an old version, the sonic I was trying was failing to work after sonic moves to the right a few hundred pixels.

[onaclov2000]

To follow up, I also tried just removing the seeding code, and that ....seemed to work... (at least based on my incredibly limited testing of using Super Metroid) To be honest, I tried looking around a bit, and can't see where the seed is passed anywhere to the emulators so it just may not be needed? (Though this was middle of the night searching ;))

[MatPoliquin]

@gaarsmu stable-retro has moved to Farama fondation: https://github.com/Farama-Foundation/stable-retro

Can you recreate your issue there so that it's under your user name? I will continue to follow over there