system_random_seed not exist in Julia Arcade Learning Environment

[allenanie]

Hi Cassidy,

I'm trying to rerun your experiments (or at least the MDP tabular representation collection part). Everything works out great and thanks for the amazingly detailed documentation. However, I ran into a Julia error (My Julia version is 1.9.1). The Julia ALE package version is 0.6.0. Package installation dependency says:

ArcadeLearningEnvironment v0.2.4

The error comes from https://github.com/cassidylaidlaw/effective-horizon/blob/main/EffectiveHorizon.jl/src/atari.jl#L31

setInt(env.ale, "system_random_seed", 4753849)

The error is:

julia> setInt(ale, "system_random_seed", 4753849)
terminate called after throwing an instance of 'std::runtime_error'
  what():  The key system_random_seed you are trying to set does not exist.

Do you have any clue what this might be? Should I downgrade Julia to 1.8 instead?

[allenanie]

I did a clean installation of Julia 1.8.5 and all packages. I don't think it fixed anything.

[allenanie]

Commenting out this line setInt(env.ale, "system_random_seed", 4753849) works and allows me to store MDPs.

Although this might lead to discrepancy between tabular MDP results vs. Deep RL through RLLIb.

[cassidylaidlaw]

Thanks for your interest in the code! Setting that system_random_seed is important to get exact determinism for some of the Atari environments which is why that line of code is there. Unfortunately, the ALE version where the authors introduced the system_random_seed option is also when they stopped supporting the C API (they switched to just providing C++), which the Julia ArcadeLearningEnvironment package depends on. I ended up custom compiling a version of the ALE C library which includes the system_random_seed option. To install it in place of the one from the Julia package, run

sudo cp -v EffectiveHorizon.jl/libale_c.so $(julia --project=EffectiveHorizon.jl -e 'using Libdl, ArcadeLearningEnvironment; print(dlpath(ArcadeLearningEnvironment.libale_c))')

I should have included this in the docs but forgot—will update them now. Let me know if this fixes the issue!

[allenanie]

Hi cassidy, thank you so much for the prompt response! The code you provided copies libale_c.so from your repo to override the julia package! So I did:

sudo cp EffectiveHorizon.jl/libale_c.so /home/t-allennie/.julia/artifacts/0cff36dca1973ee2546cd26c4db6c402064b9e4a/lib/

To make sure this override happened.

After this, I did the same procedure as before:

using ArcadeLearningEnvironment
ale = ALE_new()
setInt(ale, "system_random_seed", 4753849)

The error is still there, but here's the error trace:

terminate called after throwing an instance of 'std::runtime_error'
  what():  The key system_random_seed you are trying to set does not exist.

setInt at /home/t-allennie/.julia/artifacts/a6dbb39ee59c02a61d8a860d3ed7fb7875299408/lib/libale_c.so (unknown line)
setInt at /home/t-allennie/.julia/packages/ArcadeLearningEnvironment/7UVtT/src/aleinterface.jl:75
unknown function (ip: 0x7fd3c555e3e3)

I can see that it's correctly pointing at the file that I overrode.

[cassidylaidlaw]

Hmm, that actually looks like it's a different libale_c.so between the first line where you copy in my version and where the error appears. That is, one is in ~/.julia/artifacts/0cff... and one is in ~/.julia/artifacts/a6db.... What if you do

cp EffectiveHorizon.jl/libale_c.so /home/t-allennie/.julia/artifacts/a6dbb39ee59c02a61d8a860d3ed7fb7875299408/lib/libale_c.so

[allenanie]

It worked!! Thank you very much!

I think there's a weird discrepancy between the command you provided:

sudo cp -v EffectiveHorizon.jl/libale_c.so $(julia --project=EffectiveHorizon.jl -e 'using Libdl, ArcadeLearningEnvironment; print(dlpath(ArcadeLearningEnvironment.libale_c))')

I get 'EffectiveHorizon.jl/libale_c.so' -> '/home/t-allennie/.julia/artifacts/0cff36dca1973ee2546cd26c4db6c402064b9e4a/lib/libale_c.so'

And running these commands in an interactive julia console:

using ArcadeLearningEnvironment
using Libdl
print(dlpath(ArcadeLearningEnvironment.libale_c))

I get /home/t-allennie/.julia/artifacts/a6dbb39ee59c02a61d8a860d3ed7fb7875299408/lib/libale_c.so (which is the correct path)

Not sure what caused it or why, but kinda interesting :)

Thanks for all the help and debugging!

[cassidylaidlaw]

Hi @windweller—if you're still interested in the project, I just wanted to let you know that the full BRIDGE dataset of tabular representations is now available here: https://zenodo.org/record/8226192. Hope it's helpful!

[allenanie]

Thanks for making it available!! :)