search

trackmania-rl / tmrl

Reinforcement Learning for real-time applications - host of the TrackMania Roborace League
https://pypi.org/project/tmrl
MIT License
482 stars 67 forks source link

TMRL `unsupported operand type(s) for +: 'NoneType' and 'int'` while trying to run `--check-environment` #90

Closed vBazilevich closed 5 months ago

vBazilevich commented 5 months ago

Hello! I'm getting the following error while running --check-envoronment subcommand:

  File "/home/vladimir/.conda/envs/tmrl/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/vladimir/.conda/envs/tmrl/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/vladimir/.conda/envs/tmrl/lib/python3.10/site-packages/tmrl/__main__.py", line 84, in <module>
    main(arguments)
  File "/home/vladimir/.conda/envs/tmrl/lib/python3.10/site-packages/tmrl/__main__.py", line 63, in main
    check_env_tm20lidar()
  File "/home/vladimir/.conda/envs/tmrl/lib/python3.10/site-packages/tmrl/tools/check_environment.py", line 16, in check_env_tm20lidar
    lidar = Lidar(window_interface.screenshot())
  File "/home/vladimir/.conda/envs/tmrl/lib/python3.10/site-packages/tmrl/custom/utils/window.py", line 180, in screenshot
    monitor = {"top": self.x + self.x_offset, "left": self.y + self.y_offset, "width": self.w, "height": self.h}
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

As you could see I'm running TMRL on linux inside conda environment. I've followed readme/install_linux.md and readme/get_started.md tutorials. I've installed the xdotool as it's requrested. Yet I'm getting the error above. Here we config I got based on mentioned tutorials:

{
  "RUN_NAME": "SAC_4_LIDAR_pretrained",
  "RESET_TRAINING": false,
  "BUFFERS_MAXLEN": 500000,
  "RW_MAX_SAMPLES_PER_EPISODE": 1000,
  "CUDA_TRAINING": true,
  "CUDA_INFERENCE": false,
  "VIRTUAL_GAMEPAD": true,
  "DCAC": false,
  "LOCALHOST_WORKER": true,
  "LOCALHOST_TRAINER": true,
  "PUBLIC_IP_SERVER": "0.0.0.0",
  "PASSWORD": "==>TMRL@UseASecurePasswordHere!<==",
  "TLS": false,
  "TLS_HOSTNAME": "default",
  "TLS_CREDENTIALS_DIRECTORY": "",
  "NB_WORKERS": -1,
  "WANDB_PROJECT": "<IT'S IN HERE JUST DONT WANT TO PUBLISH IT>",
  "WANDB_ENTITY": "tmrl",
  "WANDB_KEY": "<IT'S IN HERE JUST DONT WANT TO PUBLISH IT>",
  "PORT": 55555,
  "LOCAL_PORT_SERVER": 55556,
  "LOCAL_PORT_TRAINER": 55557,
  "LOCAL_PORT_WORKER": 55558,
  "BUFFER_SIZE": 536870912,
  "HEADER_SIZE": 12,
  "SOCKET_TIMEOUT_CONNECT_TRAINER": 300.0,
  "SOCKET_TIMEOUT_ACCEPT_TRAINER": 300.0,
  "SOCKET_TIMEOUT_CONNECT_ROLLOUT": 300.0,
  "SOCKET_TIMEOUT_ACCEPT_ROLLOUT": 300.0,
  "SOCKET_TIMEOUT_COMMUNICATE": 30.0,
  "SELECT_TIMEOUT_OUTBOUND": 30.0,
  "ACK_TIMEOUT_WORKER_TO_SERVER": 300.0,
  "ACK_TIMEOUT_TRAINER_TO_SERVER": 300.0,
  "ACK_TIMEOUT_SERVER_TO_WORKER": 300.0,
  "ACK_TIMEOUT_SERVER_TO_TRAINER": 7200.0,
  "RECV_TIMEOUT_TRAINER_FROM_SERVER": 7200.0,
  "RECV_TIMEOUT_WORKER_FROM_SERVER": 600.0,
  "WAIT_BEFORE_RECONNECTION": 10.0,
  "LOOP_SLEEP_TIME": 1.0,
  "MAX_EPOCHS": 10000,
  "ROUNDS_PER_EPOCH": 10,
  "TRAINING_STEPS_PER_ROUND": 1000,
  "MAX_TRAINING_STEPS_PER_ENVIRONMENT_STEP": 4.0,
  "ENVIRONMENT_STEPS_BEFORE_TRAINING": 1000,
  "UPDATE_MODEL_INTERVAL": 1000,
  "UPDATE_BUFFER_INTERVAL": 1000,
  "SAVE_MODEL_EVERY": 0,
  "MEMORY_SIZE": 1000000,
  "BATCH_SIZE": 256,
  "ALG": {
    "ALGORITHM": "SAC",
    "LEARN_ENTROPY_COEF":false,
    "LR_ACTOR":0.00001,
    "LR_CRITIC":0.00005,
    "LR_ENTROPY":0.0003,
    "GAMMA":0.995,
    "POLYAK":0.995,
    "TARGET_ENTROPY":-0.5,
    "ALPHA":0.01,
    "REDQ_N":10,
    "REDQ_M":2,
    "REDQ_Q_UPDATES_PER_POLICY_UPDATE":20,
        "OPTIMIZER_ACTOR": "adam",
        "OPTIMIZER_CRITIC": "adam",
        "BETAS_ACTOR": [0.997, 0.997],
        "BETAS_CRITIC": [0.997, 0.997],
        "L2_ACTOR": 0.0,
        "L2_CRITIC": 0.0
  },
  "ENV": {
    "RTGYM_INTERFACE": "TM20LIDAR",
    "WINDOW_WIDTH": 958,
    "WINDOW_HEIGHT": 488,
        "IMG_WIDTH": 64,
    "IMG_HEIGHT": 64,
        "IMG_GRAYSCALE": true,
    "SLEEP_TIME_AT_RESET": 1.5,
    "IMG_HIST_LEN": 4,
    "RTGYM_CONFIG": {
      "time_step_duration": 0.05,
      "start_obs_capture": 0.04,
      "time_step_timeout_factor": 1.0,
      "act_buf_len": 2,
      "benchmark": false,
      "wait_on_done": true,
      "ep_max_length": 1000,
          "interface_kwargs": {"save_replays": false}
    },
        "REWARD_CONFIG": {
                "END_OF_TRACK": 100.0,
                "CONSTANT_PENALTY": 0.0,
                "CHECK_FORWARD": 500,
                "CHECK_BACKWARD": 10,
                "FAILURE_COUNTDOWN": 10,
                "MIN_STEPS": 70,
                "MAX_STRAY": 100.0
        }
  },
  "__VERSION__": "0.6.0"
}

I had to install keyboard explicitly. To record the reward I have to run the command with sudo. Thus, I've added a symlink to ~/TmrlData to /root to make config available.

vBazilevich commented 5 months ago

Note: I've also tried to replace config with TmrlData/resources/config_lidar.json and still get the same error.

yannbouteiller commented 5 months ago

Hmm keyboard? You shouldn't need keyboard nor running with sudo, vgamepad is supposed to work on Linux

vBazilevich commented 5 months ago

Seems like it's needed for recording of the reward. And sudo is needed because keyboard needs root access.

yannbouteiller commented 5 months ago

Oh I see, indeed, I never tried recording the reward on Linux, I forgot we were using keyboard for that.

I am trying to reinstall TrackMania with steam on my Ubuntu to reproduce your issue but the game doesn't want to launch, did you run into any issue when installing TrackMania/OpenPlanet?

yannbouteiller commented 5 months ago

(Also just to narrow it down, did you try running the default pipeline with the default config.json file and default reward in the tmrl-test track before switching to LIDAR and trying to record a custom reward function?)

vBazilevich commented 5 months ago

No, I had no issues with installation of TrackMania or OpenPlanet, I just followed the tutorials provided. However I'm using a rolling-release distribution so I might have a newer versions of some packages and/or kernel.

And yes, I've tried to test the default pipeline with default config and it worked well on a test map. However, I haven't tried to train the model.

yannbouteiller commented 5 months ago

OK, then it means that something in the LIDAR environment doesn't like Linux. I think I have only tested the default "Full" environment on Linux, I will fix this as soon as I manage to get TrackMania up and running again on my Linux.

yannbouteiller commented 5 months ago

Actually no it was a bug in the environment-checking script. It should be fixed now. I also removed the calls to keyboard on Linux in record.py so that sudo is not required anymore.

Please upgrade to version 0.6.3 and see if this fixes it for you :)

vBazilevich commented 5 months ago

Is 0.6.3 published to PyPI?

yannbouteiller commented 5 months ago

Yes I just uploaded it

vBazilevich commented 5 months ago

Yeah, seems to be working! Thank you very much for instantaneous support!

yannbouteiller commented 5 months ago

No problem, don't hesitate if you need help :grin:

Glemhel commented 5 months ago

I am trying to reinstall TrackMania with steam on my Ubuntu to reproduce your issue but the game doesn't want to launch, did you run into any issue when installing TrackMania/OpenPlanet?

Hello! I am also facing issues with launching the TrackMania on Ubuntu (22.04): it fails to lauch asking for a Ubisoft Connect. I've tried installing Ubisoft Connect and lauch via wine but it seems I failed to do so or steam does not see it. If you had similar issue, could you please tell how you managed to do it?

yannbouteiller commented 5 months ago

I am trying to reinstall TrackMania with steam on my Ubuntu to reproduce your issue but the game doesn't want to launch, did you run into any issue when installing TrackMania/OpenPlanet?

Hello! I am also facing issues with launching the TrackMania on Ubuntu (22.04): it fails to lauch asking for a Ubisoft Connect. I've tried installing Ubisoft Connect and lauch via wine but it seems I failed to do so or steam does not see it. If you had similar issue, could you please tell how you managed to do it?

I had to uninstall steam, winetricks and protontricks, and. delete the ~/.steam folder entirely.

Then I reinstalled steam via apt, I reinstalled TrackMania within steam, set the parameters of TrackMania to force launch using proton experimental, reinstalled winetricks via apt, and finally reinstalled protontricks but NOT via apt: via pipx install protontricks.

Then I installed uplay via protontricks, and the game launched properly.

After that, I closed the game, stopped it in steam because it would not stop automatically, and proceeded to installing OpenPlanet via protontricks.