An Open Source package that allows video game creators, AI researchers and hobbyists the opportunity to learn complex behaviors for their Non Player Characters or agents
MIT License
902
stars
63
forks
source link
Fix inference when action names are not alphabetic #159
One way to test this is to try e.g. current version of FlyBy (may need to update plugin before testing). For me, when I try the onnx file, the result doesn't work unless I swap the order of the two actions.
After the changes, here is a quick test with the onnx working properly after retraining the agent for ~50_000 steps using the SB3 example:
The is fine for training and Python inference since it sends a dictionary containing action names and values back to Godot, but during onnx inference the only information about action order available is from the action space as defined in gdscript. In envs where that was already sorted, there weren't any issues with onnx, but as I tried training an env with action names ordered differently, I noticed the onnx wasn't working correctly.
This should fix those cases, changing to OrderedDict seems to remove the conversion and sorting, so the order sent from Godot is preserved and should be what onnx inference expects.
Fixes onnx inference when the action names are not sorted, together with this PR on the plugin side: https://github.com/edbeeching/godot_rl_agents_plugin/pull/27
Related issue: https://github.com/edbeeching/godot_rl_agents_plugin/issues/25
One way to test this is to try e.g. current version of FlyBy (may need to update plugin before testing). For me, when I try the onnx file, the result doesn't work unless I swap the order of the two actions.
After the changes, here is a quick test with the onnx working properly after retraining the agent for ~50_000 steps using the SB3 example:
https://github.com/edbeeching/godot_rl_agents/assets/61947090/0471ab23-e8d5-4719-b356-15fe2dce41f9
(The extra actions and unnecessary dimensions were added just for debugging)
As for what causes the issue, during conversion to
spaces.Dict, the items of the dictionary were getting sorted while converted to OrderedDict internally: https://github.com/edbeeching/godot_rl_agents/blob/ac7241b17a3b3001964adb3e5dcb33e6c037442d/godot_rl/core/godot_env.py#L332The is fine for training and Python inference since it sends a dictionary containing action names and values back to Godot, but during onnx inference the only information about action order available is from the action space as defined in gdscript. In envs where that was already sorted, there weren't any issues with onnx, but as I tried training an env with action names ordered differently, I noticed the onnx wasn't working correctly.
This should fix those cases, changing to OrderedDict seems to remove the conversion and sorting, so the order sent from Godot is preserved and should be what onnx inference expects.