search

asieradzk / RL_Matrix

Deep Reinforcement Learning in C#
Other
58 stars 2 forks source link

How to save and load models trained using Godot #11

Open kerodkibatu opened 1 month ago

kerodkibatu commented 1 month ago

Hey there, I'm curious about how to save and load models trained using Godot. I've been experimenting with reinforcement learning algorithms, and now I'm at a point where I want to persist my trained models for future use. Can anyone guide me through the process? Any tips or examples would be greatly appreciated! Thanks in advance!

asieradzk commented 1 month ago

Hi Kerod.

Thanks for trying RL Matrix. I am in the process of making it a lot more polished and I'll be ditching binary serialiser for buffer saving.

I never exposed the saving method for RLMatrix.Godot and for plain RLMatrix if I remember correctly its deleted/commented out.

Are you using DQN or PPO? I might be able to publish something today for you. But there will be small breaking changes.

kerodkibatu commented 1 month ago

Thanks a lot for the prompt response! I'm currently using PPO.

asieradzk commented 1 month ago

Thanks a lot for the prompt response! I'm currently using PPO.

Thats a shame. I'm a bit behind on this one. Let's see what I can do.

kerodkibatu commented 1 month ago

No worries at all! I understand that these things take time, and I'm really grateful for your willingness to help out.

I just wanted to reiterate how much I admire what you're doing with RL Matrix. It's a fantastic tool for the C# RL community, and I'm excited to see how it evolves.

Count me in for support—I'd be more than happy to contribute examples and functionality to help push the project forward.

asieradzk commented 3 weeks ago

Just an update that I have commited an updated version. Its not on nuget yet

For you this means massive performance improvements for both PPO & DQN. Also RNN now works as intended and I am satisfied with it.

I have exposed save/load methods temporarily in the TrainingManagerBaseDiscrete so you'll have to find opportune time to access it (in between frame updates) or edit it yourself - let me know if you need help.

You may have to change some of your implementations as IEnvironment as been replaced with Async variant to enable some weird DRL experiments...