alex-petrenko
commented
1 year ago @derekwin enjoy module provides all of the facilities for the deployment of the trained model, including loading model from the checkpoint and running inference. https://github.com/alex-petrenko/sample-factory/blob/master/sample_factory/enjoy.py
If you want to just observe/evaluate a trained model, you can use this script directly (there are plenty of example command lines in the documentation).
If this needs to be a part of the larger system, I suggest that you copy-paste and modify this file for your needs.
The choice to make the normalizer a part of the model is deliberate: a model trained with certain input normalization parameters won't work without the normalizer. Hence the normalizer parameters are actually saved into a checkpoint alongside the parameter of the model. If this doesn't work for your usecase, one option is to turn the normalization off and retrain (it's an optional feature).
Please let me know if this summary works for you! Happy to help you further
derekwin
Is it possible for sample-factory to provide a convenient way to provide model services like rllib？like this https://docs.ray.io/en/latest/serve/tutorials/rllib.html
I try to implement the neural network model by myself and load the trained parameters from the generated checkpoint. But I noticed that the sample-factory itself has a lot of parameter settings, and there are optimizations such as "Observation normalization", so it seems a bit difficult to implement model serving.
So I would like to ask if there is a good way to implement the deployment of the model.
Thank you very much!