the ppo_baselines train problem

[Suvian-wy]

I'm sorry to trouble you. And thank you for providing this platform. But I have some questions. Could you help me?

When I run the ppo_baselines_train.py, it work well, but when the train end. I see the flight lines use plot_flight.py.I find the profromance is bad. The ppo1-gymfc_nf-step-v1-10003968.ckpt profromance just like in the picture. I don't know if the train is enough or not. Or may the reward should be negated?

Another problem when I run thetf_checkpoint_evaluate.py, it will not stop. So how to fix this bug?

Thanks!

[Suvian-wy]

When I change a seed . I get a better result like in fig. And I want to know how to contiune train in the base of the weight which has being train. So could you help me. But the second problem also occur.

[Suvian-wy]

I want to use the continuous env to test the model, but when I use gym.make('gymfc_nf-continuous-v1'), the error No module named 'neuroflight_trainer', I don't know where to find the file. Could you help me?

Thanks!

[wil3]

Hi @Suvian-wy I'm not really sure what you are asking for help for "And I want to know how to contiune train in the base of the weight which has being train." Can you describe this in more detail. Are you asking how to resume training from another checkpoint? My fork of OpenAI baselines has a restore_dir you can look at for restoring a checkpoint. This was meant to recover from a failure. I can't remember its state though, I don't think I ever had to use it but that's where you'd want to look.

If you run python3 ppo_baselines_train.py --gym-id gymfc_nf-continuous-v1 you get that error? There's no other output? Can you provide the exact command and output you get?

[Suvian-wy]

The first problem just as you said. I want to train it from a checkpoint! Thank you! And for the second problem, just like in the picture! when I run the command python ppo_baselines_train.py --gym-id gymfc_nf-continuous-v1, just tell me No module named 'neuroflight_trainer'

So I find in the file 'continous.py', the 'neuroflight_trainer.gyms.rewards' is used, but I can't where the file or dir exit.

[wil3]

Yep that's a bug left over from the refactor, good catch. In continuous.py importing RewardEnv should be the same as here, https://github.com/wil3/gymfc/blob/master/examples/gymfc_nf/envs/step.py#L8.

Please consider opening a PR to fix this issue.

[Suvian-wy]

Yeah, I have opening a PR to fix this issue. But I have another problem. Why the trained model of step env can't be used in the continous env? The targets of two env are same, all to train a controller to control the angle_rate. So in my mind, I think if the model proform well in step env, it should be also proform well in continous env. However, that doesn't seem to be the case.

[Suvian-wy]

I have write a continous_test.py to verify my mind, the result is well. The mode trained in step-env also proform well in continous-env. The profromance in the picture.

But when I train model incontinous-env base on a pre-model trianed in step-env. The model profromance go alone the a bad direction, it become bad.

So next step I will trian model in continous-env without a pre-model to verify the problem.

[Suvian-wy]

I have trained a model just use continous-env, and used the model to test in step-env, It also proformed well. So in this issue, I have no question. I have a another problem want ask for help, so I will open a new issue.

Thanks!

[NahomGebremekel]

Hi Suvian-wy. I would like to know how you were able to get that graph. My graph is different from yours. Can you help me?

Thanks!

[Suvian-wy]

@NahomGebremekel　　I don't know whick graph in your words, could you say in detail.