KHR-3HV - fix coordinate system

[KelvinYang0320]

Resubmit KHR-3HV in https://github.com/aidudezzz/deepworlds/pull/49 with FLU/ENU and refactoring.

• [x] Conversion to FLU/ENU
• [x] Removed KHR-3HV tutorial and added a simple README
• [x] Restructured the directories
• [x] Code Refactoring
  • [x] Removed unused imports
  • [x] Fixed typos and grammatical errors
  • [x] Custom PEP8
  • [x] line limit: 120 characters
  • [x] Naming Convention
  • [x] utilities.py→KHR3HV_env.py

[KelvinYang0320]

@tsampazk Should I move KHR-3HV readme.md to deepbots-tutorials?

[tsampazk]

@tsampazk Should I move KHR-3HV readme.md to deepbots-tutorials?

Yes, @KelvinYang0320, you can move it there and i'll fix some of its issues on the tutorials repo

[KelvinYang0320]

@tsampazk Since we have moved KHR-3HV tutorial to deepbots-tutorials, should I remove it and create another simple and short README?

[tsampazk]

Yes @KelvinYang0320 that sounds great.

[tsampazk]

Along with that, you can restructure the directory to be similar to other example directories in deepworlds, instead of the tutorials directory structure

[KelvinYang0320]

@tsampazk Hi, there! There are still some warning messages when running this example. SB3: Ray:

Also, I am not sure if it converges well since it does not converge in my experiments and this reward-episode plot looks not good enough. There might be some problem with these lines. self.motorPositionArr[i] += ac is not synchronized with self.motorList[i].setPosition(ac).

I think we can merge this PR first and open another issue/PR to address these problems since this PR is mainly for conversion to FLU/ENU.

[KelvinYang0320]

@tsampazk Thank you for the detailed review. I update the to-do list and convert this PR to draft.

[KelvinYang0320]

@tsampazk I have resolved all conversations and opened a new training issue for this example so that we can focus on the coordinate system and coding style in this PR.

[KelvinYang0320]

@tsampazk There are still a few warning messages:

• SB3

  /home/test/.local/lib/python3.8/site-packages/stable_baselines3/common/env_checker.py:272: UserWarning: We recommend you to use a symmetric and normalized Box action space (range=[-1, 1]) cf https://stable-baselines3.readthedocs.io/en/master/guide/rl_tips.html
  warnings.warn(

  • Ray

    /home/test/.local/lib/python3.8/site-packages/ray/tune/callback.py:287: FutureWarning: Please update `setup` method in callback `<class 'ray.tune.integration.wandb.WandbLoggerCallback'>` to match the method signature in `ray.tune.callback.Callback`.
    warnings.warn(
    (PPOTrainer pid=290390) 2022-11-08 10:42:42,488 WARNING ppo.py:386 -- `train_batch_size` (64) cannot be achieved with your other settings (num_workers=1 num_envs_per_worker=1 rollout_fragment_length=200)! Auto-adjusting `rollout_fragment_length` to 64.
    (PPOTrainer pid=290390) 2022-11-08 10:42:42,488 INFO ppo.py:414 -- In multi-agent mode, policies will be optimized sequentially by the multi-GPU optimizer. Consider setting simple_optimizer=True if this doesn't work for you.
    (PPOTrainer pid=290390) 2022-11-08 10:42:42,488 INFO trainer.py:903 -- Current log_level is WARN. For more information, set 'log_level': 'INFO' / 'DEBUG' or use the -v and -vv flags.
    (RolloutWorker pid=290457) 2022-11-08 10:42:44,747  WARNING env.py:135 -- Your env doesn't have a .spec.max_episode_steps attribute. This is fine if you have set 'horizon' in your config dictionary, or `soft_horizon`. However, if you haven't, 'horizon' will default to infinity, and your environment will not be reset.

    Should we fix them in this PR?

[tsampazk]

@tsampazk There are still a few warning messages: ... Should we fix them in this PR?

If you think that they can be fixed with not a lot of effort then feel free to fix them, otherwise i feel that it is not critical to get this merged. Most glaring issues in the example have been fixed!

[KelvinYang0320]

If you think that they can be fixed with not a lot of effort then feel free to fix them, otherwise i feel that it is not critical to get this merged. Most glaring issues in the example have been fixed!

@tsampazk I think it will take a while to fix them, so I agree with you! Should I squash all commits to one commit?

[tsampazk]

Should I squash all commits to one commit?

Yes, also there seems to be some conflicts in the wbt file that need to be resolved, then we can go ahead and merge this!

[KelvinYang0320]

@tsampazk Does the rebasing look fine?:smile: