visualization interface

[fromlimbo]

Hi，great to see your work, btw how do I create a training visualization interface when I run python run.py or train somthing else

[yufeiwang63]

Hi,

For manipulation tasks,

• For RL: we use ray for RL training when learning a skill. The log of ray learning will be stored at data/local/ray_results/{task_name}_{time_stamp}. You can check that folder for visualizations of the reward curves. For example, you can plot episode_reward_mean field in progress.csv to show the episode reward during training. I think ray also uses tensorboard to store the results in events.out.tfevents*, so you can probably also use tensorboard for visualization (I didn't personally try this though). The RL learning end results will be stored at {path_to_your_task}/RL_sac/{time_stamp}/{substep_name}, e.g., data/generated_tasks/close_the_oven_door_Oven_101940_2023-09-21-22-28-23/task_close_the_oven_door/RL_sac/2023-09-22-02-05-05/close_the_oven_door. A gif showing the execution of the learned RL policy, the simulation states during the policy execution, and the best policy weights will be stored.
• For action primitives: there is not much to visualize here since the action primitive are just sample-based motion planners. Also all primitives should run pretty fast. The results of the primitives will be stored at {path_to_your_task}/primitive_states/{time_stamp}/{substep_name}, e.g., data/generated_tasks/close_the_oven_door_Oven_101940_2023-09-21-22-28-23/task_close_the_oven_door/primitive_states/2023-10-06-03-00-07/grasp_the_oven_door. A gif showing the execution of the primitive, as well as all the simulation states during the primitive execution will be stored.

For locomotion tasks, we use CEM to solve it. It should be pretty fast. After it finishes, the results should be stored at {path_to_your_task}/cem/, e.g., data/generated_tasks/locomotion_2023-10-30-17-43-31/task_Turn_right/cem/. A mp4 showing the results and all simulation states during the execution will be stored.

[lz-azure904]

Great stuff! The locomotion learning example works!

python execute_locomotion.py --task_config_path example_tasks/task_Turn_right/Turn_right.yaml

However, when I tried to run this:

python execute.py --task_config_path example_tasks/Change_Lamp_Direction/Change_Lamp_Direction_The_robotic_arm_will_alter_the_lamps_light_direction_by_manipulating_the_lamps_head.yaml

It had the following errors:

Traceback (most recent call last):
  File "/data/programs/RoboGen/execute.py", line 272, in <module>
    execute(task_config_path, resume=args.resume, training_algo=args.training_algo, time_string=args.time_string,
  File "/data/programs/RoboGen/execute.py", line 192, in execute
    rgbs, states = execute_primitive(task_config_path, solution_path, substep, last_restore_state_file, save_path,
  File "/data/programs/RoboGen/execute.py", line 14, in execute_primitive
    env, safe_config = build_up_env(task_config, solution_path, task_name, last_restore_state_file,
  File "/data/programs/RoboGen/manipulation/utils.py", line 155, in build_up_env
    module = importlib.import_module("{}.{}".format(solution_path.replace("/", "."), task_name))
  File "/data/programs/miniconda3/envs/robogen/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/data/programs/RoboGen/example_tasks/Change_Lamp_Direction/task_Change_Lamp_Direction/grasp_the_lamps_head.py", line 4, in <module>
    from manipulation.gpt_primitive_api import *
  File "/data/programs/RoboGen/manipulation/gpt_primitive_api.py", line 5, in <module>
    from manipulation.motion_planning_utils import motion_planning
  File "/data/programs/RoboGen/manipulation/motion_planning_utils.py", line 2, in <module>
    import pybullet_ompl.pb_ompl as pb_ompl
  File "/data/programs/RoboGen/pybullet_ompl/pb_ompl.py", line 9, in <module>
    from ompl import base as ob
  File "/data/programs/RoboGen/ompl-1.5.2/py-bindings/ompl/base/__init__.py", line 1, in <module>
    from ompl import util
  File "/data/programs/RoboGen/ompl-1.5.2/py-bindings/ompl/util/__init__.py", line 4, in <module>
    from ompl.util._util import *
ModuleNotFoundError: No module named 'ompl.util._util'

After reinstalling the impl several times and even swamped the VM a couple of time; still the same error. here is my /data/programs/miniconda3/envs/robogen/lib/python3.9/site-packages/ompl.pth

/data/programs/RoboGen/ompl-1.5.2/py-bindings

Not sure what next steps on this and would be much appreciated if you could have a look!

[koupoo]

Encountered the same problem and took some time to solve it, and found that the version of pypy3 was incorrect , just upgrage pypy3 and compile ompl again:

sudo apt-get remove pypy3
sudo add-apt-repository ppa:pypy/ppa
sudo apt-get install pypy3
sudo apt update sudo apt install pypy3
pypy3 -V
Python 3.9.18 (7.3.13+dfsg-1~ppa1~ubuntu20.04, Oct 17 2023, 11:00:05)
[PyPy 7.3.13 with GCC 9.4.0]

[lz-azure904]

Thx, @koupoo will give a try and keep u posted!