new robot training

[lianuo]

Thank you for your excellent work. If I want to train my own robot, what do I need to prepare

1. Add your own robot's URDF file
2. Modify the code to include it in robots. py
3. Modify the model_ Descriptions.yaml file
4. Generate Dataset
5. Training Model
6. Validate the model I have tried the above steps, but there are still many things that seem incorrect. Can you give me some guidance?

[cbames]

Can you provide more detail on the errors that you’ve encountered?

Sent from my iPad

On May 3, 2023, at 3:06 AM, lianuo @.***> wrote:

 Thank you for your excellent work. If I want to train my own robot, what do I need to prepare

Add your own robot's URDF file Modify the code to include it in robots. py Modify the model_ Descriptions.yaml file Generate Dataset Training Model Validate the model I have tried the above steps, but there are still many things that seem incorrect. Can you give me some guidance? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.

[jstmn]

Hi @lianuo,

Happy to help with this. First of all, what robot is this for? I have some additional trained models not in this repo at the moment so there's a chance I have a trained model already that you can use.

The correct order of steps are:

1. Create a new Robot subclass in jrl / robots.py (see https://github.com/jstmn/Jrl/blob/master/jrl/robots.py). This will involve adding a urdf file, and setting the active and inactive joints and end effector. BTW I can help you get this permenantly added to the jrl repo so that others can use it.
2. "Generate Dataset". Try: python scripts/build_dataset.py --robot_name=fetch --training_set_size=25000000 --only_non_self_colliding
3. "Training Model": A good hyperparameter set to try is python scripts/train.py --robot_name=<your new robots name> --nb_nodes=12 --batch_size=512 --learning_rate=0.00025. This step will likely take some hyperparameter tuning which I can help out with.
4. "Validate the model.". See scripts/evaluate.py, scripts/visualize.py

"Modify the model_ Descriptions.yaml file" this is only required once a final trained model is deployed/made public. You can use this snippet to load a local model file:

hyper_parameters = IkflowModelParameters()
ik_solver = IKFlowSolver(hyper_parameters, robot)
ik_solver.load_state_dict(model_weights_filepath)

• Jeremy

[lianuo]

hi @jstmn @cbames Thank you for your quick replies.

The robot I use is Reachy, an open source platform for home service robots, and related information can be found on this website. https://www.pollen-robotics.com/

I am trying the way you said, and I will sort out the problems I encountered later and list them together.

[newuhe]

Hi @lianuo,

Happy to help with this. First of all, what robot is this for? I have some additional trained models not in this repo at the moment so there's a chance I have a trained model already that you can use.

The correct order of steps are:

1. Create a new Robot subclass in jkinpylib / robots.py (see https://github.com/jstmn/jkinpylib/blob/master/jkinpylib/robots.py). This will involve adding a urdf file, and setting the active and inactive joints and end effector. BTW I can help you get this permenantly added to the jkinpylib repo so that others can use it.
2. "Generate Dataset". Try: python scripts/build_dataset.py --robot_name=fetch --training_set_size=25000000 --only_non_self_colliding
3. "Training Model": A good hyperparameter set to try is python scripts/train.py --robot_name=<your new robots name> --nb_nodes=12 --batch_size=512 --learning_rate=0.00025. This step will likely take some hyperparameter tuning which I can help out with.
4. "Validate the model.". See scripts/evaluate.py, scripts/visualize.py

"Modify the model_ Descriptions.yaml file" this is only required once a final trained model is deployed/made public. You can use this snippet to load a local model file:

hyper_parameters = IkflowModelParameters()
ik_solver = IKFlowSolver(hyper_parameters, robot)
ik_solver.load_state_dict(model_weights_filepath)

• Jeremy

Hi, the problem is how to define collision capsule as in other robots when adding new robot.

[jstmn]

Ah, yep that was a recent change and isn't documented anywhere.

@newuhe what robot/urdf is this for? Might be easier for me to do it on my end - would you be comfortable having your URDF added to the Jrl library? https://github.com/jstmn/Jrl

If not I can try and help you get the code running yourself

[newuhe]

Ah, yep that was a recent change and isn't documented anywhere.

@newuhe what robot/urdf is this for? Might be easier for me to do it on my end - would you be comfortable having your URDF added to the Jrl library? https://github.com/jstmn/Jrl

If not I can try and help you get the code running yourself

Thanks for replying! Here is urdf and mesh file.urdf.zip

[jstmn]

hey @newuhe I finished adding your robot to the jrl repo (commit here) so you should be good to start some training runs. Let me know if you need any help with this

[newuhe]

hey @newuhe I finished adding your robot to the jrl repo (commit here) so you should be good to start some training runs. Let me know if you need any help with this

Thanks a lot for your help! Since other kinds of new robots may need to be tested, is there any plan to put collision sphere approximation related in the repo?

[jstmn]

Can you share more about this? is there some functionality you get with sphere's vs. capsules?

or is the issue that there aren't DIY instructions for creating capsules?

[jstmn]

oh btw I've uploaded a partly trained rizon4 model that you can use: rizon4__snowy-brook-208__global_step=2.75M

you can test it out with python scripts/evaluate.py --testset_size=500 --model_name=rizon4__snowy-brook-208__global_step=2.75M

[newuhe]

Can you share more about this? is there some functionality you get with sphere's vs. capsules?

or is the issue that there aren't DIY instructions for creating capsules?

To clarify the issue is lack of instructions for creating capsules.

[newuhe]

oh btw I've uploaded a partly trained rizon4 model that you can use: rizon4__snowy-brook-208__global_step=2.75M

you can test it out with python scripts/evaluate.py --testset_size=500 --model_name=rizon4__snowy-brook-208__global_step=2.75M

Thanks. I'll have a try.

[jstmn]

the script is here: https://github.com/jstmn/Jrl/blob/master/scripts/calculate_capsule_approximation.py

Its a somewhat complicated processes to properly add a new robot and I don't have time to simplify it so I don't think I'll add any official instructions any time soon