How to use model

[AdrianSkib]

Hi.

First of all thank you so much for providing this repo. This model opens up so many possibilities, but only if it's open source.

From the paper I understand that the output of the model is either a string of words, a string of words with an action, or just an action, depending on the output mode. The action consists of a token indicating whether to finish previous action before starting new, changes in x, y, z, rotations around x, y and z, and a desired extension of the end effector (gripper).

The example provided runs smoothly, but I'm unsure how to convert the eval_logits to the above output format.

Could you provide an example of this final step?

Thanks a lot in advance!

Upvote & Fund

• We're using Polar.sh so you can upvote and help fund this issue.
• We receive the funding once the issue is completed & confirmed by you.
• Thank you in advance for helping prioritize & fund our backlog.

[kyegomez]

@AdrianSkib this outputs just textual tokens it does not output the action sequences, if you would like to make this happen you can integrate token learner

https://zeta.apac.ai/en/latest/zeta/nn/modules/token_learner/