search

aikkala / user-in-the-box

GNU General Public License v3.0
32 stars 8 forks source link

Timeline for camera support #13

Open M-Colley opened 1 year ago

M-Colley commented 1 year ago

Hey there, very cool work!

I saw that you mentioned that you want to support two cameras for input, is there any timeline for adding this functionality?

Kind regards

aikkala commented 1 year ago

This is not something we're currently working on, unfortunately. Adding more cameras to the environment (i.e. cameras that are not used to produce observations) is easy, but adding cameras to produce observations is more complicated (like having one camera per eye). The latter is because the code currently expects that there is only one "vision" observation in the observation dictionary that is input to the policy. If you need this feature and are willing to work on modifying the code/architecture such that multiple visual observations are supported, I'm happy to provide some help and pointers on how it could be achieved.