Open HosseinSheikhi opened 3 years ago
You would have to implement your own pybind wrapper in order to get the image using Python and train vision-based policy.
Specifically, you have to 1) create a new environment or modify the quadrotor_env.hpp file. attach a RGB camera on the quadrotor. 2) retrieve the image using the camera 3) modify the step function of the environment and such that it returns the image to python, instead of low level states. https://github.com/uzh-rpg/flightmare/blob/master/flightlib/src/envs/quadrotor_env/quadrotor_env.cpp#L93
I wonder if someone helps me how I can run an RL algorithm by feeding vision as input in flightmare.