search

ntnu-arl / ORACLE

(Visually-attentive) Uncertainty-aware navigation method using deep neural networks
BSD 3-Clause "New" or "Revised" License
66 stars 5 forks source link

Some questions related to Flightmare Environment #4

Closed duy12i1i7 closed 5 months ago

duy12i1i7 commented 6 months ago

Hello guys!! I'm here I'm a student trying to test this project. After all the bugs, I finally ran this project-related seVAE and demo on the Flightmare environment. However, I have some questions about the process I am trying to test this project. Firstly, when I enabled all of the functions to visual images(at the simulation_oracle.launch file), I saw that it has 2 screens that display RGB images. It may be from the same environment, but it doesn't have the same angle of view. Is this issue arising because of an incorrect configuration on my part, or is it an inherent feature of the application?

Screenshot from 2024-03-13 19-14-51

Another question is that when I try to test your AI model, everything is OK when I try to test Demo 1 and 2. But when I try to test Demo 4, every time I try, the drone is always colliding, and it has the same parameter when it collides. Is this issue arising because of an incorrect configuration on my part, or is it an inherent feature of the application?

Screenshot from 2024-03-13 19-15-10

Thanks a lot!!!

HuanNguyenARL commented 6 months ago

Hi @duy12i1i7, 1) The viewing angle of the window rpg_flightmare is not the same as the onboard camera. This is the inherent feature of the original agile_autonomy repo. We modified the original repo to suit our needs. 2) For the demo 4, you can reduce this threshold a bit (for example COLLISION_THRESHOLD = 0.01-0.08). Additionally, you can increase the gain of the yaw controller (for example K_yaw=2.2-2.4) to make the drone track the command better. You can verify the result of the network prediction by looking at the markers in rviz. Empirically, we found that the current weights of ORACLE (Demo 3) can perform better than the current weights of seVAE-ORACLE (Demo 4) in Flightmare. It is noted that these 2 networks are not fine-tuned at all in Flightmare.

duy12i1i7 commented 6 months ago

Hi @HuanNguyenARL, Thanks a lot for your answer. I'd like to ask you one more question about the Fightmare environment, I am eager to learn how to create a similar realistic simulation environment based on Flightmare. I just found and did it based on the flightmare_unity repo; however, this is unsuitable for your project.

HuanNguyenARL commented 6 months ago

Hi @duy12i1i7, thank you for your interest in our work. To use Flightmare with our work, I can see 2 ways: