IBVS Controller not tracking object

[elmtam]

Hi could any get any assistance regarding when switching ''ON'' the Controller but the drone does not track the desired object although the tracked object is indicated as ''IN FRAME'' and ''TRACKER-ON'' as well as Controller is indicated as ''ON'' . Any guidance and help is much appreciated.

Also, is it possible to share the source code for the DroneIBVSController.cpp here?

Again, thanks in advance for the time

[elmtam]

Also it is worth mentioning that the Controller when switched ''ON' causes the drone not to respond to any commands whatsoever when the StateEstimator is taken out of the launch file as suggested in issue #7

The only commands that is able to be executed at that time is taking off the drone and landing the drone from the interface. The desired object to track is recognised by the GUI but no responds whatsoever from the drone relative to the tracked object.

[jespestana]

Can you fly the AR Drone manually using the console interface? Look at the interface commands in "move mode" here: https://github.com/Vision4UAV/cvg_ardrone2_ibvs#interaction-with-ui

On Tue, May 3, 2016 at 7:10 AM, Muhammad Tareeq Ali Muhammad Taufiq < notifications@github.com> wrote:

Also it is worth mentioning that the Controller when switched ''ON' causes the drone not to respond to any commands whatsoever when the StateEstimator is taken out of the launch file as suggested in issue #7 https://github.com/Vision4UAV/cvg_ardrone2_ibvs/issues/7

The only commands that is able to be executed at that time is taking off the drone and landing the drone from the interface. The desired object to track is recognised by the GUI but no responds whatsoever from the drone relative to the tracked object.

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/Vision4UAV/cvg_ardrone2_ibvs/issues/8#issuecomment-216436269

[elmtam]

I have experimented with that quite a bit but to no success. With the State Estimator taken out of the launch file, the drone states ' command ignore' on whatever commands except the take off and land. On the image window, it shows that ''TRACKER-ON & CONTROLLER-ON'' and it is picking up the image but not moving with respect to the tracked object only hovering in a stationary state.

With the State Estimator in the launch, the drone is still not moving with respect to the track object. When on the image window, it shows that ''TRACKER-ON & CONTROLLER-ON'' and it is again picking up the image but not moving with respect to the tracked object at all. When IBVS Controller is then switched on, the drone moves erratically flying not in the direction of the tracked object.

I have looked at the DroneIBVSController.cpp and I have tried to make some adjustments to:

line 437 pitchco_hf = pid_fs.getOutput();

and also

line 452 dyawco = 0.0

But again no success. Any help? Is it possible to see the source code of the IBVS Controller here?

Thank you for your time. Appreciate it. @jespestana

[jespestana]

You have all the source code, about usages just stick to the license.

Obviously some part of the code expected the State Estimator to be started. But I do not remember the exact logic right now. I can assure you that it is not used for control.

I think that the best would be that you track down the bug that is not allowing the State Estimator to start in your computer. Then everything should work. From the error message, it might be that the program cannot find the configuration file.

On Tue, May 3, 2016 at 7:38 AM, Muhammad Tareeq Ali Muhammad Taufiq < notifications@github.com> wrote:

I have experimented with that quite a bit but to no success. With the State Estimator taken out of the launch file, the drone states ' command ignore' on whatever commands except the take off and land. On the image window, it shows that ''TRACKER-ON & CONTROLLER-ON'' and it is picking up the image but not moving with respect to the tracked object only hovering in a stationary state.

With the State Estimator in the launch, the drone is still not moving with respect to the track object. When on the image window, it shows that ''TRACKER-ON & CONTROLLER-ON'' and it is again picking up the image but not moving with respect to the tracked object at all. When IBVS Controller is then switched on, the drone moves erratically flying not in the direction of the tracked object.

I have looked at the DroneIBVSController.cpp and I have tried to make some adjustments to:

line 437 pitchco_hf = pid_fs.getOutput();

and also

line 452 dyawco = 0.0

But again no success. Any help? Is it possible to see the source code of the IBVS Controller here?

Thank you for your time. Appreciate it. @jespestana https://github.com/jespestana

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/Vision4UAV/cvg_ardrone2_ibvs/issues/8#issuecomment-216442987

[elmtam]

Hi @jespestana

Yes I have managed to make the errors from issue #7 to go away. The launch file is executing the StateEstimator without any errors as done previously but still when I switched on the controller, the drone does not respond with respect to the tracked object. I have calibrated the cameras again. I have installed ar drone autnomy in a seperate workspace, I think I have done it all but to no avail. I have looked through the codes for the IBVSController but to no success.

I have even tried to make a simple experiment as trying to get the drone to just turn slightly left or right autonomously when i move the object left or right respectively but the drone is not responding to such movements.

[jespestana]

Let's not concentrate on the controller for now. Can you try the "move mode" using the console interface? Can you fly the drone manually using the keys in the interface?

Can you fly the AR Drone manually using the console interface? Look at the interface commands in "move mode" here: https://github.com/Vision4UAV/cvg_ardrone2_ibvs#interaction-with-ui

On Tue, May 3, 2016 at 7:54 AM, Muhammad Tareeq Ali Muhammad Taufiq < notifications@github.com> wrote:

Hi @jespestana https://github.com/jespestana

Yes I have managed to make the errors from issue #7 https://github.com/Vision4UAV/cvg_ardrone2_ibvs/issues/7 to go away. The launch file is executing the StateEstimator without any errors as done previously but still when I switched on the controller, the drone does not respond with respect to the tracked object. I have calibrated the cameras again. I have installed ar drone autnomy in a seperate workspace, I think I have done it all but to no avail. I have looked through the codes for the IBVSController but to no success.

I have even tried to make a simple experiment as trying to get the drone to just turn slightly left or right autonomously when i move the object left or right respectively but the drone is not responding to such movements.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/Vision4UAV/cvg_ardrone2_ibvs/issues/8#issuecomment-216444424

[elmtam]

Yes I can the drone responds to the move mode and all the commands as stated here:

https://github.com/Vision4UAV/cvg_ardrone2_ibvs#interaction-with-ui

@jespestana

[jespestana]

Ok.

Can you tell us the exact sequence of console interface commands / OpenTLD interface clicks that you follow to start the control architecture?

On Tue, May 3, 2016 at 8:01 AM, Muhammad Tareeq Ali Muhammad Taufiq < notifications@github.com> wrote:

Yes I can the drone responds to the move mode and all the commands as stated here:

https://github.com/Vision4UAV/cvg_ardrone2_ibvs#interaction-with-ui

@jespestana https://github.com/jespestana

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/Vision4UAV/cvg_ardrone2_ibvs/issues/8#issuecomment-216445396

[elmtam]

1) On the terminal, I execute the launch file using,

${IBVS_STACK}/launchers/parrot_IBVSController_launcher_Release.sh

2) 4 seperate terminals are executed: i) TLD Gui ii) Image showing the real time movements iii) Terminal consisting of the 9 launchfile iv) Terminal consisting the Drone interface & openTLD Gui for IBVS controller

3) I then use the TLD Gui window to select the desired object with the boudning box, click ENTER

4) I move the tracked object ensuring that it is IN FRAME all the time for about 15- 30 seconds allowing it to learn

5) I then click on the TOGGLE LEARNING to swicth off learn

6) I then go back to the drone interface ensuring all the time the drone is still tracking the object ''TRACKER-ON IN FRAME''

7) At the drone interface, i click ''t'' to allow the drone to takeoff

8) Then click ''o'' to start the controller

9) Drone flies away erratically not keeping a fixed distance on the tracked object

@jespestana

[jespestana]

9) Drone flies away erratically not keeping a fixed distance on the tracked object

Where are you performing the experiments? What is the size of the room where you are flying?

In the TLD image window, there are two drawn squares of two different colors. What is the relative size and color of these squares? (Note: there is a compile constant in the code that sets the distance to the object. However, it sets the reference size of the object on the image; not a real distance in m)

On Tue, May 3, 2016 at 8:24 AM, Muhammad Tareeq Ali Muhammad Taufiq < notifications@github.com> wrote:

1) On the terminal, I execute the launch file using,

${IBVS_STACK}/launchers/parrot_IBVSController_launcher_Release.sh

2) 4 seperate terminals are executed: i) TLD Gui ii) Image showing the real time movements iii) Terminal consisting of the 9 launchfile iv) Terminal consisting the Drone interface & openTLD Gui for IBVS controller

3) I then use the TLD Gui window to select the desired object with the boudning box, click ENTER

4) I move the tracked object ensuring that it is IN FRAME all the time for about 15- 30 seconds allowing it to learn

5) I then click on the TOGGLE LEARNING to swicth off learn

6) I then go back to the drone interface ensuring all the time the drone is still tracking the object ''TRACKER-ON IN FRAME''

7) At the drone interface, i click ''t'' to allow the drone to takeoff

8) Then click ''o'' to start the controller

9) Drone flies away erratically not keeping a fixed distance on the tracked object

@jespestana https://github.com/jespestana

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/Vision4UAV/cvg_ardrone2_ibvs/issues/8#issuecomment-216448000

[elmtam]

I have done this almost everywhere.

Indoors:

1) In a well lit spacious room approx 45x45 2) In a normal lit natural light room 3) Tried on both normal flooring, carpeted (blue) and carpeted (red) 4) Normal corridor

Outdoors:

1) Spacious Field 2) Long Road 3) Open compound ( minimal wind disturbances)

As for the two drawn squares, I have tested it to be:

1) Both the same size 2)Blue one smaller than the green one 3) Blue one slightly smaller than the green one

@jespestana

[elmtam]

Also @jespestana

I have restarted it again and this just popped up..

[elmtam]

This is an example of the box sizes I usually use whenever testing the drone @jespestana

[jespestana]

About the boxes shown in the image: One of the boxes shows the currently detected object. The other one is shown overlayed on the detected object, showing the "reference size" of the object in the image. When both boxes have the same size, this should be the reference distance in m for the current visual servoing operation. You can set the reference size in the image with the compile constanst

define FS_REFERENCE (I don't remember exactly the name)

Any of those flying space should do, as long as there is not too much wind. I would probably start from the indoors space.

Try using the AR Drone 2 box as a target, as I did in this video: https://www.youtube.com/watch?v=K9uKbp_KEbs

The OpenTLD tracker works really well with that box. Also start with a target that has a reference distance at around 2m (you have to test until the target box and the reference box have the same size at the desired distance).

On Tue, May 3, 2016 at 8:47 AM, Muhammad Tareeq Ali Muhammad Taufiq < notifications@github.com> wrote:

This is an example of the box sizes I usually use whenever testing the drone @jespestana https://github.com/jespestana [image: e00fa47a-1102-11e6-8e51-b16bef4500b8] https://cloud.githubusercontent.com/assets/9165681/14976400/533e8ba0-1103-11e6-976a-47efd4972fbf.png

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/Vision4UAV/cvg_ardrone2_ibvs/issues/8#issuecomment-216452650

[elmtam]

@jespestana I have done all that as explained above but I have told you the main problem, as soon as I switch on the controller, the drone is not following the tracked object.

I am sure that the tests I have done confirms what you are asking me to do but why is the drone not responding to the tracked object?

[elmtam]

@jespestana how do i get the drone to maneuver and track the object as shown in your videos? Is the state estimator causing the problem?

[jespestana]

The steps you are following seem correct to me. The only thing that I would try is starting the state estimator pressing 'l' (step 8b in your list). Again, I would suggest using the AR Drone box as first target. You don't even need to take it, just put it on top of a table and check that the drone is able to hover in front of it with the architecture started.

It is hard to help you with the debugging remotely. Also, I cannot test the code myself right now. The last time I tested it, it worked in Ubuntu 14.04 (late 2014), and it was working quite well. My colleagues in Madrid are still using this controller, but with some modifications. I don't know if the public code for the Visual Servoing is already there, but this should be the updated repo (it has much more functionalities than the visual servoing): https://github.com/Vision4UAV/Aerostack

On Tue, May 3, 2016 at 9:18 AM, Muhammad Tareeq Ali Muhammad Taufiq < notifications@github.com> wrote:

@jespestana https://github.com/jespestana how do i get the drone to maneuver and track the object as shown in your videos?

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/Vision4UAV/cvg_ardrone2_ibvs/issues/8#issuecomment-216456454

[elmtam]

And what can i do regarding this error?

I'll do the test more thoroughly later but I do hope im able to debug it as how youve managed to make it work