Optical flow not working

[mandelyoni]

Hi,

I followed both websites: https://docs.px4.io/en/flight_controller/intel_aero.html GitHub - intel-aero/aero-optical-flow

please see version in picture attached. please also see "Optical flow Active" picture attached

I'm in position mode and, still, it's not working - drifting to the sides

Thank you

[zehortigoza]

What is the flow quality in the floor that you are flying? Check that in QGroundControl>Widgets>MAVLink Inspector>OPTICAL_FLOW_RAD>quality.

[mandelyoni]

Above 100 all the time I can send a video

[lbegani]

@mandelyoni

1. Check if the flow quality is good. To make it better you can add more texture in the floor
2. Check if the sensors are properly calibrated and you are having stable flight in manual mode
3. Check, if possible, the difference in performance flying indoors vs outdoors
4. You can capture the px4 logs and try to debug. The code is here - https://github.com/intel-aero/aero-optical-flow. The optical flow algo from PX4 is here - https://github.com/PX4/OpticalFlow

[mandelyoni]

1. Flow quality is good
2. Sensors calibrated and stable flight in manual mode

I followed all the instructions on both websites:

https://pixhawk.org/peripherals/rangefinder#lidar-lite https://docs.px4.io/en/sensor/rangefinders.html

In the Analyze and Inspector tools of QGC (see pics) the readings seam ok (may be a bit noisy though).

In the Console tool, there is a problem when typing the 'test' and 'start' commands (see pic). May be it's because the lidar is connected to the Telemetry connector (and not the Compass connector).

In addition to that, please see that the optical flow is active:

and the version:

Any ideas?

Thank you, Yoni.

[zehortigoza]

You don't need to start the lidar driver as is is being initialized in the board boot. If you had set the parameters in PX4 I don't see any reason why it is not working. Please send the PX4 log during a flight with optical-flow enabled and switch to position mode.

[mandelyoni]

Hi,

How do I record the log? Is it through the QGC or through an external sd card on the AERO (or another way)?

Thank you, Yoni.

[zehortigoza]

Check the mavlink-router config file, it should be storing logs automatically, you just need to know what is the one from the flight and upload it to http://logs.px4.io/ https://github.com/intel/mavlink-router#flight-stack-logging

[mandelyoni]

Ok, I'll check it and upload it. Thank you

[mandelyoni]

Where are the the logs stored? On the Aero or sent to QGC and stored there? What's the name of a log file?

[mandelyoni]

Hi,

Please see results in the following links: https://logs.px4.io/plot_app?log=31695a90-f645-47f5-9078-af32c084ca4f https://logs.px4.io/plot_app?log=54ccb723-85aa-493c-bc45-01ca119f4003

Thank you, Yoni.

[zehortigoza]

Look at "local position" graphs, it is flat at zero during the flight, so it means your floor don't have a good texture that the down-faced camera can extract features. As said in the readme of aero-optical-flow this camera is not good for optical-flow. Another option for you would be connect a PX4Flow to aero, it is way better to detect floors with lower texture.

[mandelyoni]

Can you instruct us, or send a link, on how to connect the PX4Flow to the Aero? We already have the hardware. Thank you.

[trgiman]

Hi Mandelyoni,

Currently I am having PX4Flow related connectivity discussion here. Maybe it can help? https://github.com/intel-aero/meta-intel-aero/issues/320 . I would like to ideally connect trough i2c telemetry port.

[mandelyoni]

Ok, thanks

[mandelyoni]

Hi zehortigoza,

We have been trying for a couple of month now to operate the optical flow in the Aero drone. According to the following link, you write that the Garmin Lidar 3 is not a good option: https://github.com/PX4/Firmware/pull/8409#issuecomment-349535687

Could you please point us to the latest checked (and working) firmware and hardware needed to operate the optical flow in the Aero (PX4 firmware version, Lidar type, etc ... ) ?

Thank you, yoni.

[zehortigoza]

We flew using optical-flow with Garmin Lidar 3 but the readings are not as good as the one from LeddarOne. The support to LeddarOne was merged recently in PX4 so there still no release with it, you need to build PX4 master branch to use.

[colineRamee]

Hello @zehortigoza , I am currently working with the PX4Flow (with the maxbotix sonar) sensor on the Aero. So far I haven't been able to switch to position hold mode without GPS, so I want to use a laser based range finder and see if I get better results. I see that you are advising to use the Leddar One and I was wondering how to connect it to the board. Right now the PX4Flow is connected to the telemetry port which is enabled for I2C. However the setup presented there https://docs.px4.io/en/flight_controller/intel_aero.html has the Leddar One (interface 3.3 UART) also connected to the telemetry port. Could the PX4Flow be connected to the Compass port instead or the Leddar one to the GPS port? What would be required to make it work with the FPGA? Thanks in advance for your advice

[zehortigoza]

@Torgenn You can connect to compass port, you would just need a I2C splitter but I would avoid it. Or you can make a connector and then split the wires you need for each sensor from telemetry port: VCC UART TX UART RX I2C CLK I2C Data GND

[colineRamee]

Thanks for the quick reply @zehortigoza ! I'll follow your advice and make a special connector for the telemetry port then. Will I need to make any changes to the FPGA code in that case? Out of curiosity why would you avoid making a I2C splitter?

[zehortigoza]

@Torgenn You don't need any FPGA or PX4 changes. The problem is adding more devices to the I2C compass bus could cause delay in the compass readings because the bus is in use by another device, degrading EFK performance.

[trgiman]

@zehortigoza I did followed your advice above. But I am using Ardupilot 3.5.5. PX4Flow works ok on I2C Telem port. But I am not able to connect Leddar One to serial port on Telem. Serial port on Telem works correctly. I was trying to listen for Mavlink messages and switching it off and on or setting it work with Lidar param (value 9) according Ardupilot guidehttp://ardupilot.org/copter/docs/common-leddar-one-lidar.html . Is LeddarOne driver starting automatically on ArduPilot FC? Suggestions appreciated. Sonarrange and Sonarvoltage in MissionPlanner both show 0. I saw your video using LedOne on youtube so I was wondering if you can provide more insights. Thank you.

[zehortigoza]

I did not worked much with Ardupilot maybe @lucasdemarchi can help you or maybe ask that directly into Ardupilot Github or Gitter.

[lucasdemarchi]

@trgiman what param are you setting? You should be setting SERIAL1_PROTOCOL. I never used leddar one, but it should work if it works on other boards.

[lucasdemarchi]

@trgiman btw this should fix your issue to use master branch on ardupilot: https://github.com/ArduPilot/ardupilot/pull/8609

[trgiman]

hi @lucasdemarchi thanks for fast response. Sure I am setting SERIAL1_PROTOCOL. But it does not work. I realized that maybe I did not return back to standard my settings for nsh console to be forwarded to SERIAL1 as I stated in this issue https://github.com/ArduPilot/ardupilot/issues/8456

Many thanks for #8609. I will try it. Currently I am using 3.5.5 Ardupilot

[trgiman]

hi @lucasdemarchi I am really sorry. In regards of Leddar One. It was my fault due to trying to accessing nsh console trough SERIAL1_protocol (an not returning Nuttx configs back to default, however can not access nsh console) as highlighted in this issue https://github.com/ArduPilot/ardupilot/issues/8456

[colineRamee]

I've installed the LeddarOne and the PX4Flow on the drone, but I'm still having trouble engaging Position Control. In the following log the drone went into Position Control for a few seconds and went back to altitude control as the local position estimate went to zero: https://logs.px4.io/plot_app?log=4c6e3f1e-7ff0-48bc-9aa3-2ae9175cf238. It might be more of a question for the PX4 forum, but I was wondering if anyone might have an idea why that's happening. The office I am testing in has a very uniform carpet, so maybe there's just not enough features for the sensor to work properly. I know there might be parameters I could change to improve performance, unfortunately the documentation seems to be gone https://pixhawk.org/dev/px4flow#parameters Thanks for the help!

[trgiman]

@Torgenn documentation is at https://github.com/PX4/px4_user_guide/blob/master/en/sensor/px4flow.md

[zehortigoza]

@Torgenn

You can hand held the Aero RTF and check the quality of the optical-flow in the desire floor before actually fly by looking to QGroundControl>Widgets>MAVLink Inspector>OPTICAL_FLOW_RAD>quality if you are getting values higher than 100 it should be enough, otherwise try to adjust the focus. If it still do not work it is probably because you floor have low texture.

[colineRamee]

Thanks @zehortigoza for the advice, the flow quality was jumping between 200 and 0. I was able to test in another room with a different floor and it finally worked!

[colineRamee]

I installed a feature-rich carpet in the room where I usually test and I can now switch to position control. It works pretty well, except it drifts to one side (the left one) continually, about 1m every 20s. It is pretty slow but it still caused a crash when I took my eyes off the drone for a few seconds (the room in which I test is pretty small). Here's the log of the flight (disregard anything after 3:20 it's the crash): https://logs.px4.io/plot_app?log=cea8bd99-41c3-4021-8e9e-d3cbd492bb4b. If you look at the manual control input you can see I have to correct the roll always to the same side every 20 sec. Do you have any idea what might be the cause of the drift, how to fix it or track the origin of the issue? I'm thinking it might be IMU bias, but I thought the calibration was taking care of that. The PX4 is running the EKF2 estimator.

[zehortigoza]

This is mostly liked a error from PX4Flow but this web interface do not show the data from OPTICAL_FLOW_RAD message, I don't remember the name but there is a cli tool to get all the information from logs, check in PX4 documentation. If this is the real problem you can try:

• calibrate the focus of PX4Flow
• check if PX4Flow is not sending distance data, connecting PX4Flow in the USB of your computer you can disable it. I don't remember if PX4 will discard it and only use the dedicated range sensor or if it will combine both

Other than that I would suggest to open a issue in PX4Flow/PX4 github, they may have more information to help you.

[colineRamee]

Thanks @zehortigoza ! I recalibrated the sensors (compass, accelerometer, etc) and it's much better. It still drifts a little bit, and sometimes the yaw oscillates, but it looks a lot more stable.

[RR2-IP2]

Hi @Torgenn,

We are looking at implementing the same setup as you and had a couple short questions.

1. Did you connect both the PX4 Flow and Leddar One to the telemetry port? (as suggested by @zehortigoza in an above comment)
2. Did you have a distance sensor connected to the PX4 Flow directly (like an ultrasonic)?

[colineRamee]

Hi @RR2-IP2 , Yes I followed zehortigoza advice and connected both sensors to the telemetry port. I already had a ultrasonic sensor connected to the PX4 flow board and I left it there. The firmware automatically used the values provided by the laser range sensor to compute the flow.

[RR2-IP2]

Thank you for the quick response @Torgenn.

With that same setup, we received promising values under the MAVLink Inspector some of the time but then the values under OPTICAL_FLOW_RAD would all randomly jump to zero together. The DISTANCE_SENSOR/current_distance also randomly jumped to zero when the PX4 Flow was connected (although not synchronously with the optical flow rad values). Without the Leddar One attached, we still experienced the same issues with the data (however current_distance remained zero).

Did you experience anything similar or have any idea what might be causing this issue?

[colineRamee]

I had an issue where the optical_flow_rad quality and some other parameters would jump to zero when there were not enough features on the ground for the sensor to work properly. If you connect directly to the px4flow board you can check:

1. if its properly focused at the distance you want to work,
2. the pattern of dots that indicates the quality of the features (if you see the dots disappearing or flickering a lot the quality is bad)

Once I operated over a carpet with some patterns the optical_flow_rad values were no longer jumping to zero. The optical_flow_rad measurements and distance_sensor/current_distance are fused in the state estimator and are independent before, so it's normal that they don't go to zero synchronously. I didn't have any issue with the distance_sensor/current_distance values after the leddar one was connected. Since you're getting some values when the laser is connected, I am assuming you've set sens_en_leddar1 to 1 and the ekf_hgt_mode to 2, and I don't know what might cause the issue.

[RR2-IP2]

Thank you for those tips, we will definitely give them a go and hopefully they resolve our issues as well.

Yes, we have set those parameters and the leddar one was working correctly before the PX4 Flow was attached. That's okay, we will look into this further ourselves. Thanks for your help!