Adding optical flow module

[mhkabir]

I am trying to add a optical flow update module to the framework, but have been unable to do so. I'm rather confused on how to integrate the optical flow velocity into the estimate.

I read the older tutorial in single sensor fusion for adding a custom sensor, but didn't understand too much.

@simonlynen Would you be able to help with the steps? Some examples for a similar velocity update sensor would be greatly helpful too.

[stephanweiss]

Hi,

What steps in the tutorial were difficult to follow? With your feedback we can improve them for others.

As for the optical flow (OF) sensor there are different ways to implement it. If you have sufficient computation power and a flat ground, you can use a a tightly coupled approach (e.g. cf. herehttp://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=6669273&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D6669273http://), if computation power is an issue and if your ground is not completely flat you may use a loosely coupled approach (e.g. cf. herehttp://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=6696955&url=http%3A%2F%2Fieeexplore.ieee.org%2Fiel7%2F6679723%2F6696319%2F06696955.pdf%3Farnumber%3D6696955). Or even go with a commercial sensor like the PX4Flow (and use similar equations for the update as for the loosely coupled approach described herehttp://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6225147&navigation=1 or herehttp://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=6696955&url=http%3A%2F%2Fieeexplore.ieee.org%2Fiel7%2F6679723%2F6696319%2F06696955.pdf%3Farnumber%3D6696955.)

Hope this helps. Best, Stephan

---

From: mhkabir [notifications@github.com] Sent: Sunday, August 31, 2014 12:01 AM To: ethz-asl/ethzasl_msf Subject: [ethzasl_msf] Adding optical flow module (#77)

I am trying to add a optical flow update module to the framework, but have been unable to do so. I'm rather confused on how to integrate the optical flow velocity into the estimate.

I read the older tutorial in single sensor fusion for adding a custom sensor, but didn't understand too much.

@simonlynenhttps://github.com/simonlynen Would you be able to help with the steps? Some examples for a similar velocity update sensor would be greatly helpful too.

— Reply to this email directly or view it on GitHubhttps://github.com/ethz-asl/ethzasl_msf/issues/77.

[mhkabir]

Thanks @stephanweiss :)

Presently I'm using a PX4Flow sensor which provides filtered flow for X-Y axes. I have this data in MSF via a callback in the new "flow" module I created. I'm a bit confused about :

1. The actual integration of the data into the EKF
2. Rotation and translation parameters

Some example code for a body velocity sensor would clear up most of the doubts for the first problem.

Now, as you said, I would prefer to move to a tightly coupled approach as I have enough computational power onboard as I'm using fast and efficient, direct SLAM methods. Would there be any open-source optical-flow based implementations of the papers you cited? Or perhaps access on a limited basis? Your help on this would be greatly appreciated.

Otherwise, I will stick to the PX4Flow, which has some limitations which I'm hoping to improve on. The update rate is pretty good (100Hz +), but the range of its onboard sonar module used to get ground distance and the fixed-focus lens it is paired with makes it difficult to make it operate at different altitudes, especially in conditions where there may be a failure of SLAM tracking.

Thanks and Regards, Kabir

[stephanweiss]

unfortunately, we did not yet release the optical flow code and module to the public. We are working on this but cannot give a time line just yet.

If the MSF framework is a bit too complex you could start with the SSF framework (http://wiki.ros.org/ethzasl_sensor_fusion) to which you can apply the tutorials 1:1. Since you only have one sensor (i.e. PX4Flow and IMU) SSF can be used.

Best, Stephan

---

From: mhkabir [notifications@github.com] Sent: Monday, September 01, 2014 5:51 PM To: ethz-asl/ethzasl_msf Cc: Stephan Weiss Subject: Re: [ethzasl_msf] Adding optical flow module (#77)

Thanks @stephanweisshttps://github.com/stephanweiss :)

Presently I'm using a PX4Flow sensor which provides filtered flow for X-Y axes. I have this data in MSF via a callback in the new "flow" module I created. I'm a bit confused about :

1. The actual integration of the data into the EKF
2. Rotation and translation parameters

Some example code for a body velocity sensor would clear up most of the doubts for #1https://github.com/ethz-asl/ethzasl_msf/pull/1.

Now, as you said, I would prefer to move to a tightly coupled approach as I have enough computational power onboard as I'm using fast and efficient, direct SLAM methods. Would there be any open-source implementations of the papers you cited? Or perhaps access on a limited basis? Your help on this would be greatly appreciated.

Thanks and Regards, Kabir

— Reply to this email directly or view it on GitHubhttps://github.com/ethz-asl/ethzasl_msf/issues/77#issuecomment-54099653.

[mhkabir]

Thank you Stephan, I need to use the MSF framework since I actually have 6 sensor sources (2 monocular SLAM, GPS , PX4Flow, laser rangefinder (ground distance) and barometer)

I will keep you updated on the progress.

Kabir

[mhkabir]

@stephanweiss If possible, can you please help me out with some code samples for the loosely coupled optical flow approach?

Is it OK if I send you an email?

[stephanweiss]

Unfortunately, the inertial-optical-flow (IOF) module is not yet released as open source. We are cleaning up the code and plan to release it but have no timeline just yet. That said, using the indicated paper, it should be straight forward to implement it yourself. There are no tricks and tweaks in our implementation w.r.t. what the paper explains.

Best, Stephan

---

From: mhkabir [notifications@github.com] Sent: Saturday, September 06, 2014 9:53 AM To: ethz-asl/ethzasl_msf Cc: Stephan Weiss Subject: Re: [ethzasl_msf] Adding optical flow module (#77)

@stephanweisshttps://github.com/stephanweiss If possible, can you please help me out with some code samples for the loosely coupled optical flow approach?

Is it OK if I send you an email?

— Reply to this email directly or view it on GitHubhttps://github.com/ethz-asl/ethzasl_msf/issues/77#issuecomment-54720372.

[mhkabir]

@stephanweiss Thank you :) I am going with the PX4Flow for now. I got the sensor header written out okay, and am making(slow) progress. I'm slightly confused about the artificial measurement. I have 2 sensor measurements, scaled flow velocity in X and Y axes. How do we use the artificial measurement? (I should have 3 measurements, if I'm not mistaken. 2 actual and one for the unobservable yaw)

Kabir

[stephanweiss]

"artificial measurements" are not needed if you do not want to bound unobservable states. So I suggest you go ahead without it for the time being to test your OF implementation first.

Best, Stephan

---

From: mhkabir [notifications@github.com] Sent: Tuesday, September 09, 2014 4:28 AM To: ethz-asl/ethzasl_msf Cc: Stephan Weiss Subject: Re: [ethzasl_msf] Adding optical flow module (#77)

@stephanweisshttps://github.com/stephanweiss Thank you :) I am going with the PX4Flow for now. I got the sensor header written out okay, and am making(slow) progress. I'm slightly confused about the artificial measurement. I have 2 sensor measurements, scaled flow velocity in X and Y axes. How do we use the artificial measurement? (I should have 3 measurements, if I'm not mistaken)

Kabir

— Reply to this email directly or view it on GitHubhttps://github.com/ethz-asl/ethzasl_msf/issues/77#issuecomment-54956018.

[simonlynen]

@mhkabir If you want you can take a look at the optical flow implementation within the msf. It is not officially supported (yet) but we used it and it works pretty good: https://www.youtube.com/watch?v=ieFseoBF1OU

It is currently on this branch: https://github.com/ethz-asl/ethzasl_msf/tree/single_feat_iof

The paper that describes the approach is here: http://www.ifac-papersonline.net/Detailed/63827.html

[mhkabir]

Thanks Simon, Looked a bit into the code. Looks a bit overwhelming first glance. I will continue to inspect it over the next few days :)

Please let me know if you can fast-forward :)

Kabir

On Fri, Sep 12, 2014 at 9:46 PM, Simon Lynen notifications@github.com wrote:

@mhkabir https://github.com/mhkabir I guess you have to just take a look at the branch in the current state. There are a bit many conflicts to fix in a short time.

— Reply to this email directly or view it on GitHub https://github.com/ethz-asl/ethzasl_msf/issues/77#issuecomment-55425505.

[mhkabir]

@simonlynen Unfortunately, as silly as it sounds, I haven't been able to make any sense of the single_feature_iof branch. Incidentally, I cannot find any references to OF integration in it... I think that the actual optical flow algorithm isn't there, right?

[mhkabir]

@simonlynen Any help please? Can I please have your email?

[simonlynen]

@mhkabir I will see how I can make the actual code available. It's true that what is up is only half of the story.

[mhkabir]

@simonlynen Can you please assist / follow up on this?

[art-mx]

Hi, @mhkabir , have you been able to implement the imu+flow sensor solution?