T.265: Question to to contributors here

neilyoung commented 4 years ago

I'm close to finish a little web application, which is designed to run client-only, so no internet connectivity required.

This app completes my already published proof of concept T.265 REST API and thus forms the basis of a turnkey solution, so to speak. All you need is a T.265, a host for the T.265 (laptop, miniPC, Raspberry PI), my Python scripts and a browser for the webapp. Although still far from going productive, the maturity level of the overall solution is not bad. First of all it allows to test the T.265 and especially its firmware and SDK very fast and to evaluate it in different environments w/o overwhelming pre-requirements. At the moment, I'm still completely undecided how I'm going to "bring it to the market"; it is a spare time activity. Right now I'm only interested in opinions.

In particular, the webapp in cooperation with the Python scripts described above makes the following things possible:

Geo-referencing a floor plan and defining a geo-graphical start reference point on a map or floor plan, from which the initial mapping of a site has to be done. Following runs can then rely on the stored reference point and its meaning in real real world,
Remote control of a T.265 via the REST API described above,
Real-time display of WGS84 or X/Y/Z coordinates on the map or the floor plan (optional).

If you agree, I would (before the release) produce a small video and link it here as well as provide more details. If not, let me know too, then this post will disappear from the issues ASAP :)

dorodnic commented 4 years ago

Everyone are welcomed to share and discuss ideas I'll label the issue to make sure it's not closed prematurely

neilyoung commented 4 years ago

@dorodnic Cool. Thanks for this. So then - will produce the little video tomorrow :)

neilyoung commented 4 years ago

OK, please find the videos here. I'm a bit tired today, need a bit sleep. No time anymore for explanations.

But I hope you will enjoy the videos and will not be overwhelmed be the mass of information.

The videos are not fully published yet, maybe some review is required. But they have all been recorded just with one take.

Mall Of Berlin I The video shows the definition of a "geo-reference" on a map and the remote control of the T.265 from a browser using the T.265 REST API

Mall Of Berlin II Pretty much the same as above, but it now shows, how to geo-reference your floor plan so that it can be used as canvas for the coordinate display.

Home A little test in the first floor of my home, using a hand drawn but pretty good scaled floor plan. About 15 by 5 m area covered. Coordinates anonymized.

I hope you find it informative and not too boring...

Let me know, what you think.

EDIT: The position display in the browser is completely fluent. The videos make it seem sometimes like it "jerks" every now and then. This is just a stroboscopic effect, created during screen recording.

smartynenko commented 4 years ago

Could you by any chance also add real path to see the drift?

neilyoung commented 4 years ago

The drift is really not remarkable. It is most the time less than +- 50cm, if not at the point. Please note: All the videos have been created with the first or at maximum second run on a map.

The accuracy indoors is pretty good, but once I have published the web app everybody can judge by himself.

zainmehdi commented 4 years ago

Hi @neilyoung . Thanks for sharing your work. Did you face any random crashes while using T265 ? I am operating in a corridor sort of an area so it can be a bit offsetting for the tracker because of fewer features and lighting but T265 crashes sometimes because of frame arrival timeout. Please share your experience if you had a similar scenario. Thanks

neilyoung commented 4 years ago

@zainmehdi Well, you know about my main concerns (https://github.com/IntelRealSense/librealsense/issues/5756), a lot of them are still present, even with the latest version.

In my environments I don't have that "frame arrival timeouts" very often, but if it appears, my camera recovers from that, if I move to more featured places. The crash while saving a map is unfortunately more annoying.

If I would be asked today, if the T.265 is a production ready device for me, then I would have to deny it. It might be sufficient for demoing and prototyping, but the stability of the firmware/SDK is not sufficient for productive usage, which needs absolute reliability or at least no crashes. This is not the case in March 2020.

Anyway, the potential of the camera is pretty high and I still love it. I'm convinced the team and community will be able to fix the most horrible issues finally, even though I sometimes wish, it would go faster.

zainmehdi commented 4 years ago

@neilyoung Thanks alot for replying. I wish the same. And hats off for your application. I will be looking forward to checking it out.

neilyoung commented 4 years ago

@zainmehdi Thanks. I expect to be able to publish it during this week. I can only work on this in my spare time.

smartynenko commented 4 years ago

am operating in a corridor sort of an area so it can be a bit offsetting for the tracker because of fewer features and lighting but T265 crashes sometimes because of frame arrival timeout.

It crashes very consistently in low lighting conditions. The easiest way to reproduce is to put camera in black box or make it face dark wall in dim light conditions and it won't even start.

neilyoung commented 4 years ago

I'm having published it. Maybe you find it useful.

https://neilyoung.serveblog.net/

neilyoung commented 4 years ago

I'm having provided an update to the web app part. There is now a new geo-reference option, which allows you to drop the floor plan where you like, in case you are not interested in real GPS (WGS84) coordinates. In order to restore the real world dimensions you would have to provide width and height of a known structure on your floor plan.

Find version 0.3.1 here: https://neilyoung.serveblog.net

There is also a little 30 secs video, which describes the new feature:

https://youtu.be/4XoHloDS8ag

neilyoung commented 4 years ago

Hi all. The great discussion here did encourage me to prepare some enhancements in the upcoming v.0.4.0. :) The solution is now able to record video along with poses and navigate using a recorded ROSBAG.

The video here shows what is about to come soon. It demonstrates a synchronised split view of the "left eye video" from the T265 and the display of the coordinates on a floor plan.

https://youtu.be/_qAW69yfbe4

neilyoung commented 4 years ago

I have bundled the web app with the Python scripts. More on this here https://github.com/IntelRealSense/librealsense/issues/6063#issuecomment-600548083

neilyoung commented 4 years ago

Deployed the stuff to a Raspberry Pi driven Robot car. A video: https://youtu.be/riBsZoSrcpI

patrickpoirier51 commented 4 years ago

@neilyoung one step closer to DIY ROOMBA ... Wife will be happy ;-)

neilyoung commented 4 years ago

@patrickpoirier51 Right :) Not my business, but this is the idea... Thanks

patrickpoirier51 commented 4 years ago

BTW we had a great presentation from @radfordi at the ArduPilot 2020 conference: https://youtu.be/nO_y6BRBBOg?t=3095

I want to thank him for this very interesting presentation... and for answering my "tricky" question on the startup issue ;-)

neilyoung commented 4 years ago

Thanks for sharing.

neilyoung commented 4 years ago

...and nice to meet Jim in person. He was not too chatty with me the last weeks :) ....flies away....

IntelRealSense / librealsense

T.265: Question to to contributors here #5975