search

keijiro / Rsvfx

An example that shows how to connect RealSense depth camera to Unity VFX Graph
Other
734 stars 111 forks source link

Support for Intel RealSense L515 (LiDAR scanner) #39

Open leeprobert opened 3 years ago

leeprobert commented 3 years ago

I have tried to get the LiDAR L515 working with your project but it doesn't recognise the device. I updated the RealSense SDK with no luck. I am going to see if I get the same issues with the RealSense package on its own (I suspect they haven't updated it for this device).

I will keep you informed of what I find, and may fork the repo to make adjustments if I can. Be great to get your fantastic VFX working with this new device.

leeprobert commented 3 years ago

Interesting development ... in the official package it only works with Depth and the Infrared Y8 stream. Trying to access Color fails with an error: ExternalException: rs2_pipeline_start_with_config(pipe:000001CCB1986E00, config:000001CCB1986720) Rethrow as Exception: Couldn't resolve requests Intel.RealSense.ErrorMarshaler.MarshalNativeToManaged (System.IntPtr pNativeData) (at <6d5b8eabc0b341e3bb8bf3481e1594d0>:0) Intel.RealSense.Pipeline.Start (Intel.RealSense.Config cfg) (at <6d5b8eabc0b341e3bb8bf3481e1594d0>:0) RsDevice.OnEnable () (at Assets/RealSenseSDK2.0/Scripts/RsDevice.cs:71)

Will try switching your stream over to Infrared to see if it works.

stents commented 3 years ago

Interesting development ... in the official package it only works with Depth and the Infrared Y8 stream. Trying to access Color fails with an error: ExternalException: rs2_pipeline_start_with_config(pipe:000001CCB1986E00, config:000001CCB1986720)

I found that the latest firmware does not work with the unity samples that use the color feed - though interestingly works fine with the Realsense viewer. The firmware I have found works best currently is 01.05.00.00.

Rsvfx still has issues with the L515 as you have mentioned though. Not sure if anyone else has worked on a fix for it? I will try investigate further when I get some time as well.

keijiro commented 3 years ago

FYI: I don't have L515, so I have never tried it with Unity. There is no plan for looking into this issue at the moment.

leeprobert commented 3 years ago

I'm going to have another go at it and will post the results 👍

bjornsyse commented 3 years ago

Please do, I would also be interested to try the L515 in a couple of weeks time.

kitschpatrol commented 3 years ago

First, thank you to Keijiro for this immensely useful example project.

Got things sort of working with the L515 + Rsvfx in Unity 2020.3.2.f1.

Here's a quick and ugly way to get going:

  1. First, update the firmware on your L515. Can't hurt. (I think it's simplest to do this through the latest version of the Intel RealSense Viewer. I tested with firmware version 1.5.5.0.)
  2. After opening and upgrading the Rsvfx sample project to Unity 2020.3.2.f1 (or whatever), be sure to run the HDRP wizard to update materials.
  3. Import / overlay the latest version of Intel's RealSense Unity package. (I tested with v2.44.0.)
  4. Set Occlusion Removal in RsProcessingProfile's Point Cloud block to either on or off. This must be a recent addition to the API since it starts with an invalid null value in Rsvfx. (As discussed in issue #40 -- thanks to getarobo.)
  5. The L515 doesn't support the frame rate and dimensions specified by default in Rsvfx's RealSenseVGA60 prefab, resulting in a "Couldn't resolve requests" error at runtime. Set the depth stream to 30 FPS instead of 60, and set the color stream to 960x540 at 30 FPS. (640x480 color frames are not supported! If you're not sure what's supported, explore the available streams in the RealSense Viewer -- and watch the log for info about failing over to stream texture formats that differ from what was requested.)
  6. Edit the ColorVGA and PositionVGA render textures to be 960x540. (And consider renaming them ; )
  7. Run the Simple scene example to test.

The above at least gets L515 --> VFX content flowing into the game view. There are obviously still some issues to resolve at this point given the mismatched dimensions between the position and color frames. I only need the position data at the moment, so I won't have time to dive into the details any time soon, but I hope this sets someone else on the right track.

leeprobert commented 3 years ago

@kitschpatrol - fantastic work! I will try this over the weekend to see if I can get it working too. 👍

bjornsyse commented 3 years ago

@kitschpatrol - fantastic work! I will try this over the weekend to see if I can get it working too. 👍

Any Luck @leeprobert ?

cecilemuller commented 3 years ago

Sadly, Intel is discontinuing RealSense, so it might be pointless to add support ?

bjornsyse commented 3 years ago

Oh, I wasn't aware. Thanks for pointing that out. Wonder what the next horse to bet on in this field would be. Azure Kinect perhaps? Is out of stock here however: https://www.microsoft.com/en-us/d/azure-kinect-dk/8pp5vxmd9nhq?activetab=pivot:overviewtab

burhanahmad02 commented 11 months ago

Hi @kitschpatrol I followed each and every step of yours, I even put the serial number displayed in Intel RealSense Viewer on RealSenseVGA60 GAMEOBJECT UNDER Rs Device Script. But it always logs this error ExternalException: rs2_pipeline_start_with_config(pipe:000002CB026F8EE0, config:000002CB026F8BC0) Rethrow as Exception: Couldn't resolve requests Intel.RealSense.ErrorMarshaler.MarshalNativeToManaged (System.IntPtr pNativeData) (at <cb539c8577184d8bb3db94250e9792dc>:0) Intel.RealSense.Pipeline.Start (Intel.RealSense.Config cfg) (at <cb539c8577184d8bb3db94250e9792dc>:0) RsDevice.OnEnable () (at Assets/RealSenseSDK2.0/Scripts/RsDevice.cs:71)

I am using L515