Radar sensor - NPC count delete on range exit - not happening - 2020.03 version

[dr563105]

In continuation of this issue, I toggled the traffic option in the simulator UI, if the sensor detects correctly.

In 2020.03 version, even if the traffic and pedestrian toggle is OFF, the last number of detected objects is retained and looped over. If it had detected 5 before I turned off, the same 5 was retained. In 2020.05, results are perfect; working as intended.

Unfortunately due the exposure of the UI being so dark, the ego vehicle struggles to recognise the lanes. Therefore I have but no choice to continue in 2020.03.

So, what changed between these versions for the detection to work differently and consequently correctly? Could you give the files, lines to be changed, to mimic the same behaviour as 2020.05?

Thank you.

Update: Replacing 2020.05 RadarSensor.cs gets me dependency errors relating to NPCManager and NPCController. Update 2 Resolving those changes still don't have an effect. Update 3: After adding the despawn callbacks in rader sensor, the toggle resets the vehicle count to 0. However, getting hit with null exception reference.

[EricBoiseLGSVL]

@dr563105 I went through the changes from 03 to 05 and there are just too many to see what this issue is. Can you please post images of the exposure issue that you are talking about? The fix that we are talking about are for bridge sent images that are dark and I think your issue is different. If it is, I can post the code needed to change here.

[dr563105]

Yeah I'm talking about the images sent through the bridge. Some of the workaround suggested in one of the other issue threads, didn't work well as I hoped.

w.r.t to the 03 to 05 changes, yes there are so much. I understand it. If not for the dark image problem I would gladly use the 05 version. Hence this cherry picking hack method.

[EricBoiseLGSVL]

@dr563105 I got the diffs for the fix for dark images. The only change that I would make is the private SensorRenderTarget(int width, int height, bool cube, GraphicsFormat colorFormat = GraphicsFormat.R8G8B8A8_UNorm) change in line 69 of SensorRenderTarget.cs. You can use the diffs to fix any conflicts. DarkFixDiffs.zip

[dr563105]

Thanks Eric. line 69, 78, 100, 102, 138, 145 in SensorRenderTarget and line 278 CameraSensorBase. The change GraphicsFormat.R8G8B8A8_SRGB makes the ultimate difference. Without this, the images are dark.

I observe that the depth camera images, are a bit cloudy/hazy. Is this right output of a depth image? What can I do so that I can easily see the NPCs? depth sensor.zip

In Shalun map, the segmentation camera looks weird. Is this normal behaviour? shalun.zip

Finally before I made the SensorRenderTarget changes, I saw this anomaly w.r.t segmentation camera. In some NPCs, I saw only wheels. seg camera anomaly.zip

[EricBoiseLGSVL]

@dr563105 We have fixes for depth and segmentation sensors that will release soon. It is too complex to diff

[dr563105]

Ok that's great. I ask because I need to collect data tomorrow and during the weekend. Trying to solve all the problems before that. Can I expect the release before that?

[EricBoiseLGSVL]

@dr563105 No, we will not be able to have a public release tomorrow, sorry.

[EricBoiseLGSVL]

@dr563105 Here are the files for the fix that we merged after 05 release. I hope this helps DepthFix.zip

[dr563105]

Wow. Thank you so much. This is great.

The SensorPassRenderer.cs, is it a new one? Which location should I place it? In /Assets/Scripts/Sensors ? The Simulator.asmdef, shall I just replace it above the old one present in /Assets/Scripts ?

Update: Placed the sensorpassrender.cs in Utilities and replaced the other. No errors so far. Will compile and tell you.

[EricBoiseLGSVL]

SensorPassRenderer.cs goes in Assets/Scripts/Utilities. Yes you can replace

[dr563105]

Thanks again Eric. Compiled and built. Running successfully.

Regarding the depth camera images - I save them using python cv2 library by reading them as Grayscale. Is this correct approach to save them? Can I read them in another way so as to not lose quality or data?

[EricBoiseLGSVL]

@dr563105 No problem. I think greyscale is fine. You shouldn't lose any data or quality

[dr563105]

I ask because the images I see have a bright cloudy layer. The cars that become visible are only seen as another bright mass. I would prefer if the objects are much easier to spot — Darker solid shade. Are am I misunderstanding how a depth image looks like? I also observed if I’m in the open space with sunlight, I get better, clearer depth images.

[EricBoiseLGSVL]

That doesn't seem right. This is not the same as you posted above?

[dr563105]

Yes. You can use them as well. Here are some fresh samples. depth camera images anamoly.zip

Samples inside 'between buildings' is at time 12.00(fixed) and 'in sunlight' at 13.18(fixed). Using SF map. My problem is I can spot the vehicles as a bright mass but the surrounding is also in the same , bright layer which makes it harder to spot.

[EricBoiseLGSVL]

@dr563105 I think we fixed this issue already and I'll check with the team. If not we will file a bug.

[dr563105]

BTW I used the changes you gave me for depth and segmentation camera sensor for making those samples.