Object Following, Camera frame not updating.

[JODEagle]

The camera seems to stop updating when the ObjectDetector('ssd_mobilenet_v2_coco.engine') is executed.

We've run through all the other samples and the camera appears to work fine. (It's the RPi Camera V2 + Lens Attachment from the Bill of Materials.) When trying to run through the object_following notebook though, we seem to get an initial frame then no updates.

Narrowing the issue down, we created a new notebook with only two cells. The first creates an image widget and links the camera to it. The second only runs the ObjectDetector('ssd_mobilenet_v2_coco.engine'). When we run the first, the video stream seems fine but as soon as we run the second, the stream freezes.

We ran: journalctl -u nvargus-daemon >> nvargus_log.txt I've attached the resulting file which shows issues. Any help would be appreciated. Thanks, Jim

nvargus_log.txt

[jaybdub]

Hi JODEagle,

Thanks for providing this detailed summary. This will be helpful in tracking this issue down.

I'm curious, when you reduce the frame rate of the camera (for example to 10FPS), do you still experience this?

camera = Camera.instance(width=300, height=300, fps=10)

Thanks, John

[JODEagle]

Hi John,

Thanks for the prompt reply.

I'll try the reduced frame rate when I get back on Monday morning and let you know.

Jim

[JODEagle]

Hi John,

We got the same result with the lower frame rate.

Jim

[isshiki]

I also face the same problem.

In my case, I tried the following code and it worked:

camera = Camera.instance(width=300, height=300, capture_width=300, capture_height=300, fps=10)

The cause may be insufficient memory. When ObjectDetector is executed, it has already consumed 82% of memory. If the camera image is large, it seems to consume more memory.

[isshiki]

I tried a lot more.

I reduced the memory usage of Linux OS following the article: [How to save ~ 1GB Memory on Jetson Nano by installing Lubuntu Desktop] (https://www.zaferarican.com/post/how-to-save-1gb-memory-on-jetson-nano-by-installing-lubuntu-desktop)

In addition, I reduced the camera frame rate to 10 FPS referring to the jaybdub's advise. Thanks.

camera = Camera.instance (width = 300, height = 300, fps = 10)

As a result, the camera frame came to be updated normally, and the "Example 4-Object Following" came to be able to be executed perfectly.

[VanFPGA]

Almost same here: In my case, the "camera...10fps" (isshiki) worked for a few minutes (teleoperation), then delay appears (like a pipeline too full).

Based on isshiki OS and memory observations/experiments=> would be great to have a new version for the jetbot_image.zip ?

As an alternative is there a python set of command to "clear/reset" the image/camera "pipeline" ?

[JODEagle]

Thanks Isshiki, Unfortunately, reducing the frame rate to 10fps didn't help us. We even tried cutting the widths and heights to 150.

We also tried disabling the Gnome UI altogether since we're not using it when going through the web interface. Initial used memory was 549. Still the same issue...

[JODEagle]

VanFPGA, Yours seems to be a different issue with the teleoperation and may be related to your WiFi connection. I saw a similar issue when I first put the Jetbot together and was using a WiFi dongle. I got a 8265.NGWMG.NV card with the antennas and it worked a little better but then I connected it to the 5MHz WiFi and it seemed much better.

[jaybdub]

Hi All,

Thanks for the suggestion on reducing runtime memory.

We've recently noticed that insufficient disk space was causing the camera to freeze at initialization for some of our examples.

Could you report the disk space usage when the camera freeze is occuring? It would be good to see if this is the case for most.

We were recently made aware that one of the daemon's is logging lots of information into a syslog file which is consuming disk space. Currently, the workaround is to disable logging

sudo service rsyslog stop
sudo systemctl disable rsyslog

Please let me know if this helps.

Thanks! John

[JODEagle]

Ours seems to be an issue with the Intel WiFi driver... iwlwifi

We're seeing errors on the console on boot. It seems to be trying to load iwlwifi-8265-26.ucode then falls back to -25, -24, -23 then I assume -22 loads since those failed files don't exist in the /lib/firmware folder but iwlwifi-8265-22.ucode does.

Then when we hit the web interface on 8888 it works, but there are error logs scrolling on the console screen constantly.

We plugged in the Ethernet cable and unloaded iwlwifi (using sudo modprobe -r iwlwifi) and the object recognition appears to be working.

I'm not sure how to fix the driver. There are higher version files in the /lib/firmware folder (up to iwlwifi-8265-36.ucode). I'm not sure how to try a different version.

[isshiki]

Currently, My JetBot is working properly. ... I don't know why... There is no problem with disk space.

But the camera is throwing some errors like these:

nvargus-daemon[4745]: NvCameraIspConfigFileLoad: type = 0x15 Config file "isp3_1.cfg" Line 2564: Error: Invalid isp config attribute: "contrastEnhancement.v2.Video[3]={0,0,0,0,0,0,0}"
nvargus-daemon[4745]: SCF: Error BadParameter: Vector index out of bounds (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamerautils/inc/Vector.h, function get(), line 337)
nvargus-daemon[4745]: message repeated 188 times: [ SCF: Error BadParameter: Vector index out of bounds (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamerautils/inc/Vector.h, function get(), line 337)]

I'm using the following camera module:

• Option 2 - RPi Camera V2 + Lens Attachment](https://github.com/NVIDIA-AI-IOT/jetbot/wiki/bill-of-materials#option-2---rpi-camera-v2--lens-attachment)

[JODEagle]

Well, we re-imaged one of the SD cards from the jetbot_image_v0p3p0.zip file, copied the best_model.pth into the collision_avoidance folder and the ssd_mobilenet_v2_coco.engine to the object_following folder then "Installed latest software" following step 5 in the software setup and the object following works now.

I'm not sure what the difference is. We did the same thing originally. I guess third time's a charm (maybe?).

[MetrixLambert]

I tried a lot more.

I reduced the memory usage of Linux OS following the article: [How to save ~ 1GB Memory on Jetson Nano by installing Lubuntu Desktop] (https://www.zaferarican.com/post/how-to-save-1gb-memory-on-jetson-nano-by-installing-lubuntu-desktop)

In addition, I reduced the camera frame rate to 10 FPS referring to the jaybdub's advise. Thanks.

camera = Camera.instance (width = 300, height = 300, fps = 10)

As a result, the camera frame came to be updated normally, and the "Example 4-Object Following" came to be able to be executed perfectly.

Thanks, it works for me.

[kiddjmadd]

UPDATE => Burned same image to a new U3 V30 sd card, and it's working even without setting the fps. Leaving comment for posterity in case it helps anyone else.

//=\=//=\=//=\=//=\

I'm having this issue as well. I've gone as far as shutting down x-windows completely with

sudo systemctl stop lxdm.service # since I'm on 18.04 and replaced gdm with lxdm to save memory per link above

Tried to keep x-windows off at startup but then wifi didn't connect so I decided to maintain graphical startup even though it's a headless system, and then stop it thru ssh. Stopping X-windows was good for ~100 MB of free RAM. Still, based on the "htop" output it seems once I get to the "from jetbot import bgr8_to_jpeg" code chunk, I'm already using 3.4GB of RAM and 550 MB of swap. Even with refresh set to 5 fps, a static image appears. Also, once that chunk completes, the jetbot starts spinning in a circle. Other demos seem ok. I souped up the jetbot to use a Makita 18V battery so I can power thru the 4A input and get full "MAXN" power and fan. I used same power supplies as those in the Kaya model (https://docs.nvidia.com/isaac/isaac/doc/tutorials/assemble_kaya.html). Is the use of swap expected for these? If so, would a higher performance SD card help accelerate? My card is an XC with the "1" in the "U" so 10 MB/s.