Combining Jetson Inference with Jetbot native code

[MichalZelazko]

Hello,

I am having trouble combining your repository code with the Jetbot code coming to the Jetson Nano right after flashing it with the SD card image. What I've done:

Followed the building-repo-2.md without any problems, cloned it to the workspace directory (where all of the other directories are like Desktop, Downloads and most importantly jetbot and it works perfectly with Object Detection or Pose Estimation examples. I wanted to combine it with Jetbot motor to make it follow a person (rewriting what I found in this repository), however, using Jetbot's code outside of JupyterLab is a big trouble for me. Whenever I'm trying to use sth from the Jetbot directory, it is missing packages, like ipywidgets in heartbeat.py or Adafruit_MotorHAT in motor.py. I'm aware that ipywidgets is Jupyter-specific, but still after trying to install it just with pip3 install ipywidgets it runs for a minute or two and ends up with an error about not being able to import tags.

On the other hand, trying to run it from Jupyter Lab terminal, results in errors like "Module jetson. not found" in line import jetson.inference and when I go to /usr/lib/Python3.6/dist-packages I have different outputs dependent on using Jupyter or SSH.

SSH:

Jupyter:

Is there any way to install the jetson-inference globally (including jupyter) or is there any other way I should use it to combine the Inference with Jetbot code.

run.py code:

import jetson.inference
import jetson.utils
import time
from jetbot import Robot

cam_width = 1280
cam_height = 720
cam_port = "csi://0"

net = jetson.inference.detectNet("ssd-mobilenet-v2", threshold=0.5)

camera = jetson.utils.gstCamera(cam_width, cam_height, cam_port)

display = jetson.utils.glDisplay()

robot = Robot()

def motor(center,area):

    if center < 620:
        robot.left(0.50)
        robot.right(0.60)

        if center < 400:
            robot.left(0.50)
            robot.right(0.70)

            if center < 200:
                robot.left(0.50)
                robot.right(0.80)

    elif center > 660:
        robot.left(0.60)
        robot.right(0.50)

        if center > 880:
            robot.left(0.70)
            robot.right(0.50)

            if center > 1080:
                robot.left(0.80)
                robot.right(0.50)
    else:
        robot.left(0.55)
        robot.right(0.55)

try:

    err_prev = 0
    time_prev = 0
    Kp = 0.5
    Kd = 0
    Ki = 0.3
    while True:
        img, width, height = camera.CaptureRGBA()
        detections = net.Detect(img, width, height)
        display.RenderOnce(img, width, height)
        display.SetTitle("Object Detection | Network {:.0f} FPS".format(net.GetNetworkFPS()))

        for i in detections:

            if i.ClassID == 1:

                target_center = cam_width / 2

                err = i.Center[0] - target_center

                Kp = Kp+i.Area/100
                de = err - err_prev
                dt = time.time() - time_prev

                control = Kp*err + Kd*de/dt + Ki*err*dt

                time_prev = time.time()
                err_prev = err

                if i.Area > cam_width * cam_height * 0.5:
                    robot.left(0)
                    robot.right(0)
                    break

                motor(i.Center[0], i.Area)

            else:
                print("no person stopping motor")
                robot.left(0)
                robot.right(0)
except:
    robot.left(0)
    robot.right(0);