how to run a jetbot right after bootup?

HannahZhangVW commented 5 years ago

I would like to have the jetbot car run collision avoidance algorithm automatically after boot up. Here is what I did:

converted "live_demo.ipynb" to a python file "live_demo.py".
Put "python /home/jetbot/Notebooks/collision_avoidance/live_demo.py in /etc/rc.local. However, the live_demo.py can't run because no pytorch, pytorchvision, jetbot could be import. I installed AMR version pytorch and pytorchvision sucessfully. how to intall jetbot? Why "live_demo.ipynb" don't have import problems when it runs on jupytor?

jaybdub commented 5 years ago

Hi HannahZhangVW,

Thanks for reaching out! Do you know which python interpreter you’re calling?

We install dependencies for Python 3 only. My best guess is your new script is being executed with python 2.

Best, John

HannahZhangVW commented 5 years ago

Hi John Thank you very much for such rapid reply. We talked briefly during the GTC. I already built a jetbot car. It works amazing well. I believe Jetson Nanno will be another very success product. Just as you guessed, I am using python 2. On the command line $python –version $Python 2.7.15rc1 I used the released image jetbot_image_v0p3p0.zip to boot up. Should I install python 3 manually? Thanks! Hannah

From: John [mailto:notifications@github.com] Sent: Tuesday, April 30, 2019 9:19 AM To: NVIDIA-AI-IOT/jetbot jetbot@noreply.github.com Cc: Zhang, Hannah (VWoA-ERL) Hannah.Zhang@vw.com; Author author@noreply.github.com Subject: Re: [NVIDIA-AI-IOT/jetbot] how to run a jetbot right after bootup? (#55)

Hi HannahZhangVW,

Thanks for reaching out! Do you know which python interpreter you’re calling?

We install dependencies for Python 3 only. My best guess is your new script is being executed with python 2.

Best, John

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/NVIDIA-AI-IOT/jetbot/issues/55#issuecomment-488017645, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AL6PUEIVKFW4ULOBU2FGISDPTBWNNANCNFSM4HJJH5UQ.

jaybdub commented 5 years ago

Awesome! Nice to talk to you again, glad to hear you're enjoying it!

Python3 is actually already installed, all you have to do is call python3 instead of python.

Hopefully that works, if you run into any issues let me know.

Best, John

HannahZhangVW commented 5 years ago

Hi John, Thank you very much for your suggestion. I create a symbolic link python to point to python3 instead of pyton2. Using python3, import torch, torchvision and jetbot successfully. I found there is no rc.local anymore. What is the best way to run python script automatically after jetson nanno boot?

Thank you very much! Hannah

From: John [mailto:notifications@github.com] Sent: Tuesday, April 30, 2019 11:53 AM To: NVIDIA-AI-IOT/jetbot jetbot@noreply.github.com Cc: Zhang, Hannah (VWoA-ERL) Hannah.Zhang@vw.com; Author author@noreply.github.com Subject: Re: [NVIDIA-AI-IOT/jetbot] how to run a jetbot right after bootup? (#55)

Awesome! Nice to talk to you again, glad to hear you're enjoying it!

Python3 is actually already installed, all you have to do is call python3 instead of python.

Hopefully that works, if you run into any issues let me know.

Best, John

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/NVIDIA-AI-IOT/jetbot/issues/55#issuecomment-488071590, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AL6PUELHOIMWW5ANCZV4SCTPTCIQDANCNFSM4HJJH5UQ.

jaybdub commented 5 years ago

Hi Hannah,

One way is to create a system service that calls your script and enable it to run at boot. We do this for the stats display on JetBot. We actually generate the system service using this script:

https://github.com/NVIDIA-AI-IOT/jetbot/blob/master/jetbot/utils/create_stats_service.py

That script automatically generates the system service file (just filling in a few parameters like the current user). You can easily create a file like this manually. Say you want to create a service called run_my_python_script which executes the python script /home/jetbot/my_python_script.py. You would:

Create a file /etc/systemd/system/run_my_python_script.service

Edit the following with text like the following

Description=Runs my python script
[Service]
Type=simple
User=jetbot
ExecStart=/bin/sh -c "python3 /home/jetbot/my_python_script.py"
WorkingDirectory=/home/jetbot
Restart=always
[Install]
WantedBy=multi-user.target

Start the service manually by calling this in a terminal
```
sudo systemctl start run_my_python_script
```
Enable the service at boot by calling this in a terminal
```
sudo systemctl enable run_my_python_script
```

Please let me know if you run into any issues. Hope this helps!

John

HannahZhangVW commented 5 years ago

Hi John, Yesterday, I trained the collision avoidance with my office desk pictures. In the original train script, it saves the best accuracy epoch of 30 epoch. I changed it to save every epoch. Surprisingly, it not only runs well on my desk, it also runs well in my kitchen which I didn’t take any picture. See the video I record yesterday:

https://youtu.be/Z4k6rm935rI

Another question, could you please share the training script and live demo script for the car following the Lego lane?

Thank you very much!

Hannah

From: John [mailto:notifications@github.com] Sent: Tuesday, April 30, 2019 11:53 AM To: NVIDIA-AI-IOT/jetbot jetbot@noreply.github.com Cc: Zhang, Hannah (VWoA-ERL) Hannah.Zhang@vw.com; Author author@noreply.github.com Subject: Re: [NVIDIA-AI-IOT/jetbot] how to run a jetbot right after bootup? (#55)

Awesome! Nice to talk to you again, glad to hear you're enjoying it!

Python3 is actually already installed, all you have to do is call python3 instead of python.

Hopefully that works, if you run into any issues let me know.

Best, John

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/NVIDIA-AI-IOT/jetbot/issues/55#issuecomment-488071590, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AL6PUELHOIMWW5ANCZV4SCTPTCIQDANCNFSM4HJJH5UQ.

hachpai commented 5 years ago

Hi all!

I'm into the same goal, thanks for those instructions. Someone could give a trained model for obstacle avoidance? I'm lacking of time to produce the dataset and train it by myself... I've two jetbots and i just need them to avoid each others and evident walls in good light conditions.

RyanC1681 commented 5 years ago

Hi, How to run live demo live_demo.py with downloaded pre-trained model? I tried python3 live_demo.py, but it didn't spin the wheel or no outputs. Can I put ipynb in in /etc/rc.local or let it continuous run? Thanks,

jaybdub commented 5 years ago

@HannahZhangVW

Thanks for sharing! Very impressive!

The road following code will be released soon, but we don’t have an exact date. If you’re eager to get started you can check out the notebooks in the road_following_ai branch.

It’s under development, we still need clean the samples and add documentation.

Please let me know if you have any questions or feedback.

Best, John

HannahZhangVW commented 5 years ago

Hi John, During running the live mode for collision avoidance, sometimes the camera only return one picture, so that the update function only be called once. When this problem happens, if the first picture is free, the car will go forward forever and if the first picture is blocked, the car will spinning forever. Since it happens 50% chance, I have to reboot all the time. Is there a way to make the camera more stable?

Best Regards Hannah From: John [mailto:notifications@github.com] Sent: Wednesday, May 08, 2019 12:36 AM To: NVIDIA-AI-IOT/jetbot jetbot@noreply.github.com Cc: Zhang, Hannah (VWoA-ERL) Hannah.Zhang@vw.com; Mention mention@noreply.github.com Subject: Re: [NVIDIA-AI-IOT/jetbot] how to run a jetbot right after bootup? (#55)

@HannahZhangVWhttps://github.com/HannahZhangVW

Thanks for sharing! Very impressive!

The road following code will be released soon, but we don’t have an exact date. If you’re eager to get started you can check out the notebooks in the road_following_ai branch.

It’s under development, we still need clean the samples and add documentation.

Please let me know if you have any questions or feedback.

Best, John

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/NVIDIA-AI-IOT/jetbot/issues/55#issuecomment-490381803, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AL6PUEPF3DSOM65FS7M6MU3PUJ7FVANCNFSM4HJJH5UQ.

jaybdub commented 5 years ago

Hi Hannah,

Rather than rebooting, the following should be an easier method to recover from this state.

Restart notebook
Restart camera daemon (from terminal)
```
sudo systemctl restart nvargus-daemon
```

Could you try the following and see if this issue persists:

Convert model to half precision (and update preprocessing function)


model = model.half()
...
def preprocess(camera_value):
    ...
    return x.half()

Configure Jetson Nano to 5W mode (from terminal)
```
sudo nvpmodel -m1
```

Please let me know if this works for you.

Thanks! John

HannahZhangVW commented 5 years ago

Hi John, I am using color tape to layout a simple road for the Jetbot car to run. The road also has 90 degree right turns and left turns. The model’s output should be “arrived”, “go forward”, “left turn”, and “right turn”. I am going to make 4 directories under the “dataset” folder. The 4 directories names are “arrived”, “forward”, left”, and “right” . I will collect images and put them into the corresponding directory. In the training python script, I will modify the “collision avoidance” ‘s training_model.ipynb in the following place:

Change from: model.classifier[6] = torch.nn.Linear(model.classifier[6].in_features, 2)

To: model.classifier[6] = torch.nn.Linear(model.classifier[6].in_features, 4)

In the live_demo.ipynb, I would like to modify collision_avoidance/live_demo.ipynb in the update function (changes shown in green color and red color text).

////////////////////////////////////

def update(change):

global blocked_slider, robot

x = change['new']

x = preprocess(x)

y = model(x)

# we apply the `softmax` function to normalize the output vector so it sums to 1 (which makes it a probability distribution)

# y = F.softmax(y, dim=1)

arrived = float(y.flatten()[0])

forward = float(y.flatten()[1])

left = float(y.flatten()[2])

right = float(y.flatten()[3])

direction = max(arrived, forward, left, right)

if(direction == arrived)

    robot.stop()

else if (direction == forward)

    robot.forward(0.4)

else if (direction == left)

     robot.left(0.4)

else

     robot.right(0.4)

time.sleep(0.001)

Will this plan works or any better suggestion?

Hannah Zhang

From: Zhang, Hannah (VWoA-ERL) Sent: Friday, May 24, 2019 11:45 AM To: 'NVIDIA-AI-IOT/jetbot' reply@reply.github.com Cc: 'Chitoku Yato' cyato@nvidia.com Subject: jetbot camera update problem

Hi John, During running the live mode for collision avoidance, sometimes the camera only return one picture, so that the update function only be called once. When this problem happens, if the first picture is free, the car will go forward forever and if the first picture is blocked, the car will spinning forever. Since it happens 50% chance, I have to reboot all the time. Is there a way to make the camera more stable?

Best Regards Hannah From: John [mailto:notifications@github.com] Sent: Wednesday, May 08, 2019 12:36 AM To: NVIDIA-AI-IOT/jetbot jetbot@noreply.github.com<mailto:jetbot@noreply.github.com> Cc: Zhang, Hannah (VWoA-ERL) Hannah.Zhang@vw.com<mailto:Hannah.Zhang@vw.com>; Mention mention@noreply.github.com<mailto:mention@noreply.github.com> Subject: Re: [NVIDIA-AI-IOT/jetbot] how to run a jetbot right after bootup? (#55)

@HannahZhangVWhttps://github.com/HannahZhangVW

Thanks for sharing! Very impressive!

The road following code will be released soon, but we don’t have an exact date. If you’re eager to get started you can check out the notebooks in the road_following_ai branch.

It’s under development, we still need clean the samples and add documentation.

Please let me know if you have any questions or feedback.

Best, John

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/NVIDIA-AI-IOT/jetbot/issues/55#issuecomment-490381803, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AL6PUEPF3DSOM65FS7M6MU3PUJ7FVANCNFSM4HJJH5UQ.

dvillevald commented 5 years ago

Hi Hannah,

I tried a similar approach (a classifier with 4 classes - left, right, free, blocked) to teach JetBot to navigate through the maze of traffic cones and it worked pretty well. I also added FPV videos with superimposed classification and robot actions - it helped me a lot to debug. In case you are interested, the links are below (the code is on Github):

Youtube: https://youtu.be/YwxSwXEjd4c Github: https://github.com/dvillevald/Finding-path-in-maze-of-traffic-cones Hackster: https://www.hackster.io/dvillevald/transfer-learning-with-nvidia-jetbot-fun-with-cones-adf531

Hope it helps. Good luck! Dmitri

NVIDIA-AI-IOT / jetbot

how to run a jetbot right after bootup? #55