search

NVIDIA-Omniverse / iot-samples

Collection of samples to help developers build their own IoT solution for Omniverse
MIT License
45 stars 20 forks source link

readme

IoT Samples (Beta)

Table of Contents

Overview

Note: Before you clone the repo, ensure you have Git LFS installed and enabled. Find out more about Git LFS

Developers can build their own IoT solutions for Omniverse by following the guidelines set out in these samples.

IoT Samples guides you on how-to:

The repository is broken down into the following folders:

When opening the iot-samples folder in Visual Studio Code, you will be promted to install a number of extensions that will enhance the python experience in Visual Studio Code.

Architecture

Connector Architecture

The architecture decouples the IoT data model from the presentation in Omniverse, allowing for a data driven approach and separation of concerns that is similar to a Model/View/Controller (MVC) design pattern. The diagram above illustrates the key components to a solution. These are:

Note: Connectors implement a producer/consumer pattern that is not mutually exclusive. Connectors are free to act as producer, consumer, or both. There may also be multiple Connectors and Consumers simultaneously collaborating.

Prerequisites

Before running any of the installation a number of prerequisites are required.

Follow the Getting Started with Omniverse to install the latest Omniverse version.

If you've already installed Omniverse, ensure you have updated to the latest

Installation

Once you have the latest Omniverse prerequisites installed, please run the following to install the needed Omniverse USD resolver, Omni client, and related dependencies.

Windows
> install.bat
Linux
> ./install.sh

App Link Setup

If app folder link doesn't exist or becomes broken it can be recreated. For a better developer experience it is recommended to create a folder link named app to the Omniverse Kit app installed from Omniverse Launcher. A convenience script to use is included.

Run:

Windows
> link_app.bat
Linux
> ./link_app.sh

If successful you should see an app folder link in the root of this repo.

If multiple Omniverse apps are installed the script will automatically select one. Or you can explicitly pass an app:

Windows
> link_app.bat --app create
Linux
> ./link_app.sh --app create

You can also pass an explicit path to the Omniverse Kit app:

Windows
> link_app.bat --path "%USERPROFILE%/AppData/Local/ov/pkg/create-2023.2.0"
Linux
> ./link_app.sh --path "~/.local/share/ov/pkg/create-2023.2.0"

Headless Connector

Headless connectos are stand-alone applications that implements a bidirectional bridge between customer domain and USD related data. The logic implemented by a connector is use-case dependent and can be simple or complex.

There are two sample connector applications - CSV Ingest Application and MQTT Ingest Application - that transits the data as is from source to destination, whereas the Geometry Transformation Application manipulates USD geometry directly in the connector. Depending on the use cases, a connector can run as a headless application locally, on-prem, at the edge, or in the cloud.

CSV Ingest Application

To execute the application run the following:

> python source/ingest_app_csv/run_app.py
    -u <user name>
    -p <password>
    -s <nucleus server> (optional default: localhost)

Or if you are using Environment Variables (see Using Environment Variables)

> python source/ingest_app_csv/run_app.py

Username and password are of the Nucleus instance (running on local workstation or on cloud) you will be connecting to for your IoT projects.

You should see output resembling:

2023-09-19 20:35:26+00:00
2023-09-19 20:35:28+00:00
2023-09-19 20:35:30+00:00
2023-09-19 20:35:32+00:00
2023-09-19 20:35:34+00:00
2023-09-19 20:35:36+00:00
2023-09-19 20:35:38+00:00
2023-09-19 20:35:40+00:00
2023-09-19 20:35:42+00:00
2023-09-19 20:35:44+00:00

The CSV ingest application can be found in the ./source/ingest_app_csv folder. It will perform the following:

In USD Composer or Kit, open omniverse://<nucleus server>/users/<user name>/iot-samples/ConveyorBelt_A08_PR_NVD_01/ConveyorBelt_A08_PR_NVD_01.usd and join the iot_session live collaboration session. See Joining a Live Session for detailed instructions.

Once you have joined the iot_session, then you should see the following:

iot data in usd

Selecting the /iot/A08_PR_NVD_01 prim in the Stage panel and toggling the Raw USD Properties in the Property panel will provide real-time updates from the the data being pushed by the Python application.

MQTT Ingest Application

To execute the application run the the following:

> python source/ingest_app_mqtt/run_app.py
    -u <user name>
    -p <password>
    -s <nucleus server> (optional default: localhost)

Or if you are using Environment Variables (see Using Environment Variables)

> python source/ingest_app_mqtt/run_app.py

Username and password are of the Nucleus instance (running on local workstation or on cloud) you will be connecting to for your IoT projects.

You should see output resembling:

Received `{
  "_ts": 176.0,
  "System_Current": 0.003981236,
  "System_Voltage": 107.4890366,
  "Ambient_Temperature": 79.17738342,
  "Ambient_Humidity": 45.49172211
  "Velocity": 1.0
}` from `iot/A08_PR_NVD_01` topic
2023-09-19 20:38:24+00:00
Received `{
  "_ts": 178.0,
  "System_Current": 0.003981236,
  "System_Voltage": 107.4890366,
  "Ambient_Temperature": 79.17738342,
  "Ambient_Humidity": 45.49172211
  "Velocity": 1.0
}` from `iot/A08_PR_NVD_01` topic
2023-09-19 20:38:26+00:00

The MQTT ingest application can be found in the ./source/ingest_app_mqtt folder. It will perform the following:

In 'USD Composer' or Kit, open omniverse://<nucleus server>/users/<user name>/iot-samples/ConveyorBelt_A08_PR_NVD_01/ConveyorBelt_A08_PR_NVD_01.usd and join the iot_session live collaboration session. . See Joining a Live Session for detailed instructions.

Once you have joined the iot_session, then you should see the following:

iot data in usd

Selecting the /iot/A08_PR_NVD_01 prim in the Stage panel and toggling the Raw USD Properties in the Property panel will provide real-time updates from the data being pushed by the python application

Containerize headless connector

The following is a simple example of how to deploy a headless connector application into Docker Desktop for Windows. Steps assume the use of

The ollowing has to be done in WSL environment and NOT in Windows environment. Make sure you are in WSL, else you may encounter build and dependency errors.

Once you have a new repo cloned, from within WSL run.

> ./install.sh

Consuming IoT data in USD

Consume the IoT data served by a connector by building your own application logic to visualize, animate and transform with USD stage. The application logic could use one of the following approaches or all of them;

Using an Extension

The sample IoT Extension uses Omniverse Extensions, which are the core building blocks of Omniverse Kit-based applications.

The IoT Extension demonstrates;

  1. Visualizing IoT data
  2. Animating a USD stage using IoT data

To enable the IoT Extension in USD Composer or Kit, do the following:

Open the Extensions panel by clicking on Window > Extensions in the menu and then follow the steps as shown.

open settings

map to extension folder

enabling extension

  1. Visualizing IoT data

The IoT Extension leverages the Omniverse UI Framework to visualize the IoT data as a panel. Find out more about the Omniverse UI Framework

Once you have enabled the IoT extension, you should see IoT data visualized in a Panel.

iot panel

Alternatively, you can launch your app from the console with this folder added to search path and your extension enabled, e.g.:

> app\omni.code.bat --ext-folder exts --enable omni.iot.sample.panel
  1. Animating a USD stage using IoT data

In 'USD Composer' or Kit,

open omniverse://<nucleus server>/users/<user name>/iot-samples/ConveyorBelt_A08_PR_NVD_01/ConveyorBelt_A08_PR_NVD_01.usd.

Ensure the IoT Extension is enabled.

Click on the play icon on the left toolbar of the USD Composer and the extension will animate to the Velocity value change in the IoT data

open settings

and then run one of the following:

    source\ingest_app_csv\run_app.py
        -u <user name>
        -p <password>
        -s <nucleus server> (optional default: localhost)

or

    source\ingest_app_mqtt\run_app.py
        -u <user name>
        -p <password>
        -s <nucleus server> (optional default: localhost)

If you are using Environment Variables (see Using Environment Variables) then run one of the following:

> python source/ingest_app_csv/run_app.py

or

> python source/ingest_app_mqtt/run_app.py

Username and password are for the target Nucleus instance (running on local workstation or on cloud) that you will be connecting to for your IoT projects.

You will see the following animation with the cube moving:

animation playing

When the IoT velocity value changes, the extension will animate the rollers (LiveRoller class) as well as the cube (LiveCube class).

Using ActionGraph

The ConveyorBelt_A08_PR_NVD_01.usd contains a simple ActionGraph that reads, formats, and displays an attribute from the IoT prim in the ViewPort (see Omniverse Extensions Viewport).

To access the graph:

You should see the following:

action graph

The Graph performs the following:

Direct to USD from Headless Connector

Sample demonstrates how to execute USD tranformations from a headless connector using arbtriary values.

To execute the application run the the following:

> python source/transform_geometry/run_app.py
    -u <user name>
    -p <password>
    -s <nucleus server> (optional default: localhost)

Username and password are of the Nucleus instance (running on local workstation or on cloud) you will be connecting to for your IoT projects.

The sample geometry transformation application can be found in source\transform_geometry. It will perform the following:

If you open omniverse://<nucleus server>/users/<user name>/iot-samples/Dancing_Cubes.usd in Composer or Kit, you should see the following:

Rotating Cubes

Joining A Live Session

Here's how-to join a live collaboration session. Click on Join Session

join session

Select iot-session from the drop down to join the already created live session.

joint iot session

API Key Authentication

To authenicate the connector application using an API Key, start Nucleus Explore from the Omniverse Launcher application and right click on the server you wish to connect to and select API Tokens

select API Tokens

Provide a token name and click Create

create API Tokens

Copy the token token value and store it somewhere safe.

If you are using the run_app.py application launcher you can do the following:

> python source/ingest_app_csv/run_app.py
    -u $omni-api-token
    -p <api token>
    -s <nucleus server> (optional default: localhost)

Or if you are using Environment Variables (see Using Environment Variables) you can do the following:

> python source/ingest_app_csv/run_app.py

Using Environment Variables

The samples supports Nucleus authentication via Environment Variables.

For Windows Powershell with User Name/Password:

$Env:OMNI_HOST = "<host name>"
$Env:OMNI_USER = "<user name>"
$Env:OMNI_PASS = "<password>"

For Windows Powershell with API Token:

$Env:OMNI_HOST = "<host name>"
$Env:OMNI_USER = "`$omni-api-token"
$Env:OMNI_PASS = "<API Token>"

For Linux Bash with User Name/Password:

export OMNI_HOST=<host name>
export OMNI_USER=<user name>
export OMNI_PASS=<password>

For Linux Bash with API Token:

export OMNI_HOST=<host name>
export OMNI_USER=\$omni-api-token
export OMNI_PASS=<API Token>