Initial repo for PTG project.
This repo contains:
Note: This repository contains submodules that will need to be initialized upon first checkout:
$ git submodule update --init --recursiveUnity Hub - 3.0.0 - https://unity.com/unity-hub
Unity - 2020.3.25f1 - https://unity3d.com/get-unity/download/archive
Visual Studio 2019 - 16.11.8 - https://visualstudio.microsoft.com/vs/older-downloads/
Mixed Reality Feature Tool - 1.02111-Preview - https://www.microsoft.com/en-us/download/details.aspx?id=102778
HoloLens 2 headset - OS version 20438.1432
Anaconda/Miniconda - py3.9
HL2SS plugin - 1.0.15 - https://github.com/jdibenes/hl2ss
1) Open the unity/ARUI project with Unity Hub.
2) Open the arui_engineering scene in the Unity editor (unity\ARUI\Assets\my_scene).
3) Modify the default project build settings (File -> Build Settings...).
Switch Platform after applying the new settings.Build and specify your desired build folder. After the build completes, a new file explorer windows will pop up with the location of the build.
7) Open the .sln file with Visual Studio.Solution Configurations to Release and Solution Platforms to ARM64.Solution Explorer pane, right-click Angel_ARUI and select Properties) and switch to the Configurations -> Debugging tab. Enter the IP address of your HoloLens in the Machine Name field.
10) Deploy the app to the Hololens by clicking Build -> Deploy Solution
11) After deployment completes, open the Windows menu in the Hololens and select All Apps, and then click on the Angel_ARUI application.1) In Unity, click Robotics -> ROS Settings to open the ROS Settings menu.
2) Set the protocol to ROS2.
3) Enter the IP address of the machine the TCP endpoint node will be running on (i.e. the machine the HoloLens 2 will be connecting to) and close the ROS Settings menu.
1) In Unity, click Robotics -> Generate ROS Messages... to open the ROS Message Browser.
2) Set the ROS message path to the directory containing the ROS2 messages for the project (../../ros/angel_msgs for this project's ANGEL message folder). The Message Browser should display the .msg files it found in the ROS message path.
3) Click Build msgs and wait for message generation to finish. You should now see new files in the Built message path location (default location is unity/ARUI/Assets/RosMessages).
4) Close the ROS Message Browser.
See Unity README.md for instructions on creating an application package and installing it via the HoloLens 2 device portal.
ROS 2 Foxy is utilized to support our system of streaming analytics and report-back to the HoloLens2 platform.
Workspace root: ./ros/
System requirements
apt install docker-compose-plugin
OR
pip install docker-composeSome files required from the https://data.kitware.com Girder service require
authentication due to their protected nature.
The environment variable GIRDER_API_KEY must be defined with a valid API key,
otherwise an authentication token cannot be retrieved.
We currently require a pip-installed docker-compose tool, as opposed to a
system package-manager installed docker-compose-plugin package.
The package manager docker plugin behaves a little differently that our current
docker-compose configuration and scripting does not yet handle.
External large files should be provisioned by running the ansible tool:
./angel-provision-files.shThis may include large files for running the system, like ML model files, or other files required for building docker images.
This provisioning may require additional configuration and variables set in your environment in order to satisfy some permissions:
GIRDER_API_KEY will need to be set in order to acquire protected files from
data.kitware.com.The configuration that controls what is staged and where is located
in the ansible/roles/provision-files/vars/main.yml file.
We incorporate various submodules in this repository to reference externally developed or maintained packages.
Some of these packages however are things that we have authored to support this system but in a modular way to decouple their usage and functionality from this system. We may desire to develop or fix aspects of those submodules while developing or testing aspects of ANGEL System. We can test/develop/fix these submodules right in their checked-out locations within this repository.
cd into the submodule in question../angel-workspace-shell.sh, build and test as desired.Intention: Use containerization to standardize development and runtime environment and practice.
Docker functionality is located under the ./docker/ directory.
workspace-base-dev provides a base environment that supports building and
running our workspace.workspace-build provides a build of our workspace.Quick-start
./angel-docker-build.shQuick-start
# Start the containerized environment shell.
./angel-workspace-shell.sh
# Build your local workspace for use.
./workspace_build.shTo develop adequately in docker, the container environment needs to have access
to the source code that is being edited, likely by an IDE on the host.
Running ./angel-workspace-shell.sh will create a temporary docker container
and drop into a bash shell.
This script has some usage output that can be seen by passing a -h/--help
option.
By default, the workspace-shell-dev-gpu service will be run.
The definition of this service is found in the docker/docker-compose.yml
configuration.
This will mount the ./ros/ subtree on the host system into the
/angel_workspace/src/ directory in the run container.
This "workspace" context additionally mounts build, install and log output directories to a spot on the host in order to:
Due to this, there will not be a build available upon first shell start-up.
The script /angel_workspace/workspace_build.sh is available to run.
This script is used during the image build process, so using this script
ensures that the same build method is performed.
Other directories and files are mounted into the container environment for development purposes and external file sharing.
This shell will NOT have the local installation sourced in order to
facilitate further safe build actions.
ros2 run actions should be performed in a separate shell from where builds
are performed.
NOTE that any new system requirements may of course be installed locally, but these will be lost upon container shutdown.
./docker/workspace-build/ image definition../docker/workspace-build/ image definition../docker/workspace-base-dev image definition.A basic template config may be auto-generated to specify which host network
interface for it to use by uncommenting and setting the CYCLONE_DDS_INTERFACE
in the docker/.env or exporting it on your commandline to the desired value.
The "standard" way to set up and run collections of nodes in ROS2 is via launch files. This is all well and good and provides great features, but what it doesn't do is provide access to individual components at runtime. During development, and often in the field as well, it is important to be able to get at individual components to see what might be going wrong or to restart/tweak one thing without impacting everything else that is running. We utilize tmux, configured and run by tmuxinator, to manage multiple windows/panes to host individual components.
Tmuxinator configurations are stored in the ./tmux/ directory.
./angel-workspace-shell.sh -r -- tmuxinator start fragment_object_detection_debugAnatomy of the call:
1) (Re)uses angel-workspace-shell.sh to run a command in a docker container.
2) -r option to script sets "use our built workspace" flag.
3) Things to the right of -- are run inside the container.
4) tmux directory is (by default) in the same directory as the working
directory the container starts in.
5) tmuxinator command creates a new server session defined by the given
config.
To stop the system after using the above command, we simply need to exit the
tmux instance or kill the tmux session.
This can be done by providing the tmux keyboard command <Ctrl-B, D>.
Alternatively, in a new or existing pane, running tmux kill-session or
tmux kill-server.
When using the above example "start" command, exiting the tmux session will
also shut down the container.
If starting an angel-workspace-shell.sh -r first, dropping into a bash
terminal, and then calling tmuxinator start fragment_object_detection_debug,
we have the option to exit the tmux session and stop it using another
tmuxinator command:
tmuxinator stop fragment_object_detection_debugSome ROS nodes utilize the smqtk-core plugin system so specification of what algorithm is utilized is determined based on a configuration file.
Configuration files are currently located in: ros/angel_system_nodes/configs/
default_object_det_config.json
angel_system_nodes object_detector.py node.default_activity_det_config.json
angel_system_nodes activity_detector.py node.The annotation_event_monitor ROS node uses the up and down arrow keys to
generate AnnotationEvent messages. These messages can be used to determine
when the beginning and end of an activity or error occur during a recording.
To help with this process, a foot pedal can be used to map foot pedal presses to keyboard presses. For this system, we are using the Infinity 3 USB foot pedal.
To configure your Linux system to recognize foot pedal presses as keyboard presses, see this guide.
In the .hwdb file, make sure to map the keyboard presses to the up and down arrow keys.
angel_system/ contains the interfaces and implementations for the
various components in the ANGEL system python package.
See angel_system/eval/README.md for details.
rosdepReferences to the lists that rosdep uses to resolve names: /etc/ros/rosdep/sources.list.d/20-default.list