To get started setup the required environments by following below steps
If you install transport_challenge , it will automatically install tdw and magnebot. The transport_challenge repo has instructions for how to downgrade magnebot and tdw .
The dataset is modular in its design, consisting of several physical floor plan geometries with a wall and floor texture variations (e.g. parquet flooring, ceramic tile, stucco, carpet etc.) and various furniture and prop layouts (tables, chairs, cabinets etc.), for a total of 15 separate environments. There are 10 kinds of scenes in training dataset and 5 kinds of scenes in testing. Every scene has 6 to 8 rooms, 8 objects, and some containers.
move forward at 0.5
dict {"type": 0} turn left 15 degrees
dict {"type": 1} turn right 15 degrees
dict {"type": 2} grasp the object with arm
dict {"type": 3, "object": object_id, "arm": "left" or "right"} put the object into the container
dict {"type": 4, "object": object_id, "container": container_id} drop objects
dict {"type": 5}Makesure the environment is setup by following above instructions
Here is an example of how to instantiate an environment and use a simple baseline agent.
import gym
import pickle
import pkg_resources
from tdw_transport_challenge.h_agent import H_agent
from agent import init_logs
# Create gym environment.
env = gym.make("transport_challenge-v0", train = 0, physics = True, port = 1071, launch_build=False)
# Load training scenes
with open(pkg_resources.resource_filename("tdw_transport_challenge", "train_dataset.pkl"), 'rb') as fp:
dataset = pickle.load(fp)
# Load training scene. scene_number is from 0 - 100
scene_number = 0
obs, info = env.reset(scene_info=dataset[scene_number])
# create logger
logger = init_logs()
# Instantiate baseline agent
agent = H_agent(logger=logger)
while True:
action = agent.act(obs, info)
obs, rewards, done, info = env.step(action)
if done:
break
env.close()Set the TRANSPORT_CHALLENGE environment variable to path of the downloaded data.
# Run TDW with port 1071
./TDW/TDW.x86_64 -port 1071 &
# Run the python file
python test.pyIf you set launch_build=True then build will automatically launch with your controller and running ./TDW/TDW.x86_64 -port 1071 & separately is not required.
To run multiple environments / vectorize environments you can use stable baselines. Here is an example:
from stable_baselines.common.vec_env.subproc_vec_env import SubprocVecEnv
import gym
import random
def create_env(port):
env = gym.make("transport_challenge-v0",train = 0, physics = True, port = port, launch_build=True)
env.reset()
return env
def make_env(port):
def _init():
env = create_env(port)
return env
return _init
def make_vec_env(num_processes):
ports = random.sample(range(1071, 1171), num_processes)
env = SubprocVecEnv([make_env( port=ports[i]) for i in range(num_processes)])
return env
def main():
num_env = 5
v_env = make_vec_env(num_env)
obs = v_env.reset()
for i in range(10):
action = get_action(obs)
obs, rewards, dones, info = v_env.step(action)
v_env.render()
v_env.close()
if __name__ == '__main__':
main()
Follow these steps to set up the environment to run docker container. Once completed
you can edit the run_locally.sh file to run your python code.
Next build the container
docker build --no-cache -t submission_image .Run container
nvidia-docker run --network host --env="DISPLAY=:4" --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" --volume="/tmp/output:/results" -e NVIDIA_DRIVER_CAPABILITIES=all -e TRANSPORT_CHALLENGE=file:////model_library -e NUM_EVAL_EPISODES=1 -d submission_image sh run_baseline_agent.sh 7845Running multiple environments
To prepare your agent for submission instantiate your agent in agent.py. This file will
test your agent on testing scenes. Use docker build command
docker build --no-cache -t tdwbase --build-arg TDW_VERSION=1.8.4 . to create final submission image. You can follow
the instructions on EvalAI Challenge page for submission of the image.
To Make a submission:
Create account on https://eval.ai/
Find the challenge: Dashboard -> All Challenges, and look for TDW-Transport Challenge
Clickl on participation tab :
Then click on submit tab and follow steps to setup Evalai cli and submit the image
Before submitting the image make sure that the docker image works by evaluating locally. To do this use this docker run command
mkdir /tmp/output
nvidia-docker run --network host --env="DISPLAY=:4" --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" --volume="/tmp/output:/results" -e NVIDIA_DRIVER_CAPABILITIES=all -e TRANSPORT_CHALLENGE=file:////model_library -e NUM_EVAL_EPISODES=1 -d submission_image sh run_submission.sh 7845In this command the Display server is :4 (see this for setting up display server). Local directory /tmp/output is volume mapped. The evaluation result will be generated in this directory and can be accessed even after the container has finished.
TDW can be run in a docker container on a remote server with GPU. The container needs access to x-server in order to leverage GPU for 3-D acceleration. The contanerization process mentioned here is only supported on a headless remote machine with Ubuntu.
At this point we have only tested this on a remote ubuntu machine (16.04 and 18.04). The user will need to have sudo access in-order for x-server to work
sudo apt-get install -y gcc make pkg-config xorgnvidia-xconfig --query-gpu-info. This will list all the GPUs and their bus idsnvidia-xconfig --no-xinerama --probe-all-gpus --use-display-device=none. This will generate xorg.conf
(/etc/X11/xorg.conf) file nvidia-xconfig --query-gpu-info nvidia-xconfig --query-gpu-info outputs two gpus:
Number of GPUs: 2GPU #0: Name : Tesla V100-PCIE-16GB UUID : GPU-bbf6c915-de29-6e08-90e6-0da7981a590b PCI BusID : PCI:0:7:0
Number of Display Devices: 0
GPU #1: Name : Tesla V100-PCIE-16GB UUID : GPU-2a69c672-c895-5671-00ba-14ac43a9ec39 PCI BusID : PCI:0:8:0
Number of Display Devices: 0
Then create two xorg.conf files and edit the device section for first file:
```text
This ->
Section "Device"
Identifier "Device0"
Driver "nvidia"
VendorName "NVIDIA Corporation"
EndSection
To ->
Section "Device"
Identifier "Device0"
Driver "nvidia"
VendorName "NVIDIA Corporation"
BoardName "Tesla V100-PCIE-16GB"
BusID "PCI:0:7:0"
EndSection
And for the second file:
This ->
Section "Device"
Identifier "Device0"
Driver "nvidia"
VendorName "NVIDIA Corporation"
EndSection
To ->
Section "Device"
Identifier "Device0"
Driver "nvidia"
VendorName "NVIDIA Corporation"
BoardName "Tesla V100-PCIE-16GB"
BusID "PCI:0:8:0"
EndSectionsudo nohup Xorg :<Display server name> -config <configuration file name> &
e.g.
sudo nohup Xorg :1 -config /etc/X11/xorg-1.conf &
sudo nohup Xorg :2 -config /etc/X11/xorg-2.conf &
.
.
.
sudo nohup Xorg :n -config /etc/X11/xorg-n.conf &nvidia-smi should show the x-server proccess with corresponding gpu version
You can follow docker and nvidia-docker installation instructions
Finally, build the container by doing
docker build --no-cache -t submission_image .You can test the code by running a simple baseline agent. You can replace /tmp/output with any other directory.
Replace :4 with display server name created in previous step
nvidia-docker run --network none --env="DISPLAY=:4" --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" --volume="/tmp/output:/results" -e NVIDIA_DRIVER_CAPABILITIES=all -e TRANSPORT_CHALLENGE=file:////model_library -e NUM_EVAL_EPISODES=1 -d submission_image sh run_baseline_agent.sh 7845