search

qcr / benchbot

BenchBot is a tool for seamlessly testing & evaluating semantic scene understanding tools in both realistic 3D simulation & on real robots
BSD 3-Clause "New" or "Revised" License
110 stars 12 forks source link

unable to install benchbot, issac-sim docker pull fails #37

Closed cjpurackal closed 2 years ago

cjpurackal commented 2 years ago

The issac-sim docker pull fails at round 1.74GB. I've tried this multiple times at different times. What could possible be the issue here? PS: I'm running this on a 90 GB partition.

` sha256:059811a8ca6017b38b060975cacf598979aa4326921f1e6f7584da1278935a68 1.74GB / 3.74GB FROM nvcr.io/nvidia/isaac-sim:2021.2.1@sha256:4a9a1cfadc276c72dc15b64fde4b08c1030ca92005f480605950e54c324c0708 error: failed to solve: failed commit on ref "layer-sha256:059811a8ca6017b38b060975cacf598979aa4326921f1e6f7584da1278935a68": "layer-sha256:059811a8ca6017b38b060975cacf598979aa4326921f1e6f7584da1278935a68" failed size validation: 1742733312 != 3743189127: failed precondition

ERROR: Building BenchBot simulator 'sim_omni' returned a non-zero error code: 1

Ensure that Docker has been installed correctly AND that you can run Docker WITHOUT root access (there is no need to ever run Docker with root). See https://docs.docker.com/install/linux/linux-postinstall/ for details on how to fix this.

If the error is more generic, please contact us so that we can update our pre-install host system checks. `

btalb commented 2 years ago

Thanks for sharing this @cjpurackal .

This error appears to be coming from further down, at the Docker level.

This issue (containerd/containerd#2306) seems to suggest it can be caused by an incomplete download of a layer.

Can you try doing the following:

  1. Check the space on your partition, how much free space do you have remaining?
  2. Delete any downloaded images so the download can start again from scratch. A system prune (docker system prune) should be sufficient (double check the images are gone from your docker images list afterwards).
cjpurackal commented 2 years ago

When I'm installing benchbot, I have around 71 GB of free memory. I did try docker system prune (removed all the images) but still getting the same error. Do I need to have more memory ?

btalb commented 2 years ago

I tried reproducing this locally but haven't had any success.

After an installation has failed can you show me how much free space you have remaining by posting the output of df -h please.

cjpurackal commented 2 years ago

Seems like it worked when I tried today but there seems to be an issue when I'm running it (benchbot_run --robot carter_omni --env miniroom:1 --task semantic_slam:active:ground_truth). The issue seems similar to #18

btalb commented 2 years ago

I'm going to close this issue off @cjpurackal .

If the remedies in #18 don't help resolve your issue, please open another issue with some logs and I'll be happy to help.