adamrehn / ue4-docker

Windows and Linux containers for Unreal Engine 4
https://docs.adamrehn.com/ue4-docker/
MIT License
784 stars 171 forks source link

Installation of ue4 docker is working fine but having issues when using it to build images #312

Closed dstkibrom closed 1 year ago

dstkibrom commented 1 year ago

Output of the ue4-docker info command:

ue4-docker version:         0.0.106 (latest available version is 0.0.106)
Operating system:           Linux (Ubuntu 20.04.6 LTS, 5.15.0-69-generic)
Docker daemon version:      23.0.3
NVIDIA Docker supported:    Yes
Maximum image size:         No limit detected
Available disk space:       1.54 TiB
Total system memory:        46.8 GiB physical, 2 GiB virtual
CPU:                        12 physical, 20 logical (x86_64)

Additional details:

I am trying to use ue4-docker to run a simulator called AirSim. The command that i need to build is:

ue4-docker build 4.17.2 --cuda=10.0 --no-full

when running the above command, I get the following error:

 19.48 Get:37 http://archive.ubuntu.com/ubuntu bionic-backports/main i386 Packages [64.0 kB]
#0 19.60 Reading package lists...
#0 20.59 W: GPG error: https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY A4B469963BF863CC
#0 20.59 E: The repository 'https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  InRelease' is not signed.
------
Dockerfile:23
--------------------
  22 |     # Install our build prerequisites
  23 | >>> RUN apt-get update && apt-get install -y --no-install-recommends \
  24 | >>>      build-essential \
  25 | >>>      ca-certificates \
  26 | >>>      curl \
  27 | >>>      git \
  28 | >>>      git-lfs \
  29 | >>>      python3 \
  30 | >>>      python3-dev \
  31 | >>>      python3-pip \
  32 | >>>      shared-mime-info \
  33 | >>>      software-properties-common \
  34 | >>>      sudo \
  35 | >>>      tzdata \
  36 | >>>      unzip \
  37 | >>>      xdg-user-dirs \
  38 | >>>      zip && \
  39 | >>>  rm -rf /var/lib/apt/lists/* && echo '' && echo 'RUN directive complete. Docker will now commit the filesystem layer to disk.' && echo 'Note that for large filesystem layers this can take quite some time.' && echo 'Performing filesystem layer commit...' && echo ''
  40 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c apt-get update && apt-get install -y --no-install-recommends \t\tbuild-essential \t\tca-certificates \t\tcurl \t\tgit \t\tgit-lfs \t\tpython3 \t\tpython3-dev \t\tpython3-pip \t\tshared-mime-info \t\tsoftware-properties-common \t\tsudo \t\ttzdata \t\tunzip \t\txdg-user-dirs \t\tzip && \trm -rf /var/lib/apt/lists/* && echo '' && echo 'RUN directive complete. Docker will now commit the filesystem layer to disk.' && echo 'Note that for large filesystem layers this can take quite some time.' && echo 'Performing filesystem layer commit...' && echo ''" did not complete successfully: exit code: 100
[ue4-docker build] Error: failed to build image "adamrehn/ue4-build-prerequisites:cudagl10.0-ubuntu18.04".

I know the work around for this error in my workstation. I just need to add the following to my docker file and it will run without problem. I got stumbled to this solution when I was trying to address similar issue in a case not related to ue4-docker.

RUN rm /etc/apt/sources.list.d/cuda.list
RUN rm /etc/apt/sources.list.d/nvidia-ml.list

To add the above two lines to a docker file of ue4-docker, i need to build ue4-docker from source but there seems tobe no resource explaining on how users can install ue4-docker from source. Is there any resource explaning how to install ue4-docker from source or would you suggest a solution to the problem that I am having while building ue4-docker?

TBBle commented 1 year ago

Oops, we diagnosed this in a discussion earlier but never actioned it. >_< I didn't realise until now that there isn't a matching bug report.

The proposed fix is to change our CUDA base image to a supported image (the current one is EOL'd), per https://github.com/adamrehn/ue4-docker/discussions/294#discussioncomment-4778592. In the meantime, your existing workaround is viable. If you need cuda.list to be functional, instead of deleting it, this apparently works:

RUN [ ! -e /etc/apt/sources.list.d/nvidia-ml.list ] \
    || rm /etc/apt/sources.list.d/nvidia-ml.list
RUN apt-key del 7fa2af80 \
    && apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub

As far as running from source, there's no specific documentation because it's done in a Python-standard way: Something like pip install . should work to install it from source (it'll build a package/wheel locally and then install it like a downloaded package). "Something like" here is because your local environment and use of venvs will determine if that's pip, pip3, python3 -m pip, python -m pip, etc. You'll probably also want the --user flag if you're not trying to do a system-wide install, and you're not inside a venv.

If you pip install --editable . then if you meet the requirements, you'll get an "editable" install, and hence your source checkout edits will be seen by the installed package, rather than having to reinstall it after each change. (In effect, the "install" is just a link back to your source checkout, so if you delete the checkout, editable install breaks.)

slonopotamus commented 1 year ago

I believe this is fixed by https://github.com/adamrehn/ue4-docker/pull/315