running launchClient offline

[onurkulak]

Hi all,

I am trying to train an AI using the Malmo in my university cluster, but the cluster does not have connection to the internet and I have 0 knowledge of Gradle.

What I am trying: I am building a container in my laptop with (I think) everything installed, and transferring it to the cluster. Then I submit my script to the cluster, it runs the script offline in a compute node.

But I get connection errors from gradle, even though the packages should be there.

So my problem is how can I preinstall everything, then run the Malmo without connecting to the Internet?

the error is:

Starting a new Gradle Daemon for this build (subsequent builds will be faster).

FAILURE: Build failed with an exception.

* What went wrong:
A problem occurred configuring root project 'Minecraft'.
> Could not resolve all dependencies for configuration ':classpath'.
   > Could not resolve net.minecraftforge.gradle:ForgeGradle:2.2-SNAPSHOT.
     Required by:
         :Minecraft:unspecified
      > Could not resolve net.minecraftforge.gradle:ForgeGradle:2.2-SNAPSHOT.
         > Unable to load Maven meta-data from http://files.minecraftforge.net/maven/net/minecraftforge/gradle/ForgeGradle/2.2-SNAPSHOT/maven-metadata.xml.
            > Could not GET 'http://files.minecraftforge.net/maven/net/minecraftforge/gradle/ForgeGradle/2.2-SNAPSHOT/maven-metadata.xml'.
               > Network is unreachable (connect failed)
      > Could not resolve net.minecraftforge.gradle:ForgeGradle:2.2-SNAPSHOT.
         > Unable to load Maven meta-data from https://jcenter.bintray.com/net/minecraftforge/gradle/ForgeGradle/2.2-SNAPSHOT/maven-metadata.xml.
            > Could not GET 'https://jcenter.bintray.com/net/minecraftforge/gradle/ForgeGradle/2.2-SNAPSHOT/maven-metadata.xml'.
               > Connect to jcenter.bintray.com:443 [jcenter.bintray.com/54.187.47.108, jcenter.bintray.com/35.161.162.245, jcenter.bintray.com/52.43.200.1, jcenter.bintray.com/52.88.32.158] failed: Connection timed out (Connection timed out)

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 17 mins 39.393 secs

code in my container build:

Bootstrap: library
From: ubuntu:16.04
Stage: build

%runscript
    xpra start :100
    export DISPLAY=:100
    # export CLASSPATH=`find *//opt *//usr -name "*.jar" -exec dirname {} \; | cut -d '/' -f 2- | sort -u | paste -sd ":" -`
    # echo $CLASSPATH
    cd /opt/malmo/Minecraft
    export GRADLE_USER_HOME=$PWD/.gradle
    ./launchClient.sh "$@"

%post
    apt -y update 
    apt -y upgrade
    apt install -y libboost-all-dev 
    apt install -y python3-pip
    apt install -y openjdk-8-jdk 
    apt install -y lua5.1 
    apt install -y libxerces-c3.1 
    apt install -y liblua5.1-0-dev 
    apt install -y libav-tools 
    apt install -y python-tk 
    apt install -y python-imaging-tk 
    apt install -y ffmpeg 
    apt install -y wget 
    apt install -y unzip 
    apt install -y make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev  libncursesw5-dev xz-utils tk-dev
    apt install -y git
    # skip the interactive keyboard question
    DEBIAN_FRONTEND=noninteractive apt install -y xpra

    cd opt 
    git clone https://github.com/Microsoft/malmo.git
    cd malmo/Minecraft
    (echo -n "malmomod.version=" && cat ../VERSION) > ./src/main/resources/version.properties

    # precompile stuff
    mkdir .gradle 
    export GRADLE_USER_HOME=$PWD/.gradle
    # I don't feel like this part is working? still getting a daemon message
    echo 'org.gradle.daemon=true\n' > .gradle/gradle.properties
    ./gradlew setupDecompWorkspace
    ./gradlew build
    chmod -R 777 ..

    mkdir /homes

%environment
    MALMO_XSD_PATH=/opt/malmo/Schemas
    export MALMO_XSD_PATH
    export LC_ALL=C

explanation: I am building the Singularity container in the %post section, which I am also running the "./gradlew setupDecompWorkspace", "./gradlew build" commands once. I thought these would be enough for pre-install?

Then I run my container using the %run section.

What I tried so far: I created a modified launchclient script without the setupDecompWorkspace and build steps, but this script didn't work either. I created a modified launchClient script and passed --offline option to the gradlew commands, but this didn't work too. I don't have the error logs for these two trials.

[Howuhh]

@onurkulak Were you able to find a solution?