defold / extender

Native extension build server
https://defold.com/manuals/extensions/
MIT License
42 stars 17 forks source link

Add Docker flags to disable specific platforms #252

Closed JustAPotota closed 1 week ago

JustAPotota commented 1 year ago

It would be nice to have a way to exclude specific platforms' setups in the Dockerfile via environment variables so that users don't need to mess with the build process if they don't have SDKs for every platform. I propose adding EXCLUDE_<PLATFORM> build arguments for each supported platform and checking for its existence in every RUN command that requires that SDK. It would look something like:

# build.sh
...
if [ "${EXCLUDE_WINDOWS}" != "" ]; then
    ENV="${ENV} --build-arg EXCLUDE_WINDOWS"
fi
...
# docker-base/Dockerfile
...

#
# Windows
#

ARG EXCLUDE_WINDOWS

...

RUN if [ -z "$EXCLUDE_WINDOWS" ] ; then \
  echo "WIN32 2019 SDK" && \
  mkdir -p ${PLATFORMSDK_WIN32}/MicrosoftVisualStudio2019 && \
  wget -q -O - ${DM_PACKAGES_URL}/Microsoft-Visual-Studio-2019-${WINDOWS_MSVC_2019_VERSION}.tar.gz | tar xz -C ${PLATFORMSDK_WIN32}/MicrosoftVisualStudio2019 \
  ; fi

RUN if [ -z "$EXCLUDE_WINDOWS" ] ; then \
  echo "WIN32 10 SDK" && \
  mkdir -p ${PLATFORMSDK_WIN32}/WindowsKits && \
  wget -q -O - ${DM_PACKAGES_URL}/WindowsKits-${WINDOWS_SDK_10_VERSION}.tar.gz | tar xz -C ${PLATFORMSDK_WIN32}/WindowsKits \
  ; fi

# You get the idea
...

With that setup, the user doesn't need to provide Windows SDKs if they build with:

EXCLUDE_WINDOWS=1 ./server/scripts/build.sh

I did something very similar in my Rust fork, but that used a whitelist instead of blacklist which would break existing workflows.

JCash commented 1 year ago

I agree that we need something to control which platforms to build for. Personally, I would construct different Dockerfiles somehow, and compose them together.

In my mind, our setup would be a lot easier to maintain if we could have one server for each PLATFORM+PLATFORM_SDK version.

Currently, for each platform we tend to use 2 platform sdks within the same container. Not only does it create a size issue for our containers, but it also hinders refactorings a bit. We often have to wait in order to update all the scripts and phase an sdk out.

Also, some sdks have frequent updates, and if we sometimes change the sdk for users that haven't really updated anything on their side, which can be problematic.

An idea that I have is to have the servers "linked", so that when a request comes in they get sorted this way.

build.defold.com
    Win32+Win64
        Defold version 1.3.4+: WinSDK 10.11786 ...
        Defold version 1.3.0+: WinSDK 10.10001 ...
    Android
        Defold version 1.3.5+: NDK 29 ...
        Defold version 1.3.0+: NDK 23 ...

    macOS:
        build-darwin.defold.com
            A bit special, since we dont use docker there.
aglitchman commented 1 year ago

It would be nice to have a way to exclude specific platforms' setups in the Dockerfile via environment variables so that users don't need to mess with the build process if they don't have SDKs for every platform.

That would be a great option!

From my experience - I didn't have an easy access to an Apple computer with XCode SDKs of the required versions, so I had to comment out all iOS/macOS dependencies to run the extender.

mikatuo commented 1 year ago

@JCash, perhaps this could be useful: Keep Your Dockerfile Clean

A simple real world example of a Dockerfile for .NET (here). They build the dll-s with the SDK, but then final container is based on dotnet runtime without the SDK. No cleanup needed, looks nice.