search

firebelley / godot-export

Automatically exports your Godot games.
MIT License
461 stars 53 forks source link

Update README for Ubuntu 20.04 & Godot Mono Prerequisite #51

Closed knightofiam closed 3 years ago

knightofiam commented 3 years ago

Since GitHub Actions has updated ubuntu-latest to 20.04, Godot Mono from 3.2.3 onward, requires .NET Core 3.1 SDK or .NET 5 SDK (both the latest as of this writing) to be installed. Just to clarify, previously, Ubuntu 18.04 + Godot Mono 3.2.3-stable + godot-export@v2.6.1 worked just fine without installing a .NET SDK).

Otherwise, you will get errors like the following:

>Export binaries

...

Mono: Failed to load Core API assembly
Mono: Failed to load project assembly

...

ERROR: godot_icall_GD_pusherror: Failed to export project: Failed to build project
   At: modules/mono/glue/gd_glue.cpp:250.

...

System.Exception: Failed to build project
  at GodotTools.Export.ExportPlugin._ExportBeginImpl (System.String[] features, System.Boolean isDebug, System.String path, System.Int32 flags) [0x0007e] in <c8b5a5308c6141d8acc11a71c13bb4ae>:0 
  at GodotTools.Export.ExportPlugin._ExportBegin (System.String[] features, System.Boolean isDebug, System.String path, System.Int32 flags) [0x0000b] in <c8b5a5308c6141d8acc11a71c13bb4ae>:0

Something like https://github.com/actions/setup-dotnet can be suggested to solve this. (It would have saved me many hours of frustration. See comments for more information):

- name: install .net sdk
  uses: actions/setup-dotnet@v1
    with:
      dotnet-version: '5.0.x'

Update: It's also required to use a specific custom editor setting (see comments for more information):

[resource]
mono/builds/build_tool = 3

to allow Godot to run dotnet msbuild during export instead of just msbuild.

Please see this issue comment with following solution and this Godot documentation section for more information about this problem.

knightofiam commented 3 years ago

Upon closer inspection, it seems this has less to do with the version of Ubuntu and more to do with the new csproj format Godot is using, which requires the .NET SDK installation, according to the aforementioned documentation. Either way, it's the new standard going forward so it needs to be mentioned as an explicit prerequisite imo.

knightofiam commented 3 years ago

It may even make sense to incorporate its installation by default, possibly with an option to disable it for special cases or backward compatibility.

firebelley commented 3 years ago

Thanks for the report, I will do some testing and see what can be done. It's possible the best workaround at the moment is to use an older version of Ubuntu.

knightofiam commented 3 years ago

@firebelley Thanks for the update. I'm doing my own research and testing in a Ubuntu VM to figure out the best workaround, as simply installing a .NET SDK wasn't enough, but may be part of the puzzle. I've seen multiple people dealing with these errors lately, although I have managed to get things working on a local Ubuntu 20.04 VM. I'm just trying to narrow down the exact configuration changes needed. I'll report back when I can, but yes please let me know if you can reproduce the issue.

If it helps any, I'm targeting net472. Using .NET 5 SDK works fine with that target when building / running from Rider / Godot Editor, for example. What I meant by .NET 5 SDK doesn't work yet, is that it doesn't work as a target. It does / should work targeting net472 in csproj, even with language level 9.

knightofiam commented 3 years ago

@firebelley The following setup is working for me on Ubuntu 20.04 & Godot Mono Linux Headless 3.3-rc9. I created it from examining your source code. I'm not sure why this works and your code doesn't. You'd probably be able to spot the difference better than I. The only difference I can see in the build log when using your Action is it seems to call msbuild for some strange reason, which seems unnecessary when exporting. To get msbuild to work with this setup, I had to prefix it with the dotnet command from the dotnet-sdk installation in a previous Action step, as in:

"/snap/bin/dotnet" msbuild "${HOME}/work/coa/coa/coa.sln" /restore /t:Build "/p:Configuration=ExportRelease" /v:normal "/l:GodotTools.BuildLogger.GodotBuildLogger,${HOME}/.local/share/godot/godot_executable/Godot_v3.3-rc9_mono_linux_headless_64/GodotSharp/Tools/GodotTools.BuildLogger.dll;${HOME}/.local/share/godot/mono/build_logs/${GITHUB_SHA}_ExportRelease" /p:GodotTargetPlatform=windows

but that step is completely unnecessary to create a working executable exported with:

${HOME}/.local/share/godot/godot_executable/Godot_v3.3-rc9_mono_linux_headless_64/godot ${HOME}/work/coa/coa/project.godot --export coa-windows ${HOME}/.local/share/godot/builds/coa-windows/coa.exe --verobse

csproj:

<Project Sdk="Godot.NET.Sdk/3.3.0">
    <PropertyGroup>
        <TargetFramework>net472</TargetFramework>
        <LangVersion>9.0</LangVersion>
    </PropertyGroup>
</Project>

relevant yml excerpt:

...

- name: create executables
  run: |
    mkdir -p ${HOME}/.config/godot
    mkdir -p ${HOME}/.local/share/godot/builds/coa-mac
    wget --no-verbose https://downloads.tuxfamily.org/godotengine/3.3/rc9/mono/Godot_v3.3-rc9_mono_export_templates.tpz -O ${HOME}/.local/share/godot/godot_templates.tpz
    wget --no-verbose https://downloads.tuxfamily.org/godotengine/3.3/rc9/mono/Godot_v3.3-rc9_mono_linux_headless_64.zip -O ${HOME}/.local/share/godot/godot.zip
    7z x ${HOME}/.local/share/godot/godot.zip -o${HOME}/.local/share/godot/godot_executable -y > nul
    mv ${HOME}/.local/share/godot/godot_executable/Godot_v3.3-rc9_mono_linux_headless_64/Godot_v3.3-rc9_mono_linux_headless.64 ${HOME}/.local/share/godot/godot_executable/Godot_v3.3-rc9_mono_linux_headless_64/godot
    chmod +x ${HOME}/.local/share/godot/godot_executable/Godot_v3.3-rc9_mono_linux_headless_64/godot
    unzip -q ${HOME}/.local/share/godot/godot_templates.tpz -d ${HOME}/.local/share/godot
    mv ${HOME}/.local/share/godot/templates ${HOME}/.local/share/godot/tmp
    mkdir -p ${HOME}/.local/share/godot/templates/
    mv ${HOME}/.local/share/godot/tmp ${HOME}/.local/share/godot/templates/3.3.rc9.mono
    cp ${HOME}/work/coa/coa/editor_settings-3.tres ${HOME}/.config/godot/
    ${HOME}/.local/share/godot/godot_executable/Godot_v3.3-rc9_mono_linux_headless_64/godot ${HOME}/work/coa/coa/project.godot --export coa-windows ${HOME}/.local/share/godot/builds/coa-windows/coa.exe
    ${HOME}/.local/share/godot/godot_executable/Godot_v3.3-rc9_mono_linux_headless_64/godot ${HOME}/work/coa/coa/project.godot --export coa-mac ${HOME}/.local/share/godot/builds/coa-mac/coa-mac.zip
    ${HOME}/.local/share/godot/godot_executable/Godot_v3.3-rc9_mono_linux_headless_64/godot ${HOME}/work/coa/coa/project.godot --export coa-linux ${HOME}/.local/share/godot/builds/coa-linux/coa.x86_64
    7z a ${HOME}/.local/share/godot/dist/coa-windows.zip ${HOME}/.local/share/godot/builds/coa-windows/* -y > nul
    cp ${HOME}/.local/share/godot/builds/coa-mac/coa-mac.zip ${HOME}/.local/share/godot/dist/
    7z a ${HOME}/.local/share/godot/dist/coa-linux.zip ${HOME}/.local/share/godot/builds/coa-linux/* -y > nul

...
knightofiam commented 3 years ago

Ok, upon further testing and examination, with pre-installation of a dotnet-sdk, your Action now seems to correctly preface the msbuild command with dotnet. What I find very strange, is that in spite of all these errors, the resulting exported executables still work (on macOS and Ubuntu 20.04, haven't tested Windows):

>Run firebelley/godot-export@v2.6.1
>Download Godot
>Adding Editor Settings
>Export binaries
  Using project file at /home/runner/work/coa/coa/project.godot

  /home/runner/.local/share/godot/godot_executable/Godot_v3.3-rc9_mono_linux_headless_64/godot /home/runner/work/coa/coa/project.godot --export coa-windows /home/runner/.local/share/godot/builds/coa-windows/coa.exe --verbose
  Mono: Failed to load Core API assembly
  Mono: Failed to load project assembly
  Running: "/snap/bin/dotnet" msbuild "/home/runner/work/coa/coa/coa.sln" /restore /t:Build "/p:Configuration=ExportRelease" /v:normal "/l:GodotTools.BuildLogger.GodotBuildLogger,/home/runner/.local/share/godot/godot_executable/Godot_v3.3-rc9_mono_linux_headless_64/GodotSharp/Tools/GodotTools.BuildLogger.dll;/home/runner/.local/share/godot/mono/build_logs/b400f1ea4543f8d078e2dff5d872887b_ExportRelease" /p:GodotTargetPlatform=windows
  Godot Engine v3.3.rc9.mono.official - https://godotengine.org

...

  /home/runner/.local/share/godot/godot_executable/Godot_v3.3-rc9_mono_linux_headless_64/godot /home/runner/work/coa/coa/project.godot --export coa-mac /home/runner/.local/share/godot/builds/coa-mac/coa.zip --verbose
  Mono: Failed to load project assembly
  Running: "/snap/bin/dotnet" msbuild "/home/runner/work/coa/coa/coa.sln" /restore /t:Build "/p:Configuration=ExportRelease" /v:normal "/l:GodotTools.BuildLogger.GodotBuildLogger,/home/runner/.local/share/godot/godot_executable/Godot_v3.3-rc9_mono_linux_headless_64/GodotSharp/Tools/GodotTools.BuildLogger.dll;/home/runner/.local/share/godot/mono/build_logs/b400f1ea4543f8d078e2dff5d872887b_ExportRelease" /p:GodotTargetPlatform=osx

...

  /home/runner/.local/share/godot/godot_executable/Godot_v3.3-rc9_mono_linux_headless_64/godot /home/runner/work/coa/coa/project.godot --export coa-linux /home/runner/.local/share/godot/builds/coa-linux/coa.x86_64 --verbose
  Mono: Failed to load project assembly
  Running: "/snap/bin/dotnet" msbuild "/home/runner/work/coa/coa/coa.sln" /restore /t:Build "/p:Configuration=ExportRelease" /v:normal "/l:GodotTools.BuildLogger.GodotBuildLogger,/home/runner/.local/share/godot/godot_executable/Godot_v3.3-rc9_mono_linux_headless_64/GodotSharp/Tools/GodotTools.BuildLogger.dll;/home/runner/.local/share/godot/mono/build_logs/b400f1ea4543f8d078e2dff5d872887b_ExportRelease" /p:GodotTargetPlatform=x11
  Godot Engine v3.3.rc9.mono.official - https://godotengine.org

...

I'm failing to understand the errors:

  Mono: Failed to load Core API assembly
  Mono: Failed to load project assembly
knightofiam commented 3 years ago

Using your action without pre-installing a dotnet-sdk:

  Running: "/usr/bin/msbuild"  "/home/runner/work/coa/coa/coa.sln" /restore /t:Build "/p:Configuration=ExportRelease" /v:normal "/l:GodotTools.BuildLogger.GodotBuildLogger,/home/runner/.local/share/godot/godot_executable/Godot_v3.3-rc9_mono_linux_headless_64/GodotSharp/Tools/GodotTools.BuildLogger.dll;/home/runner/.local/share/godot/mono/build_logs/b400f1ea4543f8d078e2dff5d872887b_ExportRelease" /p:GodotTargetPlatform=windows
  ERROR: godot_icall_GD_pusherror: Failed to export project: Failed to build project
     At: modules/mono/glue/gd_glue.cpp:250.
  System.Exception: Failed to build project
    at GodotTools.Export.ExportPlugin._ExportBeginImpl (System.String[] features, System.Boolean isDebug, System.String path, System.Int32 flags) [0x0007e] in <c8b5a5308c6141d8acc11a71c13bb4ae>:0 
    at GodotTools.Export.ExportPlugin._ExportBegin (System.String[] features, System.Boolean isDebug, System.String path, System.Int32 flags) [0x0000b] in <c8b5a5308c6141d8acc11a71c13bb4ae>:0

Which is a much worse situation than above. And it's using msbuild directly, with no dotnet preface. So you can see, pre-installing a dotnet-sdk is required to get this setup working. The other errors don't make sense but at least they don't prevent the proper exporting and functioning of the native executables.

Update: Even with these errors, the exported executable (for mac) amazingly actually runs without crashing, but not properly. Keyboard controls don't work. Character is frozen in midair (physics engine not working correctly).

knightofiam commented 3 years ago

Got yet another piece of the puzzle. Pre-installing a dotnet-sdk alone isn't enough to avoid the previous comment's errors, that result in malfunctioning executables. Although I thought the custom editor settings are being overwritten by Godot in #52, it appears the overwrite happens after the custom editor settings are used for the export. The reason being is that changing this one line in editor-settings_3.tres in conjunction with a pre-installed dotnet-sdk allows the build to pass.

Working editor-settings_3.tres:

[gd_resource type="EditorSettings" format=2]

[resource]
export/android/adb = "/usr/bin/adb"
export/android/jarsigner = "/usr/bin/jarsigner"
export/android/custom_build_sdk_path = "/usr/local/lib/android/sdk"
mono/builds/build_tool = 3

Relevant line:

mono/builds/build_tool = 3

Changing this value from 0 to 3 causes Godot to run dotnet msbuild instead of msbuild. That's the key difference. As long as the dotnet command is made available beforehand, this causes the build to pass.

knightofiam commented 3 years ago

45 CI builds later. And 100's of local Ubuntu VM builds. Here's my final working setup:

env:

Godot Mono 3.3 RC 9
.NET 5 SDK
C# 9
Ubuntu 20.04

csproj:

<Project Sdk="Godot.NET.Sdk/3.3.0">
    <PropertyGroup>
        <TargetFramework>net472</TargetFramework>
        <LangVersion>9.0</LangVersion>
    </PropertyGroup>
</Project>

editor-settings_3.tres:

[gd_resource type="EditorSettings" format=2]

[resource]
mono/builds/build_tool = 3

export.yml:

name: export

on:
  push:
    branches:
      - master

env:
  LANG: "en_US.UTF-8"

jobs:
  export:
    name: export executables
    runs-on: ubuntu-latest
    steps:
      - name: set locale
        run: |
          sudo locale-gen en_US.UTF-8
          sudo update-locale LANG=en_US.UTF-8
      - name: checkout code
        uses: actions/checkout@v2.3.4
        with:
          fetch-depth: 0
      - name: install .net sdk
        uses: actions/setup-dotnet@v1
        with:
          dotnet-version: '5.0.x'
      - name: use custom editor settings
        run: |
          mkdir -p ~/.config/godot
          cp ~/work/coa/coa/editor_settings-3.tres ~/.config/godot/
          cat ~/.config/godot/editor_settings-3.tres
      - name: create executables
        uses: firebelley/godot-export@v2.6.1
        env:
          GITHUB_TOKEN: ${{ secrets.GH_ACTIONS_TOKEN }}
          GODOT_MONO_FILE_VERSION: 3.3-rc9
          GODOT_MONO_URL_VERSION: 3.3/rc9
        with:
          godot_executable_download_url: https://downloads.tuxfamily.org/godotengine/${{env.GODOT_MONO_URL_VERSION}}/mono/Godot_v${{env.GODOT_MONO_FILE_VERSION}}_mono_linux_headless_64.zip
          godot_export_templates_download_url: https://downloads.tuxfamily.org/godotengine/${{env.GODOT_MONO_URL_VERSION}}/mono/Godot_v${{env.GODOT_MONO_FILE_VERSION}}_mono_export_templates.tpz
          relative_project_path: ./
          archive_export_output: true
          create_release: false
          base_version: 0.0.1
      - name: upload executables
        uses: actions/upload-artifact@v2
        with:
          name: executables
          path: ~/.local/share/godot/dist/*.zip
firebelley commented 3 years ago

Awesome, nice work! This helps a lot. I think it makes sense to document this, specifically for the Godot Mono case. Though it is still strange that installing the .NET SDK is required, since it should already be available to the runner environment: https://github.com/actions/virtual-environments/blob/main/images/linux/Ubuntu2004-README.md#net-core-sdk

Do you think it makes sense to set the mono/builds/build_tool to 3 in the editor settings provided by default? If that's what works with the latest Ubuntu runner, then I think it makes sense to change that and bump this action's version.

knightofiam commented 3 years ago

Here's a bash script that does the same thing. Just customize game, repo, user, & branch at the top:

#!/usr/bin/env bash

game="my-game-name"
repo="my-github-repo-name"
user="my-github-username"
branch="my-github-branch-name"

sudo snap install dotnet-sdk --classic

mkdir -p ${HOME}/work/${repo}
cd ${HOME}/work/${repo}
git clone https://github.com/${user}/${repo}.git
cd ${repo}
git switch ${branch}

mkdir -p ${HOME}/.config/godot
mkdir -p ${HOME}/.local/share/godot/builds/${game}-mac

wget --no-verbose https://downloads.tuxfamily.org/godotengine/3.3/rc9/mono/Godot_v3.3-rc9_mono_export_templates.tpz -O ${HOME}/.local/share/godot/godot_templates.tpz
wget --no-verbose https://downloads.tuxfamily.org/godotengine/3.3/rc9/mono/Godot_v3.3-rc9_mono_linux_headless_64.zip -O ${HOME}/.local/share/godot/godot.zip

7z x ${HOME}/.local/share/godot/godot.zip -o${HOME}/.local/share/godot/godot_executable -y > nul
mv ${HOME}/.local/share/godot/godot_executable/Godot_v3.3-rc9_mono_linux_headless_64/Godot_v3.3-rc9_mono_linux_headless.64 ${HOME}/.local/share/godot/godot_executable/Godot_v3.3-rc9_mono_linux_headless_64/godot
chmod +x ${HOME}/.local/share/godot/godot_executable/Godot_v3.3-rc9_mono_linux_headless_64/godot
unzip -q ${HOME}/.local/share/godot/godot_templates.tpz -d ${HOME}/.local/share/godot
mv ${HOME}/.local/share/godot/templates ${HOME}/.local/share/godot/tmp
mkdir -p ${HOME}/.local/share/godot/templates/
mv ${HOME}/.local/share/godot/tmp ${HOME}/.local/share/godot/templates/3.3.rc9.mono

${HOME}/.local/share/godot/godot_executable/Godot_v3.3-rc9_mono_linux_headless_64/godot ${HOME}/work/${game}/${game}/project.godot --export ${game}-windows ${HOME}/.local/share/godot/builds/${game}-windows/${game}.exe --verobse
${HOME}/.local/share/godot/godot_executable/Godot_v3.3-rc9_mono_linux_headless_64/godot ${HOME}/work/${game}/${game}/project.godot --export ${game}-mac ${HOME}/.local/share/godot/builds/${game}-mac/${game}-mac.zip --verbose
${HOME}/.local/share/godot/godot_executable/Godot_v3.3-rc9_mono_linux_headless_64/godot ${HOME}/work/${game}/${game}/project.godot --export ${game}-linux ${HOME}/.local/share/godot/builds/${game}-linux/${game}.x86_64 --verbose

7z a ${HOME}/.local/share/godot/dist/${game}-windows.zip ${HOME}/.local/share/godot/builds/${game}-windows/* -y > nul
cp ${HOME}/.local/share/godot/builds/${game}-mac/${game}-mac.zip ${HOME}/.local/share/godot/dist/
7z a ${HOME}/.local/share/godot/dist/${game}-linux.zip ${HOME}/.local/share/godot/builds/${game}-linux/* -y > nul
knightofiam commented 3 years ago

Though it is still strange that installing the .NET SDK is required, since it should already be available to the runner environment: https://github.com/actions/virtual-environments/blob/main/images/linux/Ubuntu2004-README.md#net-core-sdk

@firebelley Perhaps the command dotnet is not made available, even if installed? Or it's the wrong version? I'm not sure.

Do you think it makes sense to set the mono/builds/build_tool to 3 in the editor settings provided by default? If that's what works with the latest Ubuntu runner, then I think it makes sense to change that and bump this action's version.

I think it makes sense for Godot Mono, as it uses the command dotnet msbuild. I can't speak for other configurations, unfortunately. I think this number is determined by the Godot Editor > Editor Settings > Mono > Builds > Build Tool. For Mono builds with C# 9 & .NET 5 SDK, mine is set to dotnet CLI, which appears to be 3 in the editor settings file.

firebelley commented 3 years ago

I was just doing some testing and it seems to me that removing the mono/builds/build_tool line works as well. Could you please try one more workflow run, but this time use firebelley/godot-export@dev? I want to verify that it works for you as well, if so I think I can confidently create a new release and some additional documentation.

firebelley commented 3 years ago

I should add that I didn't add any extra .NET SDK setup steps or anything, so I'd be curious to see if excluding the .NET setup step works for you as well.

Here is my csproj:

<Project Sdk="Godot.NET.Sdk/3.3.0">
  <PropertyGroup>
    <TargetFramework>net472</TargetFramework>
    <LangVersion>9.0</LangVersion>
  </PropertyGroup>
  <ItemGroup>
    <Reference Include="System" />
  </ItemGroup>
  <ItemGroup />
</Project>

And this is using the latest Godot Mono RC 9

knightofiam commented 3 years ago

@firebelley Awesome, thank you! Yes it's working without installing a dotnet-sdk, without custom editor settings, on your dev branch, on both mac and linux. I don't have a windows setup to test, unfortunately.

csproj:

<Project Sdk="Godot.NET.Sdk/3.3.0">
    <PropertyGroup>
        <TargetFramework>net472</TargetFramework>
        <LangVersion>9.0</LangVersion>
    </PropertyGroup>
</Project>
firebelley commented 3 years ago

Great! Windows builds are working for me, so I'm going to call this good and release a new version shortly.

firebelley commented 3 years ago

Here is the PR https://github.com/firebelley/godot-export/pull/54

Please let me know if you'd like to see anything else added before I create the new version!

knightofiam commented 3 years ago

@firebelley Thanks! The only other thing I was thinking of - not that it has to be in this release - is being able to change the log level. The Godot executable itself has --quiet & --verbose (your default), for 7z I've been appending > nul as the StackOverflow recommendation, since there is no verbosity-related flag. Also for wget, I think --no-verbose is the default being used here. Between those three commands (not sure if there's others) we can control the verbosity quite well.

However, the main reason I personally want this is to completely suppress spurious errors from the Godot executable because I don't want to see it complain about missing the .import folder (it auto-generates it from the .import files as it's complaining, lol), and the rest of them are from Mono I believe. If there were a real error, the build would fail regardless, so I could always go back if I wanted to and re-enable / un-suppress errors in that case with the convenience of a yml flag. The --quiet flag for Godot headless doesn't suppress errors, so it would need to be something like &>/dev/null appended. Let me know what you think. I don't know if this is a quick fix or if I should create a new issue and have you create a release without it.

knightofiam commented 3 years ago

These are the errors directly related to this specific issue that, while looking important, don't actually affect the functionality of the final executable in any way that I've been able to detect. Which I think is understandable that I would want to suppress them until Godot fixes the situation.

Mono: Failed to load Core API assembly
Mono: Failed to load project assembly

ERROR: godot_icall_GD_pusherror: Failed to determine namespace and class for script: res://StateMachine.cs. Parse error: Unexpected token: <
     At: modules/mono/glue/gd_glue.cpp:250.

The third error is a known bug due to using a generically-typed class. I still have no idea what the first two are about.

knightofiam commented 3 years ago

Just for the sake of completeness, the .import errors such as:

ERROR: get_dependencies: Cannot open file 'res://.import/blahblahblahblah.sample At: core/io/resource_format_binary.cpp:1048

are related to this Godot feature request, and also do not affect the build / export process as Godot headless auto-generates the .import folder when run.

firebelley commented 3 years ago

Regarding log verbosity levels, please open a new issue with that feature request so we can track it there. I think that makes sense to do, but I'm going to have to think more about how to accomplish it effectively. If you have ideas on what log level control would look like in this action, please feel free to make suggestions! I'm thinking maybe an optional input to allow a comma-separated list of arguments for the various commands makes sense, but I don't know how awkward (if at all) that would be. Also, it might make sense to tone down some of the verbosity by default. Again, please feel free to post your ideas into another issue so we can track it easier.

firebelley commented 3 years ago

I have just published v2.7.0 https://github.com/firebelley/godot-export/releases/tag/v2.7.0 which addresses the original issue. Please feel free to reopen this if you encounter this issue again with v2.7.0