More options for specific use cases and fixed an issue regarding the "archiveDirectory" input.

Minimata commented 7 months ago

TL;DR

Hi, I wanted to use your Github Action for my project but ended up having to tweak a few things to my use cases. The changes shouldn't impact current action usage but allow for a wider variety of situations (including mine), therefore I think a PR is in order to complement your Action instead of creating a similar yet different one.

Features

New CLEAN input parameter: I added a CLEAN input that translates to the -clean flag in the RunUAT command. This lets the user clean the solution therefore transforming the build into a rebuild. This helped me overcome an issue when running the action after a plugin update where the simple build option would leave some targets out-of-date which spawned some errors whereas with -clean it worked fine. Defaults to false so it doesn't impact current action usage.
New NULLRHI input parameter: I added a NULLRHI input that translates to the -nullrhi flag in the RunUAT command. The flag is needed when running the RunUAT on a device that doesn't have a video output. This is useful in my case because I'm running my jobs in containers on the cloud and on my on-prem server which both lack video output. Even when jobs run on my personal machine I like to disable video output to keep them in the background so it doesn't disrupt my work too much. Defaults to false so it doesn't impact current action usage.
Changed default value for maps: I set a default value for the maps so a user could withdraw from specifying any map and the RunUAT command would build all maps. On smaller projects, prototypes or game jams this becomes useful because there's only a few maps to build so it's easier not to have to type them (prevents typos and such). The default value for the maps input translates to basically no command flag related to maps for the RunUAT command which therefore builds everything.

Fixes

ARCHIVE_PATH may now contain spaces: Due to the way this parameter was escaped, the ARCHIVE_PATH directory could not contain spaces. I simply recycled the syntax you used to escape the RUNUAT_PATH and the UPROJECT_PATH and applied it to the ARCHIVE_PATH input parameter.

Possible improvements

Linux support: I really like the legibility of this action compared to just running the command by itself. I would love to use it for Linux as well though I would not know how to proceed as I'm quite new to the Actions framework. I would think a linux branch in the repo would work with basically the same script but running a shell instead of a powershell. Don't know how clean this looks to you though, what do you think?
PDB deletion: For some reason the RunUAT command still exports the PDB symbols files when running in Shipping mode, which take some valuable storage space in the repository. What I'm usually doing when packaging for Shipping is removing these PDB files with a one-liner command. This could be a DELETE_PDB input parameter to the action though it would mean the action does more than just running the RunUAT command, which is why I didn't include it in my PR since this is quite an important change from your original Action.
Folder compression: In the same category as the PDB deletion, I usually follow up my archive creation with a folder compression command to create a ZIP file or a tarball of said archive. This could be a ZIP_ARCHIVE input or something, but again didn't want to change your action too much without your input first.

Misc

Updated README: Of course I updated the README with the new parameters that I'm suggesting so they're correctly documented.

Conclusion

Thank you for creating this little action which is a smooth entry into the world of Github Actions and CI/CD for Unreal Engine. I hope you'll accept the request so that we may continue to use a single action for this purpose instead of duplicating functionality in the Action marketplace. I look forward to getting your advice on the possible improvements listed above.

Cheers,

edwardteach42 commented 7 months ago

Thank you for this pull request. I will review it and merge in this weekend. On the note of improvements:

In regards to the Linux support, We are using the Cross Compile Toolchain via windows for Linux compiling as only our dedicated servers are Linux based. If there is a big need for this to function on linux, we could always create a new branch as you suggested.
I noticed this with the PDB files. I will create a bug report and look into addressing this.
For compression, we felt that it would be best utilized as a separate step in the main action. we currently are doing the following. We create a .unreal file in the root of the actions runner that is read in the Set Unreal Environment Variables step that contains:

ARCHIVE_PATH=\\ORCHIDNAS\OrchidIsleGames\
RUNUAT_PATH54=C:\UnrealEngine\Engine\Build\BatchFiles\RunUAT.bat
SEVEN_Z_PATH=C:\Program Files\7-Zip\7z.exe

Here is an example of our full build process for a Win64 build:

name: Build Win64 Client

on:
  release:
    types: [created]
  push:
    branches:
      - dev

jobs:
  build:
    runs-on: [UE54]
    environment: ${{ 
        (github.ref == 'refs/heads/dev' && 'Development') || 
        (github.event_name == 'release' && startsWith(github.ref, 'refs/tags/') && contains(github.ref, '-alpha') && 'Staging') || 
        (github.event_name == 'release' && startsWith(github.ref, 'refs/tags/') && contains(github.ref, '-beta') && 'Staging') || 
        (github.event_name == 'release' && startsWith(github.ref, 'refs/tags/') && contains(github.ref, '-rc') && 'Staging') || 
        (github.event_name == 'release' && startsWith(github.ref, 'refs/tags/') && 'Production') || 
        'Development' 
      }}
    steps:
    - name: Checkout Repository
      uses: actions/checkout@v2
      with:
        token: ${{ secrets.ACCESS_TOKEN }}
        lfs: false
        fetch-depth: 0 # Fetches all history for all tags and branches
        submodules: recursive

    - name: Set Unreal Environment Variables
      run: |
        Get-Content "${{ github.workspace }}\..\..\..\.unreal" | ForEach-Object {
            $split = $_ -split '=', 2
            $name = $split[0]
            $value = $split[1]
            Add-Content -Path $env:GITHUB_ENV -Value "$name=$value"
        }

    - name: Get Project Version and Build Number
      uses: OrchidIsle/UE5-Semantic-Versioning@latest
      with:
        BUILD_PREFIX: ${{ github.ref_name }}
        CONFIG_DIR_PATH: ${{ github.workspace }}\MyProject\Config
        ADD_BUILD_INFO: true
        USE_RELEASE_BUILD: ${{ github.event_name == 'release' }}

    - name: Cook, Stage & Package UE Project
      uses: OrchidIsle/UE5-Build-Project@latest
      with:
        RUNUAT_PATH: ${{ env.RUNUAT_PATH54 }}
        UPROJECT_PATH: ${{ github.workspace }}\MyProject\MyProject.uproject
        BUILD_CONFIG: ${{ vars.BUILD_CONFIG }}
        PLATFORM: Win64
        COOK: true
        STAGE: true
        PACKAGE: true
        PAK: true
        SERVER: false
        EDITOR: true
        ENCRYPT_INI: true

    - name: Compress Win64 Client using 7-Zip
      run: |
        $7zPath = "${{ env.SEVEN_Z_PATH }}"
        $sourceDir = "${{ github.workspace }}\MyProject\Saved\StagedBuilds\Windows"
        $archiveFile = "MyProject-Win64-${{ env.BUILD_ID }}.zip"
        & "$7zPath" a -tzip "$archiveFile" "$sourceDir\*"

    - name: Copy to Archive
      run: |
        $sourcePath = "MyProject-Win64-${{ env.BUILD_ID }}.zip"
        $destinationPath = "${{ env.ARCHIVE_PATH }}\MyProject\Builds\Win64\MyProject-Win64-${{ env.BUILD_ID }}.zip" # Replace with your destination folder path
        $destinationDirectory = Split-Path -Path $destinationPath -Parent

        if (-not (Test-Path -Path $destinationDirectory)) {
            New-Item -ItemType Directory -Path $destinationDirectory
        }

        Copy-Item -Path $sourcePath -Destination $destinationPath

    - name: Upload Build to Amazon S3
      uses: NotCoffee418/s3-zip-upload@v1.4
      env:
        AWS_SECRET_ID: ${{ secrets.DOWNLOAD_AWS_KEY_ID }}
        AWS_SECRET_KEY: ${{ secrets.DOWNLOAD_AWS_SECRET_ACCESS_KEY }}
        BUCKET_NAME: ${{ secrets.DOWNLOAD_AWS_BUCKET }}
        AWS_REGION: us-east-1
        SOURCE_MODE: FILE
        SOURCE_PATH: ./MyProject-Win64-${{ env.BUILD_ID }}.zip
        DEST_FILE: MyProject/Win64/MyProject-Win64-${{ env.BUILD_ID }}.zip

    - name: Upload Latest Full Release
      if: github.event_name == 'release'
      uses: NotCoffee418/s3-zip-upload@v1.4
      env:
        AWS_SECRET_ID: ${{ secrets.DOWNLOAD_AWS_KEY_ID }}
        AWS_SECRET_KEY: ${{ secrets.DOWNLOAD_AWS_SECRET_ACCESS_KEY }}
        BUCKET_NAME: ${{ secrets.DOWNLOAD_AWS_BUCKET }}
        AWS_REGION: us-east-1
        SOURCE_MODE: FILE
        SOURCE_PATH: ./MyProject-Win64-${{ env.BUILD_ID }}.zip
        DEST_FILE: MyProject/MyProject-Win64-Latest.zip

Minimata commented 7 months ago

In regards to the Linux support [...]

Okay though I think we misunderstood each other here. I try to containerize my actions as much as possible but windows containers are difficult to work with so all my container work is done on Linux, like so:

jobs:
  RunTests:
    runs-on: ubuntu-latest
    container: ghcr.io/epicgames/unreal-engine:dev-slim-5.3.2
    # ...

The need for a Linux-based action is therefore not so much to compile and build for Linux specifically but to support this container-based approach. If you think a branch is fine then that's cool with me. Do you want me to provide you with the code for it? I'm not a shell expert but I could probably manage.

I noticed this with the PDB files. I will create a bug report and look into addressing this.

Great stuff, thanks!

For compression, we felt that it would be best utilized as a separate step in the main action.

That's fair enough.

We create a .unreal file in the root of the actions runner that is read in the Set Unreal Environment Variables step that contains:

So if I understand correctly this .unreal file is sort of a .env file that points to the relevant paths on your computer for the action to run is that it ? This suggests all of this only runs on your computer.

7Zip is interesting, I'm simply doing a Compress-Archive but 7zip is probably a better approach, I should try it out.

Here is an example of our full build process for a Win64 build:

That's so cool ! Helps a lot. I didn't think a build would work without LFS and wondered how that would go now that my checkouts take an hour but I'm definitely getting rid of downloading all large files now !

Your branch handling is interesting too though quite different from mine. I strive for a trunk-based approach and have not experimented with Development, Staging or Production builds.

Thanks for all this info and for merging as well !

OrchidIsle / UE5-Build-Project