Write tests

[luttje]

Take the time to write unit tests.

Consider using the Gmod.NET Tests Project as an example.

The Gmod.NET workflow is also informative. Apparently github can even download Garry's mod and run the tests:

name: CI

on: [push, pull_request]

env:
  GMODNET_VERSION: '0.6.0'
  DOTNET_SDK_VERSION: '6.0.100-preview.5.21302.13'

jobs:
 linux-build:

   runs-on: ubuntu-20.04

   steps:
    - name: Checkout
      uses: actions/checkout@v2.0.0

    - name: Setup .NET Core SDK
      uses: actions/setup-dotnet@v1.7.2
      with:
        dotnet-version: ${{ env.DOTNET_SDK_VERSION }}

    - name: Build Gmod.NET
      run: dotnet build runtime.csproj -c Release

    - name: Upload build artifacts
      uses: actions/upload-artifact@v1.0.0
      with:
        name: Linux-Raw-Build
        path: ./build

    - name: Upload nupkgs
      uses: actions/upload-artifact@v1.0.0
      with:
        name: nupkgs
        path: nupkgs

    - name: Download Steam and Install Garry's Mod Dedicated Server
      run: |
           wget https://steamcdn-a.akamaihd.net/client/installer/steamcmd_linux.tar.gz
           tar -xvzf steamcmd_linux.tar.gz
           rm -rfv steamcmd_linux.tar.gz
           ./steamcmd.sh +login anonymous +force_install_dir gmod "+app_update 4020 -beta x86-64 validate" +quit

    - name: Prepare test run
      run: |
           dotnet publish gm_dotnet_managed/Tests/Tests.csproj -c Release -o gmod/garrysmod/lua/bin/Modules/Tests
           cp -a build/. gmod/garrysmod/lua/bin
           cp lua/Tests/run_tests.lua gmod/garrysmod/lua/autorun

    - name: Run Garry's Mod
      run: ./srcds_run_x64 -game garrysmod -systemtest -condebug +sv_hibernate_think 1 || true
      working-directory: ./gmod/
      timeout-minutes: 1
      continue-on-error: true
      env:
        COREHOST_TRACE: 1
        COREHOST_TRACE_VERBOSITY: 4
        COREHOST_TRACEFILE: corehost_trace.txt

    - name: Print Corehost trace
      working-directory: ./gmod/
      run: cat corehost_trace.txt

    - name: Print console log
      run: cat gmod/garrysmod/console.log

    - name: Print test log
      run: cat gmod/tests-log.txt

    - name: Check if tests were successful
      run: mv gmod/tests-success.txt gmod/tests.txt

    - name: Upload test log
      uses: actions/upload-artifact@v1.0.0
      with:
        name: Linux-Test-Log
        path: gmod/tests-log.txt

# Finilize nightly build for push event

    - name: Download GMS
      if: github.event_name == 'push'
      run: |
           wget https://github.com/GlebChili/GmodNetModuleSigner/releases/download/1.0.0/gms
           chmod +x gms

    - name: Download signature key
      if: github.event_name == 'push'
      env:
        GITHUB_TOKEN: ${{ secrets.GMOD_NET_PRIVATE_KEY_REPO_TOKEN }}
      run: git clone https://GlebChili:$GITHUB_TOKEN@github.com/GlebChili/GmodDotNetPrivateKey.git

    - name: Sign assemblies
      if: github.event_name == 'push'
      run: |
           cd build
           ./../gms --sign=gmcl_dotnet_linux64.dll --key=../GmodDotNetPrivateKey/gmodnet-private.modulekey --version=$GMODNET_VERSION
           mv signature.modulesign gmcl_dotnet.modulesign
           cp ../GmodDotNetPrivateKey/gmodnet-public.modulekey .
           mv gmodnet-public.modulekey gmcl_dotnet.modulekey
           cd gmodnet
           ./../../gms --sign=GmodNET.dll --key=../../GmodDotNetPrivateKey/gmodnet-private.modulekey --version=$GMODNET_VERSION
           mv signature.modulesign GmodNET.modulesign
           cp ../../GmodDotNetPrivateKey/gmodnet-public.modulekey .
           mv gmodnet-public.modulekey GmodNET.modulekey

    - name: Pack build
      if: github.event_name == 'push'
      env:
        TRAVIS_COMMIT: ${{ github.sha }}
      run: |
           cd build
           tar czfv ../linux-$TRAVIS_COMMIT.tar.gz .

    - name: Pack with MSBuild
      if: github.event_name == 'push'
      run: dotnet build runtime.csproj -target:PackRuntimeBuild

    - name: Upload versioned build as artifact
      if: github.event_name == 'push'
      uses: actions/upload-artifact@v1.0.0
      with:
        name: linux-versioned-build
        path: packed-build

 windows-build:

   runs-on: windows-latest

   steps:
     - name: Checkout
       uses: actions/checkout@v2.0.0

     - name: Setup .NET Core SDK
       uses: actions/setup-dotnet@v1.7.2
       with:
         dotnet-version: ${{ env.DOTNET_SDK_VERSION }}

     - name: Build Gmod.NET
       run: dotnet build runtime.csproj -c Release

     - name: Upload build artifacts
       uses: actions/upload-artifact@v1.0.0
       with:
         name: Windows-Raw-Build
         path: ./build

     - name: Download Steam and Install Garry's Mod Dedicated Server
       shell: bash
       run: |
            curl https://steamcdn-a.akamaihd.net/client/installer/steamcmd.zip -O -L
            powershell -Command 'Expand-Archive -LiteralPath ./steamcmd.zip -DestinationPath ./'
            ./steamcmd.exe +login anonymous +force_install_dir gmod "+app_update 4020 -beta x86-64 validate" +quit || true

     - name: Prepare test run
       shell: bash
       run: |
            dotnet publish gm_dotnet_managed/Tests/Tests.csproj -c Release -o gmod/garrysmod/lua/bin/Modules/Tests
            cp -a build/. gmod/garrysmod/lua/bin
            cp lua/Tests/run_tests.lua gmod/garrysmod/lua/autorun

     - name: Run Garry's Mod
       shell: bash
       run: |
            powershell -Command './gmod/srcds_win64.exe -console -systemtest -condebug -game "garrysmod" +exec "server.cfg" +gamemode sandbox +map gm_construct +maxplayers 16 +sv_hibernate_think 1'
            powershell -Command 'Wait-Process -Name srcds_win64'
       timeout-minutes: 5
       continue-on-error: true

     - name: Print console log
       run: cat gmod/garrysmod/console.log

     - name: Print test log
       shell: bash
       run: cat gmod/tests-log.txt

     - name: Check if tests were successful
       shell: bash
       run: mv gmod/tests-success.txt gmod/tests.txt

     - name: Upload test log
       uses: actions/upload-artifact@v1.0.0
       with:
         name: Windows-Test-Log
         path: gmod/tests-log.txt

    # Finilize full signed build on push

     - name: Download GMS
       if: github.event_name == 'push'
       shell: bash
       run: curl -O -L https://github.com/GlebChili/GmodNetModuleSigner/releases/download/1.0.0/gms.exe

     - name: Download signature key
       if: github.event_name == 'push'
       shell: bash
       env:
         GITHUB_TOKEN: ${{ secrets.GMOD_NET_PRIVATE_KEY_REPO_TOKEN }}
       run: git clone https://GlebChili:$GITHUB_TOKEN@github.com/GlebChili/GmodDotNetPrivateKey.git

     - name: Sign assemblies
       if: github.event_name == 'push'
       shell: bash
       run: |
            cd build
            ./../gms.exe --sign=gmcl_dotnet_win64.dll --key=../GmodDotNetPrivateKey/gmodnet-private.modulekey --version=$GMODNET_VERSION
            mv signature.modulesign gmcl_dotnet.modulesign
            cp ../GmodDotNetPrivateKey/gmodnet-public.modulekey .
            mv gmodnet-public.modulekey gmcl_dotnet.modulekey
            cd gmodnet
            ./../../gms.exe --sign=GmodNET.dll --key=../../GmodDotNetPrivateKey/gmodnet-private.modulekey --version=$GMODNET_VERSION
            mv signature.modulesign GmodNET.modulesign
            cp ../../GmodDotNetPrivateKey/gmodnet-public.modulekey .
            mv gmodnet-public.modulekey GmodNET.modulekey

     - name: Pack with MSBuild
       if: github.event_name == 'push'
       run: dotnet build runtime.csproj -target:PackRuntimeBuild

     - name: Upload versioned build as artifact
       if: github.event_name == 'push'
       uses: actions/upload-artifact@v1.0.0
       with:
         name: windows-versioned-build
         path: packed-build

 mac-build:

    runs-on: macos-latest

    steps:

      - name: Checkout
        uses: actions/checkout@v2.0.0

      - name: Setup .NET Core SDK
        uses: actions/setup-dotnet@v1.7.2
        with:
          dotnet-version: ${{ env.DOTNET_SDK_VERSION }}

      - name: Build Gmod.NET
        run: dotnet build runtime.csproj -c Release

      - name: Upload build artifacts
        uses: actions/upload-artifact@v1.0.0
        with:
          name: Osx-Raw-Build
          path: ./build

# Finilize full signed build on push

      - name: Download GMS
        if: github.event_name == 'push'
        run: |
             curl -O -L https://github.com/GlebChili/GmodNetModuleSigner/releases/download/1.0.0/gms-osx
             chmod +x gms-osx

      - name: Download signature key
        if: github.event_name == 'push'
        env:
          GITHUB_TOKEN: ${{ secrets.GMOD_NET_PRIVATE_KEY_REPO_TOKEN }}
        run: git clone https://GlebChili:$GITHUB_TOKEN@github.com/GlebChili/GmodDotNetPrivateKey.git

      - name: Sign assemblies
        if: github.event_name == 'push'
        run: |
             cd build
             ./../gms-osx --sign=gmcl_dotnet_osx64.dll --key=../GmodDotNetPrivateKey/gmodnet-private.modulekey --version=$GMODNET_VERSION
             mv signature.modulesign gmcl_dotnet.modulesign
             cp ../GmodDotNetPrivateKey/gmodnet-public.modulekey .
             mv gmodnet-public.modulekey gmcl_dotnet.modulekey
             cd gmodnet
             ./../../gms-osx --sign=GmodNET.dll --key=../../GmodDotNetPrivateKey/gmodnet-private.modulekey --version=$GMODNET_VERSION
             mv signature.modulesign GmodNET.modulesign
             cp ../../GmodDotNetPrivateKey/gmodnet-public.modulekey .
             mv gmodnet-public.modulekey GmodNET.modulekey

      - name: Pack with MSBuild
        if: github.event_name == 'push'
        run: dotnet build runtime.csproj -target:PackRuntimeBuild

      - name: Upload versioned build as artifact
        if: github.event_name == 'push'
        uses: actions/upload-artifact@v1.0.0
        with:
          name: osx-versioned-build
          path: packed-build

 deploy-and-notify:

    runs-on: ubuntu-18.04

    needs: [linux-build, windows-build, mac-build]

    if: github.event_name == 'push'

    steps:

      - name: Checkout repository
        uses: actions/checkout@v2.0.0

      - name: Setup .NET Core SDK
        uses: actions/setup-dotnet@v1.7.2
        with:
          dotnet-version: ${{ env.DOTNET_SDK_VERSION }}

      - name: Prepare folders
        run: |
             mkdir publish
             mkdir versioned-publish
             mkdir nuget-publish
             cd publish
             mkdir windows
             mkdir osx

      - name: Download versioned build (windows)
        uses: actions/download-artifact@v1.0.0
        with:
          name: windows-versioned-build
          path: versioned-publish

      - name: Download versioned build (linux)
        uses: actions/download-artifact@v1.0.0
        with:
          name: linux-versioned-build
          path: versioned-publish

      - name: Download versioned build (osx)
        uses: actions/download-artifact@v1.0.0
        with:
          name: osx-versioned-build
          path: versioned-publish

      - name: Download nuget packages from previous stage
        uses: actions/download-artifact@v1.0.0
        with:
          name: nupkgs
          path: nuget-publish

      - name: Upload versioned builds to storage
        uses: jakejarvis/s3-sync-action@v0.5.1
        with:
          args: --acl public-read --follow-symlinks
        env:
          AWS_S3_BUCKET: 'gleb-krasilich'
          AWS_ACCESS_KEY_ID: ${{ secrets.DOS_STORAGE_KEY }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.DOS_STORAGE_SECRETE }}
          AWS_REGION: 'us-west-1'
          AWS_S3_ENDPOINT: 'https://fra1.digitaloceanspaces.com'
          SOURCE_DIR: 'versioned-publish'
          DEST_DIR: 'GmodNETStorage/storage'

      - name: Upload API nuget package
        run: |
             dotnet nuget update source gmodnet-packages --username CI --password ${{ secrets.DEVOPS_PASSWORD }} --store-password-in-clear-text
             dotnet nuget push nuget-publish/**.nupkg --source gmodnet-packages --api-key az --skip-duplicate

      - name: Restore local dotnet tools
        run: |
             cd csx
             dotnet tool restore

      - name: Send Discord message
        env:
          DOTNET_ROLL_FORWARD: Major
        run: |
             cd csx
             dotnet script discord-webhook.csx ${{ secrets.DISCORD_WEBHOOK }} ${{ github.sha }}

[luttje]

I imagine writing tests in annotations and creating a Build Script which will generate a Lua test file for me. That way I can easily write new tests as I think them up, near where I'm already writing code anyway:

public class MongoClient : LuaMetaObjectBinding, IDisposable
{
    // ...

#region Tests
    [LuaTest("client = MongoClient('mongodb://etc...')")]
    [LuaTest("assert(type(client) == 'MongoClient', 'Constructed object is not of the correct type!')")]
    [LuaTest("assert(etc..., 'Error!'")]
    [LuaTest("assert(etc..., 'Error!'")]
    [LuaTest("assert(etc..., 'Error!'")]
    [LuaTest("assert(etc..., 'Error!'")]
    [LuaTest("assert(etc..., 'Error!'")]
#endregion
    [LuaStatic(IsInitializer = true)]
    public static MongoClient Constructor(ILua lua, string connectionString)
    {
      // ...
    }

#region Tests
    [LuaTest("client.DropDatabase('testdb')")]
    [LuaTest("assert(client.GetDatabaseNames().IndexOf('testdb') == -1, 'Database was not removed!'")]
    [LuaTest("assert(etc..., 'Error!'")]
    [LuaTest("assert(etc..., 'Error!'")]
    [LuaTest("assert(etc..., 'Error!'")]
    [LuaTest("assert(etc..., 'Error!'")]
    [LuaTest("assert(etc..., 'Error!'")]
#endregion
    [LuaMethod]
    public void DropDatabase(string name)
        => client.DropDatabase(name);

I'll sleep on it.