Reuse of existing tests from `code.forgejo.org`

[Crown0815]

Is there anything that could easily be scavenged from code.forgejo.org regarding tests? I would like to run the same tests as are run on code.forgejo.org, also against the builds of forgejo from there, but with the windows binary of the runner. Or are the tests relying on Linux-specific dependencies (e.g. bash, etc.)?

[earl-warren]

You will want to run https://code.forgejo.org/forgejo/end-to-end, subdirectory actions. These are unlikely to run out of the box on anything but Debian or Ubuntu.

If you can manage to also run a windows machine and have them communicate, you could run launch the runner from there.

I don't know that it is possible though. It definitely is not possible on Forgejo Actions but GitHub Actions is different and may offer that possibility.

Another option would be to run a virtual machine to launch the runner (kvm / qemu). I know it works for testing different architectures but have no notion if it is possible to also launch a windows machine. If that works, the runner is isolated in this virtual machine and can connect to the Forgejo instance running on the host from which the VM is launched.

[Crown0815]

I took a look at the https://code.forgejo.org/forgejo/end-to-end repository.

From what I gathered the relevant behaviors are:

1. Build forgejo and upload artifact as forgejo-dev (build step).
2. Run prepare-end-to-end action to run the previously created forgejo-dev (which in turn uses the setup-forgejo action)
3. Execute the tests

The main challenge ist in step 2.

Does anybody know what the S3 setup in prepare-end-to-end is used for?

[earl-warren]

Does anybody know what the S3 setup in prepare-end-to-end is used for?

To verify that objects stored in S3 are still accessible / where they are supposed to on each supported version. And also in upgrade tests to verify that they are not misplaced.

https://code.forgejo.org/forgejo/end-to-end/src/commit/49e99f95d4bb98363a618423933b90fcfc979f72/storage/storage.sh#L30-L45

function test_storage_stable_s3() {
    local work_path=$DIR/forgejo-work-path
    local s3_backend=${1:-minio}

    for version in $RELEASE_NUMBERS; do
        log_info "Forgejo $version & $s3_backend"
        stop
        storage_reset stable-s3
        start $version $s3_backend
        fixture_create
        for fun in ${STORAGE_FUN}; do
            fixture_${fun}_assert_s3
        done
    done
}

[Crown0815]

I see. But the S3 setup is not relevant for the actions tests, correct?

[earl-warren]

The S3 setup is not relevant for the actions tests, you are correct.

[Crown0815]

Potential solution to running windows on Linux: https://github.com/dockur/windows

[Crown0815]

Alternative, virtual machines on macOS runner: https://github.com/marketplace/actions/vagrant-virtualbox