Open Potherca opened 2 years ago
For webid-provider-tests:v2.0.3 I see 36/36 passed.
For solid-crud-tests:v5.1.0 I see: Tests: 4 failed, 2 skipped, 63 passed, 69 total The 4 failed tests are related to concurrency.
For web-access-control-tests:v6.0.0 I see: Tests: 1 failed, 2 skipped, 88 passed, 91 total And that is the one that was changed in WAC tests v6.0;0
I'll create a PR to fix the last few failing tests!
Let's see which tests pass in https://github.com/pdsinterop/php-solid-server/pull/53
What you described with Please make sure the cookie is valid
being printed 83 times should be at least improved with that PR as well; if it was unable to get the cookie then it will detect that and exit early.
In that case (here triggered artificially by stopping the server), this will be the output:
[...]
Status: Image is up to date for solidtestsuite/solid-crud-tests:v5.1.0
docker.io/solidtestsuite/solid-crud-tests:v5.1.0
v6.0.0: Pulling from solidtestsuite/web-access-control-tests
Digest: sha256:787d0bb4eee51e9e986a8d4c6c64f0b304cab071429d0f762928f428ba12e7b2
Status: Image is up to date for solidtestsuite/web-access-control-tests:v6.0.0
docker.io/solidtestsuite/web-access-control-tests:v6.0.0
Getting cookie for Alice...
(node:8) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
(Use `node --trace-warnings ...` to show where the warning was created)
/app/node_modules/node-fetch/lib/index.js:1461
reject(new FetchError(`request to ${request.url} failed, reason: ${err.message}`, 'system', err));
^
FetchError: request to https://server/login/ failed, reason: getaddrinfo EAI_AGAIN server
at ClientRequest.<anonymous> (/app/node_modules/node-fetch/lib/index.js:1461:11)
at ClientRequest.emit (node:events:390:28)
at TLSSocket.socketErrorListener (node:_http_client:447:9)
at TLSSocket.emit (node:events:390:28)
at emitErrorNT (node:internal/streams/destroy:157:8)
at emitErrorCloseNT (node:internal/streams/destroy:122:3)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
type: 'system',
errno: 'EAI_AGAIN',
code: 'EAI_AGAIN'
}
Error obtaining cookie for Alice, stopping.
Hm, there's definitely something odd going on in the github actions - the solid crud tests pass when I run them locally with Docker, but not when they are run on GitHub Actions. Different Docker version maybe?
The GitHub Action runs on ubuntu-latest
which is 20.04. According to the environments docs the current versions are:
Docker Compose v1 1.29.2
Docker Compose v2 2.1.1+azure-1
Docker-Buildx 0.7.0
@mrvahedi68 and I are running solid-crud-tests@5.3.0 now
Result:
Test Suites: 2 failed, 5 passed, 7 total
Tests: 15 failed, 4 skipped, 42 passed, 61 total
docker run --rm --network=testnet --env COOKIE="$COOKIE" --env-file ./env-vars-for-test-image.list -it solid-crud-tests bash
root@f49e78e84faa:/app# ./node_modules/.bin/jest test/surface/conneg.test.ts
(node:117) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
(Use `node --trace-warnings ...` to show where the warning was created)
PASS test/surface/conneg.test.ts
Test Suites: 1 passed, 1 total
Tests: 2 skipped, 4 passed, 6 total
Snapshots: 0 total
Time: 6.969 s
Ran all test suites matching /test\/surface\/conneg.test.ts/i.
Currently all WAC surface tests seem to be broken:
Before a new release is tagged, this needs to be resolved.
In the logs, the following error occurs 83 times (so basically for every test):
The following occurs 94 times:
Where
undefined
is usuallysolidLogicAlice
.@michielbdejong Do you happen to know of the top of your head what would cause this? (Or how to resolve this?)
Full logs
- `test/surface/update.test.ts`The webid-provider and solid-crud tests seem stable: