fediverse-devnet / feditest-tests-fediverse

The tests for the fediverse testsuite
MIT License
5 stars 4 forks source link

Create test that fails type=template in JRD link #109

Closed jernst closed 15 hours ago

jernst commented 1 week ago

Per discussion in #105.

steve-bate commented 1 week ago

In the current test results, I see:

                {
                    "plan_test_index": 2,
                    "started": "2024-06-06T12:54:23.080073Z",
                    "ended": "2024-06-06T12:54:23.189560Z",
                    "result": {
                        "type": "HardAssertionFailure",
                        "problem_category": "hard",
                        "stacktrace": [
...
                            [
                                "tests/webfinger/server/4_2__2_perform_query.py",
                                19
                            ],
...
                        ],
                        "msg": "Link type not a valid media type: \"template\"\nAccessed URI: \"https://realize.be/.well-known/webfinger?resource=acct%3Aswentel%40realize.be\".\nExpected: None\n     but: was <Link type not a valid media type: \"template\">\n"
                    },
                    "run_steps": null
                },

This is the one I suggested should be a soft fail.

jernst commented 1 week ago

I misunderstood? We have a test that fails ... it's just that you think it should be downgraded?

If you feel strongly about it, let's do that, but it seems from the discussion that they really didn't mean it and really should fix it, there is no good reason for it. We don't really want others to start implementing the same pattern because it is sort of okay ...

steve-bate commented 1 week ago

I'm not suggesting an endorsement of the pattern (it would still be a failure), but that it doesn't represent an interop risk (which I'm currently interpreting as "soft failure").