Tests Validity

[paulushub]

How valid are the test results? The expected result for the file a-font-family-011.svg, for instance, is not valid when you read the file and manually observe the results.

<svg id="svg1" viewBox="0 0 200 200" xmlns="http://www.w3.org/2000/svg">
    <title>Bold `sans-serif`</title>

    <text id="text1" x="100" y="100" text-anchor="middle"
          font-family="sans-serif" font-weight="bold" font-size="40">Text</text>

    <!-- image frame -->
    <rect id="frame" x="1" y="1" width="198" height="198" fill="none" stroke="black"/>
</svg>

The expected result, a-font-family-011.png, does not seems to be rendered with sans-serif. Am I missing something?

[RazrFalcon]

Yes, not all tests are reproducible. Specifically fonts one. sans-serif is different on all OSes, so we cannot provide a singe referenced image.

I guess I should note this somewhere.

[paulushub]

I really appreciate the efforts into the test-suite and currently using it to enhance my implementation of SVG renderer.

Yes, not all tests are reproducible.

True, and the reason why I manually mark my test results instead of using image diff. It is time consuming but worth the effort. The results of the tests e-ellipse-002.svg and e-ellipse-003.svg, for instance, are valid under SVG 1.1, but invalid under SVG 2.0, so I add comments to such results. Firefox and Chrome are rendering the outputs defined by the SVG 2.0 in these two cases.

[RazrFalcon]

True, and the reason why I manually mark my test results instead of using image diff. It is time consuming but worth the effort.

Same: https://github.com/RazrFalcon/resvg-test-suite/tree/master/tools/vdiff

Firefox and Chrome are rendering the outputs defined by the SVG 2.0 in these two cases.

Yes, SVG 2 support is complicated. There are at least 5-10 tests that are tailored to SVG 1.1 and not 2. I'm working on it.

The problem is that I'm not sure how to clarify that some tests can produce different results depending on a supported SVG standard.

Anyway, pull requests are welcome. I plan to move current tests to SVG 2 only and start adding SVG 2 tests.

[paulushub]

SVG 2 support is complicated.

It seems they are moving away from the original targets; within and without browser to browser only.

The problem is that I'm not sure how to clarify that some tests can produce different results depending on a supported SVG standard.

Versioning is not well clarified yet, maybe the final specs will make things clearer.

Anyway, pull requests are welcome.

Looking into it.

I plan to move current tests to SVG 2 only and start adding SVG 2 tests.

Having results based on both specs could also be a good idea. It requires a bit of work. but W3C itself started a completely different test suite for SVG 2.0.

[RazrFalcon]

Versioning is not well clarified yet, maybe the final specs will make things clearer.

Yes. At least for now browsers doesn't care and use SVG 2 everywhere. You cannot force 1.1 rendering. This is what makes it complicated. Therefore I plan to drop 1.1 "support" completely. And maybe note in the test files that they may look different on older renderes.

Having results based on both specs could also be a good idea.

Too many complications and we have like 10 tests that can actually be rendered differently. Not to mention that we have to figure out what to do with deprecated features, like enable-background. Browsers are failing them, but this is technically fine.

Anyway, writing this test suite lead to a lot of uncertainties. Sadly, it's not as easy as a list of SVG-PNG pairs.

[paulushub]

@RazrFalcon How about enabling a discussion forum so that individual test cases can be examined?

Not to mention that we have to figure out what to do with deprecated features, like enable-background.

These could simply indicate SVG 1.1 mode. Stuff like SVG fonts are fine with me, even though not supported in browsers except Safari. The use of the SVG fonts made most W3C tests consistent and similar approach can be used here; restricting fonts to those found in the fonts directory, except in special cases of testing "serif", "san serif" etc. For testing purpose, even the "serif" and "san serif" used in the expected results could be defined in the documentation so that were possible, these fonts will be used.

[RazrFalcon]

How about enabling a discussion forum so that individual test cases can be examined?

What do you mean by that?

The use of the SVG fonts made most W3C tests consistent and similar approach can be used here

They are badly supported, therefore I went the local fonts route.

restricting fonts to those found in the fonts directory, except in special cases of testing "serif", "san serif" etc.

This is the expected behaviour, but most apps and libraries doesn't allow this. So you have to install those fonts and hope for the best.

For testing purpose, even the "serif" and "san serif" used in the expected results could be defined in the documentation so that were possible, these fonts will be used.

Yes, I guess updating tests desc element should be enought.

[paulushub]

What do you mean by that?

I mean discuss some tests and the expected results. These are not issues that need to be closed, and in a Discussions forum, it will be there for references. Similarly to the W3C process where the tests are initially marked as DRAFT until the results are verified.

<!-- comment out this watermark once the test is approved -->
<g id="draft-watermark">
<rect x="1" y="1" width="478" height="20" fill="red" stroke="black" stroke-width="1"/>
<text font-family="SVGFreeSansASCII,sans-serif" font-weight="bold" font-size="20" x="240"
    text-anchor="middle" y="18" stroke-width="0.5" stroke="black" fill="white">DRAFT</text>
</g>

Here there is no need for the draft-watermark, but where there is a need, we could discussion it and leave the results for future references.

[RazrFalcon]

Like this https://github.com/RazrFalcon/resvg-test-suite/discussions/31 ?

[paulushub]

Exactly, thanks.