search

w3c / publishingcg

Repository of the Publishing Community Group
https://www.w3.org/community/publishingcg/
Other
18 stars 8 forks source link

Is 0 tolerance for accessibility conformance practical? #22

Open avneeshsingh opened 2 years ago

avneeshsingh commented 2 years ago

As per the design of WCAG 2.x and EPUB Accessibility 1.x, The producer need to ensure that all Content Documents meet all the accessibility requirements for claiming conformance to EPUB Accessibility 1.x or WCAG 2.x. The publishers find it challenging to meet this requirement of nearly zero tolerance, especially when they are publishing big publications. I am not sure if we can do much about this at this stage, this issue will be addressed by WCAG 3.0. But it is worth starting a discussion. Please see the minutes of accessibility task force held on August 12, 2021 for the initial round of discussions: https://www.w3.org/publishing/groups/epub-wg/Meetings/Minutes/2021-08-12-epub-a11y#section3

gregoriopellegrino commented 2 years ago

I'll try to express some of my thoughts:

Finally, I would like to add that, as Fondazione LIA, we validate all digital publications of our member publishers and - in case of accessibility problems - we report the problem to the publisher, asking them to fix it. Until accessibility issues are resolved, we do not certify the title. This for us (and for the publisher) is a guarantee of quality.

mattgarrish commented 2 years ago

Where WCAG 3 ends up is still a bit of a mystery at this stage, but it is centred around trying to handle this issue of perfection or nothing.

Currently, it would do away with Level A conformance, as Bronze is equal to Level AA. But what it does, or is trying to figure out how to do, is weigh the impact of non-conformance against user needs. Some requirements will remain absolute -- any failures will fail the content -- but less critical requirements may pass even if there are a few errors. Publications would get scored for each method so users could also review these numbers to see where the flaws are.

But while I can sympathize with the inflexibility of dealing with WCAG 2 conformance, we really shouldn't move ourselves out ahead of the Silver group, especially when it comes to conforming with WCAG 2. If we don't follow the same conformance rules, then we can't claim that anyone is meeting WCAG.

This sounds a lot like the discussion we had when we started the 1.0 revision and were considering a new level of our own for content that met some basic requirements of WCAG but not everything. Given the sometimes legal ramifications of not adhering to WCAG, it probably remains best not muddy the waters of what publishers are achieving.

iherman commented 2 years ago

The issue was discussed in a meeting on 2021-08-26

View the transcript ### 1. Continue discussion on is 0 tolerance for accessibility conformance practical? _See github issue [#1767](https://github.com/w3c/epub-specs/issues/1767)._ **Avneesh Singh:** we had a long discussion about this issue during last call … Charles suggested to use a string "significantly" conforms to … but then we had some comments on the issue **John Foliot:** I like the idea to use "significantly conforms to" … I the VPAT you can describe it, to tell that perfect is a goal … but achieve it is complicated … in the USA an accessibility problem can goes to a legal issue **Matt Garrish:** I think it's ok for a publisher to tell it in the accessibility summary … I have concerns about writing in specs what it means to be significantly accessible **Avneesh Singh:** do you mean that you can put whatever you want in the `conformsTo` field? **Matt Garrish:** yes **John Foliot:** I'm active in the Silver WG, we are working on the score system … in `conformsTo` may we link so a self-published accessibility statement (for publishers)? **Matt Garrish:** we have `certifierReport` metadata for doing this … it's optional **John Foliot:** the `conformsTo` metadata instead is not optional, it is mandatory … what about about making only one in `conformsTo` and `certifierReport` mandatory? **Matt Garrish:** actually `conformsTo` is not mandatory **Bill Kasdorf:** to end users what is most interesting is what it doesn't conform … it would be useful to express it **Gregorio Pellegrino:** for local legislation the conformance is important … also in metadata we don't have ways for telling which accessibility issues are present in a document **George Kerscher:** I don't think we should get out from Silver > *John Foliot:* +1 to George > *John Foliot:* WCAG 3 is "far off" IMHO **Avneesh Singh:** is it a problem to have `conformsTo` linking to WCAG and then in the `accessibilitySummary` telling that there may be issues? **George Kerscher:** I think we should not worry about legal issues **John Foliot:** my background tells me that think about legal risks is important … I think that standards should not enable contractions … I think we should balance it **George Kerscher:** what does the the UX metadata guidelines tell to display when `conformsTo` is empty? **Avneesh Singh:** nothing or unknown? **George Kerscher:** so you can use `certifierReport` instead `conformsTo` **John Foliot:** some certifier require to have conformsTo metadata for their certification … so it is not "optional" for being certified … I think we need a formal reference to link to **Matt Garrish:** we can add in the specs that if partial conformance is present, then the `certifierReport` is mandatory … in the conformanceTo we may use EPUB11-WCAG21-A-PARTIAL > *John Foliot:* +1 Matt - that's what I personally would like to see: addressing a real-world issue with a "standardized" solution > *Bill Kasdorf:* +1 to Matt **John Foliot:** I would also suggest to define a "standard" way to make certifier reports … VPAT may be a solution **Matt Garrish:** sure, we can recommend it **Gregorio Pellegrino:** for me the `certifierReport` should be per publication and not for all publications **Matt Garrish:** I don't know **Bill Kasdorf:** I'm favor for this approaches, but I see many VPATs that are false … a lot of publishers are not able to read VPATs > *John Foliot:* to be clear, I am not advocating using a VPAT per-se, but rather look at VPATs as a model for conformance reporting **Bill Kasdorf:** I think that have a report per publication is useful plus a report from the publisher telling on what they're working **Wendy Reid:** I'm concerned about telling that you must have to create a report for titles … I think trade publishers would not use it, because it is not meaningful for retailers **Avneesh Singh:** why not relaxing the report, if we have `accessibilitySummary` … ? **Matt Garrish:** yes, I think there are different ways to achieve it **Avneesh Singh:** I think we had an excellent discussion, Matt can you now propose changes for the specs? **John Foliot:** I think there are differences between prose versus machine-readable metadata … so we should make the distinction clear in the specs … for example telling the conformsTo MUST be an URL **Matt Garrish:** we had this discussion for EPUB Accessibility 1.1 … we had issues for complex URLs … I think that we can find a way using `conformsTo` plus `certifierReport`
iherman commented 2 years ago

The issue was discussed in a meeting on 2021-09-09

View the transcript ### 1. Is 0 tolerance for accessibility conformance practical? (pr epub-specs#1767) _See github pull request [#1767](https://github.com/w3c/epub-specs/pull/1767)._ **Avneesh Singh:** mgarrish has done a lot of work in PRs … we had a lot of discussion on WCAG zero-tolerance. We've decided on an option to address this in an editor's meeting last week **Matt Garrish:** this is to address almost-conformance. We're not trying to create something new. Rather have publishers be able to provide a disclaimer that says they did their best, but there could still be minor imperfections … this PR adds disclaimer property. Nothing similar already exists in Schema.org … this will not be a requirement for publisher to use this. Only an option **Avneesh Singh:** backwards compatibility was important to us in picking this solution. And minimizing disruption to existing workflows > *Will Awad:* q **Will Awad:** is this disclaimer different from the a11y statement that a publisher would put on their website? **Avneesh Singh:** this would be in the epub metadata **Matt Garrish:** the reality is that things slip through when a11y conformance is a combination of machine and human checking … so publisher is saying that to the best of their ability the epub meets the standard they are claiming **Will Awad:** so will it be a failure of conformance if they leave out disclaimer? **Charles LaPierre:** so you have a conformance statement. But what we're doing in addition to a11y summary statement is to have a more formal disclaimer property that publisher can add … for Benetech certification conformance is the same, but this will be one more thing that certified publishers can put in their epubs (to mitigate liability, etc.) **John Foliot:** so the statement says that the epub is WCAG compliant, but then the disclaimer says "not really"? **George Kerscher:** when I contacted Bruce Bailey from 508 access board I asked about this … and in 508 the standard is black and white as well … he said if you look hard enough, you will be able to find some defect … but the industry has come to peace with this issue and don't have any problems with that … our disclaimer says exactly that, and no more **Charles LaPierre:** and we also have to be backwards compatibility with existing epubs, which this solution is **John Foliot:** but the statement and the disclaimer are contradictory right? So at what point are there too many non-compliances? **Matt Garrish:** not sure that there is a contradiction. But disclaimer acknowledges that we live in an imperfect world. … and it creates an avenue for reporting when incidents of non-compliance are found **John Foliot:** I understand, but, I still think that the disclaimer is contradictory. WCAG is explicit in that it requires 100% perfection. … WCAG3 will acknowledge this issue of 100% compliance or nothing … but right now, I'm concerned that in the real world a lot of organizations won't put in the work to do a11y compliance anymore **Tzviya Siegman:** my concern is that a disclaimer like this doesn't accomplish anything. If the point is to make sure that publishers are covered legally, I'm not sure this does it … i'd have to check with out legal team to see if something like this is useful … and if not, then I don't see what the purpose of this is > *John Foliot:* +1 Tzviya **Charles LaPierre:** the disclaimer adds credibility for publishers that choose to go out and get 3rd party certification **Matt Garrish:** if you don't disclaim it, you still have a false claim … if we don't have anything, are we any better off? **John Foliot:** part of the answer is how we apply the metadata. One way is to ref to a URL. And the other is using a meta-element that is a text field. … 2nd way i think is a path towards a more accurate statement … from a legal perspective VPATs have a ability to say partially supports **Tzviya Siegman:** part of my concern is that the document doesn't say how to use this … maybe if you want to say "epub is WCAG compliant except for this particular criteria in this particular circumstance" … seems like there is a lot of wiggle room here **George Kerscher:** if JF and tzviya think this disclaimer does nothing, maybe we should get rid of it and leave things they way they have been. There are a lot of documents out there that are reporting WCAG AA, companies that sell to universities that claim WCAG AA, and if you look close enough you'll find that many of those are incorrect statements … the industry has come to terms with this, and maybe its not up to us to solve it **Wendy Reid:** second tzviya that we should be clear about what is allowed … but i think this has value particularly in the retail space. When retailers start to display a11y metadata, the disclaimer is going to be important to both retailers and users. Users should be made aware of that before it gets escalated to a customer care call. … retailer gets accused to having a problem when the issue is in the epub … this is already happening, and we don't even display a11y metadata yet … this gives users as much information as possible … not just a legal issue, but empowers users to know what they are purchasing **Avneesh Singh:** one option for doing this is to include such information in the a11y summary **Gregorio Pellegrino:** we have to define what it means to strive for WCAG AA if, for example, there is something in your document that don't meet WCAG A **Matt Garrish:** i pictured this mostly being used for errors that slip though, where nobody could have caught it … once we get to partial conformance you're not complying to epub a11y spec either … so the other option is that we accept that epub just shouldn't have a `dc:conforms` statement … include details in VPAT … for content that has known and potentially serious issues with it, then let's not try to solve that from a conformance perspective > *John Foliot:* `` **John Foliot:** maybe part of the problem is that we don't want to be pointing to a specific WCAG … if we use a meta-declaration (see above), could we open the spec to point to, for example, a VPAT statement? … re. george's statement of institutions that already claim conformance, some EDUs want very richly annotated VPATs … so in some ways a short, patterned conformance statement is less useful than a detailed document **Avneesh Singh:** there are some mechanisms already built into metadata to deal with this **Matt Garrish:** doesn't seem right to say that VPAT is a conformance statement. A VPAT is a report of how you conform to something else … there's nothing something can determine from that sort of URL … you can put it in if you want. The spec can't prevent you from doing that. But not sure we want to put that in the spec. **Avneesh Singh:** `certifierReport` is the right place for that sort of thing … to summarize, we don't like the disclaimer. It may be useful, but not for the purpose for which it was created. … let's leave it for now … mgarrish are you satisfied with the `certifierReport` and other metadata that can meet this need? **Matt Garrish:** maybe we can make it clear that even if you aren't providing a conformance claim you can still provide a link to a certifierReport **Avneesh Singh:** i think it would be nice to explain that these can be used in this way **George Kerscher:** if we have no conformance statement you could still include `certifiedby` and you could clarify in your a11y summary what you conform to … when the metadata is shown according to the UX guide, for `conformsto` it would say none provided, right? **Avneesh Singh:** yes, for now, but we still have to revisit the UX guide **John Foliot:** the only other potential problem i see is WCAG3 … from the start WCAG3 is being built for various levels of conformance: bronze, silver, gold … and a scoring mechanism still being sorted out … so we're going to have to be able to declare which which level an epub is at **Avneesh Singh:** no, we've decided that we're not going with this disclaimer route, so we don't need to state closeness of conformance … and if you don't completely conform, then the epub shouldn't include a conformance statement **Charles LaPierre:** I'm not in favor of pulling the disclaimer. The disclaimer puts publishers at ease, acknowledges forgiveness for human error, provides route for reporting. I think having it is better than not. **Avneesh Singh:** let's continue discussion in issue tracker, as I don't think we can resolve on this issue today **Charles LaPierre:** I think we should vote on this today **Avneesh Singh:** per W3C practice we need consensus … at 2 months away from CR, and with some publishers missing, I'm not sure a vote is a good idea right now > *Charles LaPierre:* +1 to Matts suggestion. **Matt Garrish:** we don't need to make this either/or. Disclaimer is optional. But we can still add explanatory statement that people can leave out `conformsto` and link to VPAT. **John Foliot:** I like that route. Choice is good. **Matt Garrish:** from an editing perspective I can let you look at both and see how you feel **Avneesh Singh:** so, for people who were opposed to this, does the non-compulsory nature of the disclaimer change your mind? **Tzviya Siegman:** I think it needs to be clearer how the disclaimer is intended to be used **Matt Garrish:** sure, I can try to work on that **George Kerscher:** in my mind bottom line is that there is no catastrophic failure **John Foliot:** even though disclaimer is optional, I think we're going to see it all the time. We're back to the contradictory statements. **Avneesh Singh:** so it would serve the purpose of allowing the publisher to make a more accurate statement, rather than providing any legal certainty **Matt Garrish:** and we can clarify that putting in a disclaimer doesn't absolve you from legal liability … do we want to rename it to contact information rather than disclaimer? … I think ONIX does a similar thing **John Foliot:** and that contact info is baked into VPATs as well **Gregorio Pellegrino:** if its only simple text we could put it in the last sentence of the a11y summary **Charles LaPierre:** the whole reason for this was because publishers were worried about saying `conformsto` WCAG AA. I don't think adding a contact us address that issue. > *John Foliot:* +1 to Charles, we need both **Avneesh Singh:** it seems we don't have consensus. So let's continue discussion in issue tracker. … and let's remember that we are just 2 months away from CR, so we can't make too many changes this time around
mattgarrish commented 2 years ago

I've been trying to reason out another approach to this since we met yesterday, and am now wondering if we've overlooked the most basic means of expressing conformance to the specification -- through discovery metadata alone.

What if we formalize discovery metadata-only publications as a conformance measure, so add an additional string like:

EPUB-A11Y-11_DISCOVERABLE

This identifier would avoid having to define a negative sounding name like "partial" or "substantial" or "incomplete" or "failed". It would also add a conformsTo statement to cover:

This way we lose the current problem of how to know if a publication has been checked if a full claim can't be made, which in itself is a good thing.

We can still keep the disclaimer property if it is useful to some publishers, but what are people's opinions on including this new conformance claim?

clapierre commented 2 years ago

I know that Universities are starting to write into their procurement requirements that any digital materials purchased must be WCAG-AA compliant and be 3rd party certified. So how would having this help? Two such universities which have already done this is the University of Phenix and the University of Texas, and I fully expect other universities and other educational and government agencies will require the same moving forward.

To me it sounds like those publishers who want their educational materials to be adopted at the university level must put in that metadata of WCAG-AA conformance and certifiedBy anyways which were the publishers that are most concerned with.

I think there is no harm in adding this and if those publishers don't want to put in that conformance claim will probably not get university adoption since they won't meet the procurement requirements. Once they figure that out I bet they will start putting in that WCAG-AA conformance information since technically they have passed certification and are meeting that level of conformance with only the possibility of not 100% compliance if you really dig into it as George said, if you look hard enough you will find a problem.

If this was the case every AAA certified restaurant and hotel in this country technically shouldn't be allowed to claim that because I am sure everyone has had problems with their food or hotel stay at some point. Also every website that says they are WCAG-AA compliant are also probably in violation of some criteria if you search hard enough. So why do we care about this?

mattgarrish commented 2 years ago

The issue appears to be flexibility of reporting, not what third parties might mandate for content they accept. I'd also expect that what someone accepts for procurement purposes and what issues are in the content would be in whatever contract they have with the publisher, not solely through the inclusion of metadata. It's too unreliable given that anyone can claim anything they want, and a disclaimer would probably be unnecessary given that minor mistakes are already an understood possibility.

The issue that keeps coming up is whether there is a way for a publisher to allow a statement such as a VPAT to speak to the conformance, or lack of, instead of making a WCAG claim in the metadata. I don't see why we shouldn't allow that given that we have a base-level requirement that is exclusive of making any WCAG claim.

I expect market forces will shape the conformance claims that publishers use, but in the interests of allowing people the flexibility to choose their own paths I don't see why we can't accept both discoverability claims augmented by reports and full conformance claims offset by disclaimers.

And as I said above, requiring discoverability metadata and then having no clear way to indicate that that's the only requirement met leaves us a hole where we can't be sure if a publication has been checked at all. All you can do is make a best guess if you find some metadata present. If we plug that hole, which I believe we should for completeness, then there's automatically no way to stop a publisher from making whatever claim they prefer.

avneeshsingh commented 2 years ago

It is OK to add conformance claim for discoverable. It would be really helpful to hear from the publishers, does it help you in communicating accurate accessibility conformance claims? It is not likely to disturb existing systems, because it is an additional thing.

mattgarrish commented 2 years ago

Using the pattern we've already established, I'd propose we add the new identifier:

EPUB-A11Y-11_DISCOVERABLE

As mentioned above, in addition to this solving the minimal case of how to identify publications that only meet the metadata requirements (so we know they've been checked), this also appears to be useful for the case of wanting to report WCAG conformance only through a separate report. In that case, you might have metadata like:

<meta property="dcterms:conformsTo" id="conf">EPUB-A11Y-11_DISCOVERABLE</meta>
<meta property="a11y:certifiedBy" id="cert" refines="#conf">Acme Certifiers</meta>
<link rel="a11y:certifierReport" refines="#cert" href="https://example.com/report/path"/>
<meta property="schema:accessibilitySummary"> ... This publication substantially conforms to WCAG 2.0 Level AA as described in the linked report.</meta>

Is this sufficient to address your needs @johnfoliot ?

@clapierre We can also keep the optional disclaimer property proposed in the existing PR if you think it will be useful for your needs. I still have to make some improvements to that.

But if this sounds workable, I'll try to get an updated pull request together.

clapierre commented 2 years ago

@mattgarrish Yes I would think that having both would still be helpful. So would this mean there would be 2 conformsTo statements one with EPUB-A11Y-11_DISCOVERABLE, and the other with EPUB-A11Y-11_WCAG-21-AA or could you include both in one conformsTo statement?

Also should it just be an overarching EPUB-A11Y_DISCOVERABLE or do we need to have multiple of them for EPUB-A11Y-10_DISCOVERABLE, EPUB-A11Y-11_DISCOVERABLE, etc. as the spec evolves.

mattgarrish commented 2 years ago

So would this mean there would be 2 conformsTo statements

No, because meeting the discoverability requirements is already implied by meeting the content conformance requirements. Those strings say you've met the full requirements of the specification (metadata and epub objectives) plus WCAG. This new string say you've only met the metadata.

Adding multiple strings might also confuse anyone processing the metadata, as it wouldn't be clear if one is in error.

Also should it just be an overarching EPUB-A11Y_DISCOVERABLE

The problem comes if the metadata requirements change in the future in an incompatible way. I'd keep these version-specific just as we keep the content conformance.

(We can't retroactively add conformance for 1.0, so we have to live with this only being available going forward.)

cmussi commented 2 years ago

ha all, here are some further considerations regarding three different audiences involved that may be confused if we include a new conformance level (limited to accessibility metadata):

best

avneeshsingh commented 2 years ago

@cmussi thanks for the comments. I was getting concerned because we were not hearing a lot from implementers. You have raised good points to think. One of the objectives of EPUB accessibility 1.1 is acceptance by EU as a technical specification, so anything that can cause a confusion on EU side is concerning.

mattgarrish commented 2 years ago

so anything that can cause a confusion on EU side is concerning

I'm starting to think, given the time constraints, it might be better to defer this to the next revision so we have more time to figure out a solution that works for everyone. Rushing solutions always comes back to bite us and we don't seem to be getting any closer to a consensus view on this.

If you don't want to claim conformance, you can still use the certifier metadata and link a report.

GeorgeKerscher commented 2 years ago

I tend to agree with Matt. We do not want to create confusion. Ultimately this is a WCAG 2.x problem, and most organizations have accepted the use of WCAG 2.0 as a requirement. Deferring this to a future version, perhaps once WCAG 3 is out, would be the safer approach.

iherman commented 2 years ago

The issue was discussed in a meeting on 2021-09-23

View the transcript ### 4. Normative references to a specification of metadata properties are needed. _See github issue [#1799](https://github.com/w3c/epub-specs/issues/1799), [#1767](https://github.com/w3c/epub-specs/issues/1767)._ _See github pull request [#1818](https://github.com/w3c/epub-specs/pull/1818), [#1815](https://github.com/w3c/epub-specs/pull/1815)._ **Avneesh Singh:** These are the reminders, there are some PRs with requests to review **Avneesh Singh:** there's some further discussion on zero-tolerance for a11y review, please provide comments … AOB? **George Kerscher:** Asking for comments for these on GH? **Avneesh Singh:** The PRs need review, and the last one is an issue, please provide comments on that **George Kerscher:** Personally, the approach being taken for discovery seems ok **Avneesh Singh:** There's more emails **Charles LaPierre:** I was just looking at the editor's note FXL techniques … diff and preview have errors > *Charles LaPierre:* Add editor's note mentioning the fxl techniques #1818 > *Charles LaPierre:* [https://github.com/w3c/epub-specs/pull/1818](https://github.com/w3c/epub-specs/pull/1818) **Wendy Reid:** It's already merged, so there's no "diff" > *John Foliot:* matt, FWIW, another emergent spec with a section related to security and privacy. Note that this WG are in ongoing discussions with i18n folks at W3C. See: [https://w3c.github.io/personalization-semantics/content/index.html#privacy-and-security-considerations](https://w3c.github.io/personalization-semantics/content/index.html#privacy-and-security-considerations) **Matt Garrish:** Just look at the file changes **Avneesh Singh:** Let's continue discussion on GH … APA call times will be finalized, and I will send them to the mailing list … see you all in the 3.3 call tomorrow … have a nice day! ---
mattgarrish commented 2 years ago

I'm going to move this issue to the CG tracker and close the open pull request. It's probably better to incubate the disclaimer property there first, and it doesn't have to be in a formal specification to be used in the package document. We can also frame the note defining it to encompass the problems we've run into and seek more feedback on how to tackle this problem in the future.