Open ericwbailey opened 2 years ago
caitlynmayers
commented
2 years ago I'd like to second Eric's statements and ask — rigorous analysis of this new method is an important step that I think the W3C should follow as they form WCAG 3. Thank you!
frankelavsky
commented
2 years ago Yes, thank you Eric (and thanks for linking to my issue on APCA)! This is probably my single biggest ask of the silver WG right now. I want WCAG 3 to be successful and I want APCA to be successful too.
And part of that success is verification: claims of an empirical foundation need to be backed up with clear references, not just a working model. We need to see under the hood and how all of the constants and variables are informed by science, since that is the primary argument for APCA as a better choice. It will set a dangerous precedence if we claim to be scientific and cannot demonstrate why clearly.
To me this means some time spent on documentation. If we think in terms of an academic paper, we already have a "related works" and "results" section, but we are missing "methodology."
bruce-usab
commented
2 years ago I am questioning if the methodology behind the APCA is accurate, effective, and reproducible.
@ericwbailey (and everyone) have you dug into the algorithms a little bit? See the APCA read.me.
I am not a mathematician, but I can follow the logic well enough to appreciate that is essentially of the same form/structure as the current luminosity contrast algorithms. The RGB weights are the same to a couple decimal points.
The numbers APCA generates are accurate and reproducible. The effective quality has to be true as well, since we know 4.5/3.0 thresholds have proven, over time, to at least be good enough.
The only open question in my mind is picking defensible numbers for the thresholds for what is good enough as a minimum baseline. APCA is not strict with this, and returns a sampling. Which is exactly what I think folks should be looking for a this time.
As a bit of background, there was very little empirical data for the thresholds used for WCAG 2.0. The WG did the best we could at the time, but there just was not much research available. It has worked out. I don't think there was ever a peer review
per se. Yes, we need more testing with end-users. But if anything, the real-world validation is so that thresholds picked for using APAC are not too relaxed.
I am not understanding the demand for the scientific rigor for APCA when (1) we did not have that with the current approach (and still do not), and (2) by inspection, APCA is better (by just about any objective metric [except simplicity and familiarity]) and (3) mathematically more defensible.
frankelavsky
commented
2 years ago I am not understanding the demand for the scientific rigor for APAC when (1) we did not have that with the current approach (and still do not), and (2) by inspection, APCA is better (by any objective metric) and (3) mathematically more defensible.
As a researcher myself, when there are claims made about something being backed by empirical science,[1,2,3,4] I need to know what those studies are. If it was just a mathematically more defensible model that produces better results, I'd be fine with it. But stating that it is built with a certain level of academic rigor is something that the entire research community is going to be interested in.
In addition, there are several references to ongoing "empirical work" as well[5,6]. While this is interesting and promising, this also seems vague and potentially suspicious. While there could be "ongoing" empirical work or studies (plural), I haven't seen an open call for participation and I am not sure how someone could possibly recruit all the necessary participants across the broad spectrum of affected disabilities without multiple open calls. In addition, the budget and logistics for this work is non-trivial. We're talking some serious funding and time involved. While it is possible to do this work entirely from a protected IP perspective, I am also considering the possibility that these are attempts to make APCA appear more authoritative.
That second possibility raises questions about why we are choosing APCA and not perhaps another model which is easy to use and understand, still backed by science (and can demonstrate it), and with an open/permissive license. The closed nature of APCA and the IP concerns involved just make me nervous.
Individually, each of these pieces are not cause for suspicion. But adding up claims that this is backed by empirical work, that empirical work is ongoing (and is likely a significant undertaking), and that there are IP reasons things are not brought to light... this makes me wary from a research perspective. Can we trust it? Where are the honest signals? As I said, it perhaps would have been better if it was just a mathematically more defensible model that produces better results.
The accessibility research community has long had issues with WCAG, so to claim that part of WCAG 3 has a level of rigor that hasn't been seen before is pretty important. Researchers have wanted this for a while! So if this claim turns out to be false after WCAG 3 is out in the world (the worst-case scenario), then there could be significant damage between the standards and research communities.
But all this being said: I do want more rigor in WCAG. Just because that wasn't what was done with previous contrast models doesn't mean that should be how things are always done. The research community does have decades of work on color perception and even existing models, all of which I am sure the researchers would be happy to see have contributed to the standards community (even if towards APCA). In addition many researchers (myself included) would love to contribute to projects that have such broad impact, especially in situations where there is still work that needs to be done.
APCA really does have the chance to do this, so to me the stakes are quite high. Despite my suspicions about the worst-case scenario, I am still hopeful for the best-case scenario too. And if there is an opportunity to bring some empirical research into silver's efforts on contrast, please reach out. Perhaps claims of "ongoing research" mean that there is still work to be done. And perhaps APCA could use some assistance with that, in whatever stage it is in. (If not me, then I certainly know folks in research spaces with a willingness to contribute.)
Myndex
commented
2 years ago Well, hasn't this all become a little firestorm.
Hello @ericwbailey @bruce-usab @frankelavsky @caitlynmayers
EDITS IN PLACE:
The APCA and the related work is a product of a research and development project following the issue thread I started in April of 2019 in the WCAG GitHub issues tracker. While much of the work has been conducted at our lab, it has also been a function of the Visual Contrast Subgroup (VCS) of Silver, and the work has been under the review and oversight of the members of the VCS, one of whom is also a member of the US Access Board, as well as members of W3 and members of the public in general.
The issues and discussions tab has been open at the APCA repo since I moved the project to GitHub as open source, and I am proactive and responsive to comments and discussions there.
The VCS merged into the newly reformed LVTF, something I suggested last year as there are many more visually related issues besides contrast that need examination in terms of guidance and standards within the W3.
As it stands, there have been several independent reviews that demonstrate efficacy. There are threads in the LVTF discussions section with evaluations and review by other peers. But moreover, the evidence regarding APCA is prima facia. I have presented it openly for years, and have written about it extensively, and always with clear visual examples, and evidence that is open and easy to recreate and verify.
@caitlynmayers
I'd like to second Eric's statements and ask — rigorous analysis of this new method is an important step that I think the W3C should follow as they form WCAG 3. Thank you!
I whole heartedly agree. Such analysis has been an ongoing part of the development. There are limited resources, as the W3 is a non profit and this work is being donated to the W3 for free. But the work and the studies are continuing, and we are about "in the middle" of that process.
Something you might find shocking about academic work is that authors of scientific research do not get paid for their articles: in fact they have to pay for them to be published, and they receive nothing from the publish. And it is worse for open-access. For instance Nature charges over $11,000 to the author for publishing their paper in the open access section of Nature's website. Eleven thousand, that was not a typo.
More to the point though, peer review for publication is not a "stamp certifying efficacy", and claims that it is misunderstand the purpose and process.
@frankelavsky
We need to see under the hood and how all of the constants and variables are informed by science
Absolutely!!! When I publish that information will be laid bare.
I am considering issuing NDAs for a small team for review earlier than later. To be honest, the evidence is prima facia, and so I am a little surprised by the last couple days of "firestorm", but perhaps we should have a private conversation to discuss away from the distractions.
In the meantime, the basic math for your enjoyment:
when there are claims made about something being backed by empirical science,... But stating that it is built with a certain level of academic rigor is something that the entire research community is going to be interested in.
Yes, well understood. There is so much more I am eager to disclose, including modules that more directly account for certain CVD types in a minimally invasive manner, which while slightly less critical for sRGB is going to be very critical for UHD Rec2020 and Rec2100 color spaces.
We are in the MIDDLE of a multi-year process.
I haven't seen an open call for participation
That is why I am posting on Twitter, among other places. And there have been many calls for comments, usage, evaluation — The software for the larger scale studies is not complete, the new target date is spring summer 2022.
If you are good at grant writing, that might help. The project is underfunded to say the least.
That second possibility raises questions about why we are choosing APCA and not perhaps another model which is easy to use and understand, still backed by science (and can demonstrate it), and with an open/permissive license. The closed nature of APCA and the IP concerns involved just make me nervous.
It is not closed what makes you say that? Have you read anything about it? I have listed all the links. It will take you some time to get through it. As far as I can tell, this just leapt up at you yesterday. Have you digested all the material yet? By all I do mean the links and their sub-links listed above.
Had you, you would know that I am bound by my membership to donate the essential IP for the basic APCA for use in the WCAG_3 guidelines.
Evaluating existing models was done very early in the project in 2019 — the only reason I began development of the SAPC project is the lack of suitable models for self-illuminated displays, and a pilot study in 2019 indicated a more suitable method could be developed.
to claim that part of WCAG 3 has a level of rigor that hasn't been seen before is pretty important
I never made any such claim. As you don't know me or the project, you are not aware of the work. I have been performing all due diligence. the fact things are just now becoming more publicly noticeable is in preparation of the greater awareness needed for the larger part of the Perceptex project.
so to me the stakes are quite high.
The stakes are incredibly high — I'm the first one to say that, and here's why:
Consider, projecting forward, the implications of any society, of a world society, that no longer reads.
Consider how a public that does not read is a public that is poised on the edge of a dangerous slippery slope — the same kind of slippery slope we've seen in the past that results in devastating sociopolitical consequences.
Hi Bruce! @bruce-usab
For the last week I've been thinking "what would Bruce do"...
Thank you for providing the helpful added context. I'm adding a bit as well below:
I am not a mathematician, but I can follow the logic well enough to appreciate that is essentially of the same form/structure as the current luminosity contrast algorithms. The RGB weights are the same to a couple decimal points.
Indeed the reduction to luminance Y is similar. The changes may seem subtle, but are part of the modeling the output of real world self-illuminated displays.
APCA is using more refined algorithms, so light-on-dark is a different result than dark-on-light. It is a bit of quirk that the current algorithm is agnostic to this, when a perceptual difference is quite apparent from casual inspection. People have been okay with shrugging that off.
Yes the "APCA engine" is uniquely different, but is related to modern color appearance models. It is not creating a ratio, but instead a perceptual difference value using a series of power curves.
APCA is catching (failing) color combinations that are obviously terrible (but permissible with the current SC). It is a bit of a quirk with the current algorithm that these are missed, but the combinations are terrible enough for everyone, so designers generally avoid them. It would certainly be an improvement to have a WCAG metric to flag those!
Not just an improvement, but functionally important. There is movement in the CSS group for automated contrast properties — such automated properties are going to be a VERY bad thing if they are based on WCAG_2!!!
As you know, my studies in 2019 showed that WCAG_2 incorrectly passed 49% and incorrectly failed 23% of random colors. Sam recently did his own independent study and found it to be 55% and 30% respectively. YIKES!!!! Just think of the problems if CSS uses that for an automated contrast property?!?!
APCA might permit use of some color combinations (for example, white foreground text on a red button), which the current algorithm excludes. It would be a good thing to be able to provide developers some some additional flexibility, if accessibility is not impacted.
I've actually shown that for CVD types, accessibility is markedly improved here. I suppose it's my bad hitting the hornet's nest when I created the APCA logo using what I call "APCA Orange" #f60 which fails all WCAG_2 contrast, but passes APCA (of course) and as you can see in this slide, all CVD types do well, despite WCAG_2 wrongtrast failing it.
The only open question in my mind is picking defensible numbers for the thresholds for what is good enough as a minimum baseline. APCA is not strict with this, and returns a sampling. Which is exactly what I think folks should be looking for a this time.
That is in part the purpose of the public beta test, and also of the spring 2022 studies.
As a bit of background, there was very little empirical data for the thresholds used for WCAG 2.0. The WG did the best we could at the time, but there just was not much research available. It has worked out. I don't think there was ever a peer review per se. Yes, we need more testing with end-users. But if anything, the real-world validation is so that thresholds picked for using APAC are not too relaxed.
I am not understanding the demand for the scientific rigor for APCA when (1) we did not have that with the current approach (and still do not), and (2) by inspection, APCA is better (by just about any objective metric [except simplicity and familiarity]) and (3) mathematically more defensible.
Thank you very much Bruce for that recap. I don't seem capable of getting things across as effortlessly as you!!
Thank you ALL for the comments and I hope we can move forward postively and proactively.
Andy
vavroom
commented
2 years ago I am a "member of the public", no longer directly involved with WAI. I was active on the EOWG for 18 months and was going to work on AG but life got in the way of that. I have been involved with accessibility since the mid 1990's. I currently am an independent accessibility consultant and I deal everyday with having to explain accessibility and WCAG to clients who find the guidelines hard to understand. I need to justify success criterias all the time. I need to explain why elements of accessibility, such as color contrast work, or don't work.
In that context, the prospect of having to explain a tool that hasn't been peer reviewed by experts concerns me greatly.
APCA may, or may not, be the right approach. I don't know. I do know that the aspect of having to make a subjective call around font style fills me with dread. The last thing we need is more subjectivity in decisions. Maybe I'm misunderstanding the documents I've seen about it but from my read, after taking into account background and foreground colors, and font weight and size, which are all calculatable, the last step is relying on judgment about the font. If I'm correct, this in and of itself will be the cause of endless problems.
That said. It seems to me to be verging on conflict of interest for a tool/methodology creator to push it so strongly for adoption into a standard. It would be another matter if the tool/methodology had been peer reviewed and was approaching industry standard already.
What bothers me even more is the apparent high level of defensiveness anytime someone expresses a sentiment that could be perceived as negative towards either the tool or the tool's author. Having been part of discussions with @Myndex here and on Twitter, and having reviewed other interactions, I am seeing a lot of passive aggressiveness that has no place in standards creation. I have heard and seen complaints from multiple individuals who felt bullied by Andrew's approach. When comments are made to Andrew about these, he dismisses them out of hand, blaming a small group of people. Having known several of these people for many years, not just online, I know their integrity and if it's a question of taking sides, I'll side with them any day against someone who consistently exhibit passive aggressive behaviors verging on bullying.
To whit: His refusal to address @ericwbailey's concerns directly. Offering to go privately - this is pretty typical behavior from abusers. Note, I'm NOT accusing Andrew of being an abuser. But his behavior is on the edge. Interacting with him feels very much like being gaslighted at every turn.
I hope I'm wrong, but I fear I'm not.
Circling back to APCA, however, and dissociating the person from the tool/methodology, we still need to ensure that whatever makes it in the final version of AG has been peer reviewed and is accepted by the group through the consensus process of the W3C. And consensus cannot happen when individuals in the group behave in such a way that other group members just "give up". I suggest that because of the risk of conflict of interest, Andrew Somers should recuse himself from discussions around this tool/methodology.
ericwbailey
commented
2 years ago Hi @bruce-usab, thank you for your reply. I am also not a mathematician, and I also was able to follow the general gist of the algorithm.
When you mention accurate and reproducible, that's exactly what I'm wondering about. Internally accurate and reproducible is not the same as external, especially with confirmation bias in play.
I would also like to re-emphasize this point:
I am also not saying current color contrast calculation is effective,
I am not questioning if the current color contrast algorithm is effective enough. I am asking if the APCA is a suitable replacement, or if another algorithm should be sourced. The way to determine this is to see if the APCA can pass muster.
by inspection, APCA is better
I am not sure about this, given how supporting content falls apart at closer inspection. @frankelavsky expounds on this far better than I could in https://github.com/w3c/silver/issues/574#issuecomment-995305761.
ericwbailey
commented
2 years ago @Myndex You previously mentioned you would prefer GitHub communication. I am doing that here.
The standards process is open and transparent, and I am observing that by posting this issue. I would like to say that your request for private communication does add to my perception of a pattern of behavior of deflection and obfuscation when pressed for direct answers. This leads to an erosion of trust.
I also would like to re-stress this point:
Andrew’s communication patterns both on and off GitHub create an obfuscating effect, where longform messaging dodge or deflect direct asks. This could be intentional, accidental, or both.
We all understand the value, and impact of an updated contrast algorithm and how it affects reading on digital screens. In addition, I am not questioning your passion for your project. There is no need to expound on it here.
modulist
commented
2 years ago One of my major responsibilities at work is to create content that explains accessibility to designers, developers, and managers — who are not accessibility professionals. My standard is to create content that's not only simple enough for a designer to understand, but also simple enough for them to explain to a co-worker.
Before this flared up into a personal issue, I was hoping to be able to research APCA and be able to write a series of pieces explaining it to my audience. Here are some sample questions I was hoping to cover:
The intent was to be able to explain this to someone with a high-school level math background and a high-school level of English, which may not be their first language.
APCA and its documentation are not self-explanatory. The image of the Contrast Prediction Equation in this thread isn't particularly helpful to me or my audience. It needs a legend or a step-by-step explanation, which is missing.
Those of us who aren't familiar with APCA and its development will need a learning path through this, and it will require patience and civility from all parties.
I sincerely hope we can find a way to gain more transparency so we can put our own support behind APCA and promote it.
bruce-usab
commented
2 years ago @modulist — do you think you will be able to make your APCA explanation public facing? I think it would be helpful to a have a skeptical third-party write that up. You might make it three parts:
My own experience with @Myndex is that he is listening and does make changes with his model implementations in response to feedback and criticism. I am taking the liberty to suggest this 3-step approach, since (1) is valuable regardless (even if only for your internal audience). Also both (1) and (2) can be agnostic to an appreciation (or not) of APCA. (3) is probably the most important, but needs to be informed by (2). And even if you conclude that you cannot endorse APCA, that still gets the world closer to something better than what we have with the 2.0/2.1/2.2 metric. I do not know of anyone asserting that the current 2.x approach is sufficient for WCAG3.
@ericwbailey and @vavroom et al. -- What I have been doing for the past year or so is trying to remember to check APCA scores any time I have had call to use the WebAIM contrast checker. This habit has greatly increased my comfort with APCA. With the benefit of hindsight, (1) I wish had been more disciplined with this practice, and (2) taken notes. Maybe that should be my New Year's resolution?
For the last week I've been thinking "what would Bruce do"...
Thank you for that kindness, but that way madness lies...
vavroom
commented
2 years ago @bruce-usab Who is being pejorative here?
As for "getting comfortable with APCA", that's not an issue for me. Telling @ericwbailey and i to familiarize ourselves with the tool doesn't address the very real concerns several people have raised.
Whether or not I like the tool/methodology doesn't stop it from not having been peer reviewed. Doesn't stop it from having claims of scientific rigor behind it that have no transparent support around it.
I stand by what I said earlier.
Myndex
commented
2 years ago Hi @vavroom Thank you for your comments.
In that context, the prospect of having to explain a tool that hasn't been peer reviewed by experts concerns me greatly.
Yet you are doing that now by using the WCAG_2 SCs. They are a product of internal consensus, not independent peer review. Moreover, there is a process in place, and a process that is being followed toward consensus, and peer review publication, and this has been part of the plan from the very beginning, along with the complete openness I have provided as the studies have progressed, from the beginning in april 2019.
The rumors that "there is no peer review" are misleading at best, but otherwise somewhat faux concerns. This project has had ample oversight and review.
The last thing we need is more subjectivity in decisions. Maybe I'm misunderstanding the documents I've seen about it but from my read, after taking into account background and foreground colors, and font weight and size, which are all calculatable, the last step is relying on judgment about the font. If I'm correct, this in and of itself will be the cause of endless problems.
Here is an example. All of these fonts samples are set to font-size: 14px (except of course non-black headers) do you see the problem?
The propertyfont-size: is not a valid way to determine the actual readability of a font. This is equally a problem with font weight.
That said. It seems to me to be verging on conflict of interest for a tool/methodology creator to push it so strongly for adoption into a standard.
I am not a tool creator. I am an independent researcher, with no interest in creating tools. The tools that I have presented were only created to demonstrate the technology, a common practice in research. There are dozens of tool creators that have incorporated APCA into their tools, along side the existing WCAG_2, for the purposes of beta testing and evaluation.
I am not "pushing strongly" for adoption — I was brought in as an invited expert after starting thread 695, with the express goal of finding a workable solution for Silver.
For the backstory, you can start with issue thread 695 in WCAG GitHub where I identified the significant issues with the WCAG_2 contrast method. I am not someone to complain and leave, I am proactive in finding the solution. In this case I have devoted nearly three years of full time research focus to solving the WCAG_2 contrast problem.
It would be another matter if the tool/methodology had been peer reviewed and was approaching industry standard already.
APCA is a project in development. That it is visible is because it is being developed OPENLY and to encourage public comment while doing so. The issues and discussions tab are open for any and all questions comments and open discussions regarding APCA. This is what is done with an open project before formal peer review.
Nevertheless, peers have already been involved. Studies and live interactive experiments have been public facing for years. If you have only heard about this recently in social media is because we are working to increase awareness and generate interest for gathering comments and for future planned studies.
What bothers me even more is the apparent high level of defensiveness anytime someone expresses a sentiment that could be perceived as negative towards either the tool or the tool's author.
The project is open to all criticism and discussion as I mentioned before. The claim that it is not is part of some rumors of unknown origin. Other than certain individuals have been invited numerous times to formally discuss any issues they desire as part of the consensus and peer process — yet they have refused to do so.
This is a public beta. If you have an actual problem with the math or the methods, please file an issue so we can track and address it.
Linktree of the most key chunks of data Start at the top and work your way down.
The W3 Visual Contrast Wiki This page is divided into several pages that fairly completely cover the subject matter with additional referencing links.
List of articles/gists here on GitHub A number of writings on contrast and color include several key comparative articles that demonstrate the WCAG_2 and APCA differences.
In depth issue threads These threads contain a mass of the early research, all out in the open.
SAPC Contrast Research Tools The WCAG3/Silver Contrast Algorithm beta site. Includes interactive experiments that demonstrate the concepts. Click on "research mode" and then use the experiments to evaluate aspects of contrast.
Color Vision Deficiency Simulator This clinically accurate simulator demonstrates the way someone with a Color Vision Deficiency (incorrectly labeled "color blind") sees colors.
And the GitHub repo here, see the Discussion tab.
Peer review occurs when the "work" on the project is complete and a subsequent paper is submitted for publication. In some cases there may be concurrent review, or concurrent oversight, and there is some of that going on. But again, the evidence is prima facia—clear to see.
Myndex
commented
2 years ago @Myndex You previously mentioned you would prefer GitHub communication. I am doing that here.
Regarding APCA directly, and at the APCA repository which is over there. All questions or issues with the APCA project should be made in the appropriate repo.
Myndex
commented
2 years ago Hello Claudio @modulist
One of my major responsibilities at work is to create content that explains accessibility to designers, developers, and managers — who are not accessibility professionals. My standard is to create content that's not only simple enough for a designer to understand, but also simple enough for them to explain to a co-worker.
This is an area I am not succeeding with, LOL.
Before this flared up into a personal issue, I was hoping to be able to research APCA and be able to write a series of pieces explaining it to my audience. Here are some sample questions I was hoping to cover:
Excellent, this is the very kind of discussion/criticism that is needed. Please do not let the trolls or flame wars impede this progress.
I probably have answered all of your questions in this white-paper in progress: https://www.w3.org/WAI/GL/task-forces/silver/wiki/Visual_Contrast_of_Text_Subgroup/Whitepaper
But I will answer individually below as well. I am going to answer in "not" plain language initially, and then make my own ham fisted plain language attempt thereafter when appropriate. I tend to avoid this as I always worry about "talking down" to anyone, but I have been recently told that what I write is "obtuse and hard to grok" or words to that effect. So apparently, finding the right tone is challenging for me.
- _what's being measured? If this is a two-dimensional formula, what are the x and y axes?_
APCA is measuring the supra-threshold perceptual contrast of certain kinds of stimuli on a self illuminated monitor, device, or display. The larger SAPC model is multi-dimensional. The simple W3 licensed APCA is derived from this larger model, and an attempt has been made to fit to the 3D to 7D empirical data sets in as close to 2D or 3D as possible, but you might call it more like a "2.5D" as things like polarity and spatial frequency are important (and sometimes more important) than the color difference. Some of these are presently separated in a LUT, but some aspects are on their way to being incorporated as parametric curves. At that point, it is at least a 3D construct. There you might say that Y is the total luminance context, X as the Lc value (difference), Z as the spatial component. That is probably the irreducible de minimus.
The Lc value can be used directly as a guide, as it is in Bridge-PCA, the WCAG_2 100% backwards compatible version. Or design flexibility can be further improved by matching it to a lookup table of font values to find a minimum font size and weight.
The amount of predicted lightness for the text color is subtracted from the predicted lightness for the background color to arrived at the predicted lightness contrast, or Lc value.
- how were the formulas behind APCA created?
Some of this is part of proprietary information, and will be presented at the appropriate time. It is not open for public discussion YET but I am committed to it being open to the public.
In the meantime, I will say that it is a combination of research into all of the existing appearance and color models (like RLab, CIELAB and CIELUV, CIECAM02 and CAM16, Hunt, etc) and a lengthy series of experiments with the existing contrast methods (publicly available on or through the Myndex site) and fitting to empirically created data sets here in the lab. And other fun things that I am bursting at the seams to talk about but that my patent attorney tells me I have to shut up about for the time being.
I am only going to hint at one little thing: there is a lot more that goes beyond contrast. (In Hollywood we call that a "teaser").
- how were the constants in the formulas arrived at? can we check them?
Same as above. And yes you can check them, they are public. (!!!) I may be misunderstanding your question?
- if testing was done, what was the test experience like? how could someone else reproduce the tests?
The testing environment is that of the SAPC standard observer, which is detailed at the APCA github repo.
- can we improve or better calibrate APCA in the future?
There is a planned wide-area study for 2022. Also, the entire function is designed to be extensible and adjustable. I have hinted at what some of the extended functions are in the source code, which is publicly available.
However, right now, there seems to be a cry for "keeping it simple" so I am not going to get into the future possibilities.
- what impact do factors like font-weight, reversed text, and ambient light have on APCA?
These things are directly taken into account!!!!!
Font weight and font size are critical to understanding the APCA guidelines, as these relate to spatial frequency. Spatial frequency is a primary determiner of perceptual contrast. Here is a crude infographic that outlines this (not for publication, I'm making a nicer one, LOL)
What we see here is that as spatial frequency increases the perception of contrast (and importantly readability contrast) decreases dramatically. This is reflected in the font lookup tables for APCA.
Regarding "reversed" text, I assume you mean reversed polarity, as in light text on a dark background. I already did mention polarity as part of the fundamental, basic APCA function (above). But we can stop for a moment and look at some factors as to why. One of them is local adaptation; the HVS (human vision system) is an amazingly complex system with an incredible range of sensitivity — from a flickering candle miles away on a moonless night to the blazing noon sun. But we can't see all that at one time, we have to adapt to the conditions, known as lightness (or darkness) adaptation. (think for instance the effect of being in a dark theater for a Saturday matinee and then walking out into the sunshine.)
There is also local adaptation — in this case, I mean the background immediately around the text. Our adaptation level is a combination of our environment's ambient light, the overall light from the screen, and the very local light from the screen and text. It should be obvious that with light text on dark, the overall area lightness is darker, resulting in a darker adaptation and thus causing light text to appear "brighter" that dark text for the same (inverted) pair of colors. But this is also dependent on the overall location of that pair relative to the ambient lightness adaptation.
In the case of the basic APCA, there is some pre-estimation. There are future versions that are pending release upon IP filings that directly account for some of these factors. Those are for more critical applications.
Yes, font weight and size are very important, defined in a lookup table to provide more accurate guidance to designers. Small thin fonts need more lightness/darkness contrast to compensate for their inherently decreased contrast.
Reverse text, light on dark, is indicated with a negative number. but is scaled for similar readability. I.e. Lc 60 and Lc-60 are essentially the same perceived contrast.
The intent was to be able to explain this to someone with a high-school level math background and a high-school level of English, which may not be their first language.
January Edit: Bad math joke deleted, and I've reviewed and re-written the "Why APCA" document, and tested the "readability score" to fit that of a high school senior (ot at least my best attempt at that).
APCA and its documentation are not self-explanatory. The image of the Contrast Prediction Equation in this thread isn't particularly helpful to me or my audience. It needs a legend or a step-by-step explanation, which is missing.
Okay, on the page in the APCA repo is a plain language step-by-step. Please let me know if I can help break it down further.
Those of us who aren't familiar with APCA and its development will need a learning path through this, and it will require patience and civility from all parties.
Yes, Alastair @alastc made me aware that I need to write a one page explainer, and I am working on that among the many other year-end tasks I am buried under. At the moment, I've generated a massive amount of material, and I'm not that certain what would be best for you to start.
I did create a linktree with what "I think" is the best path to climb the learning curve. Start near the top and work your way down:
I sincerely hope we can find a way to gain more transparency so we can put our own support behind APCA and promote it.
To be honest I do find statements like this a bit frustrating, as I have been completely open internet rumors notwithstanding. Comments made to the APCA repo discussion area are promptly responded to.
Here is a further link list, feel free to read it all and please let me know if you have any questions:
The W3 Visual Contrast Wiki This page is divided into several pages that fairly completely cover the subject matter with additional referencing links.
List of articles/gists here on GitHub A number of writings on contrast and color include several key comparative articles that demonstrate the WCAG_2 and APCA differences.
In depth issue threads These threads contain a mass of the early research.
SAPC Contrast Research Tools The WCAG3/Silver Contrast Algorithm beta site. Includes interactive experiments that demonstrate the concepts. Click on "research mode" and then use the experiments to evaluate aspects of contrast.
Color Vision Deficiency Simulator This clinically accurate simulator demonstrates the way someone with a Color Vision Deficiency (incorrectly labeled "color blind") sees colors.
And the GitHub repo here, see the Discussion tab.
chaals
commented
2 years ago Dear all,
This is a request from a very long-term but now very occasional WAI contributor to keep to technical issues in W3C's issue trackers, including in this issue.
W3C is a very diverse community of people who have many skills, talents, perspectives, histories, cultures and personalities. To help ensure a respectful and productive work environment, there are several different formal mechanisms to deal with communication problems and inappropriate behaviour. They don't include "discuss the merits of individuals on Github", because that is not appropriate behaviour.
It is possible to edit recorded comments in Github. Sometimes, that is a way to improve a situation. I suggest it would be helpful in this case.
(This isn't the only issue where this problem has arisen, it is the one that most recently stood out to me).
yatil
commented
2 years ago Just commenting because there is a misquote in @Myndex' comment above:
The claims that "there is no peer review" from Eric Eggert are misleading, false, and/or misappropriated faux concerns. If I am defensive regarding Mr Eggert, it is due to his misleading statements he has been making for a year, despite good faith attempts to describe to him the actual process and state of development.
First: I never claimed there was no peer review, just that the APCA algorithm has yet been tested with colorblind users. I also object to calling concerns unilaterally "faux concerns".
Second: I don't care about peer review that much. I just want to make sure that we have checked that the extension to color spaces that are currently not covered are really an improvement. Basically not repeating the WCAG 2 error.
Third: My concern is mainly about the state and communication around WCAG 3. The algorithm is beta, the standard far out from completion. This is important to communicate clearly before people run into issues and need to redesign because they misunderstood.
Fourth: None of my statements has been misleading. WCAG 3 is not done yet, the verification of APCA as a superior tool has not happened (but should be easy to verify). The current, very early draft of WCAG 3 has all kinds of vagueness that needs to be cleared up before people should use it.
Fifth: Despite my attempts to ask you to not engage with me, you keep doing this, including the misstatement above. Please stop in accord with the W3Cs Code of Ethics and Professional Conduct.
Myndex
commented
2 years ago Hi chaals @chaals thank you for commenting.
This is a request from a very long-term but now very occasional WAI contributor to keep to technical issues in W3C's issue trackers, including in this issue.
Earlier this week I had invited Eric Bailey @ericwbailey to create a discussion at the SAPC-ACPA GitHub repo for discussion of APCA, and gave him a direct link to that area. I did not suggest that he come here to discuss APCA, with is independent of the WCAG_3. (I also did not invite him to discuss me personally at any time, but that I suppose is moot at this point).
For reasons unknown to me he placed the discussion here.
ericwbailey
commented
2 years ago I hope to address other comments soon, but to clarify the issue of doxxing and the reason for this issue:
This is the repository for Silver/WCAG_3. Several things you brought up are personal and or not relevant to the work, and paramount to doxxing and is completely inappropriate here. I consider them part of an attack on myself, personally. If you have questions or problems with the WORK, I will respond, but I am done responding to these petty personal attacks.
This is not doxxing, nor are they personal attacks. This is information that you have voluntarily shared, cross linked to, and mentioned on the public internet that was accessible via a cursory search. By your definition, investigative journalism is doxxing.
I am including this information because it helps to provide context and illustrate my journey to submitting this issue as something that needs to be brought to the Silver WG's attention. I was initially thrilled at the prospect of an updated color contrast algorithm, but the more I researched your work the more skeptical I became.
Your call of doxxing also highlights another issue, a pattern of behavior where requests for clarification or constructive criticism are construed as personal attacks, leading to a disproportionate response.
This is why I am asking for independent verification of your work.
bruce-usab
commented
2 years ago @vavroom asked me:
Who is being pejorative here?
That was a reference to @Myndex lucid criticisms of the 2.x algorithms.
Telling @ericwbailey and i to familiarize ourselves with the tool…
@ericwbailey and @vavroom et al. -- What I have been doing for the past year or so…
Thank you for your patience with me. I did not presume to tell
you (or @ericwbailey) what to do. I meant only to share only a strategy that has worked well for me.
When it comes time to firmly determine the consensus thresholds, we will benefit from the experience of people routinely using APCA. The 4.5:1 threshold in 2.0 was a best guess. There were other values considered, e.g., 4.65:1 (i think that was it) and 5:1. We have kind of gotten lucky with how well it has all worked out!
alextait1
commented
2 years ago Hi all. I'm an independent accessibility consultant and not involved in the w3 working groups. But reading this thread alone I'm disheartened by what's here. I don't have the historical context of various tensions but I can see here that there are concerning communication patterns from @Myndex.
There's an undertone of "you don't understand math so you hate me". I mean literally they said "It recently came to my attention that people do not like math. I never thought of myself as a "math guy" but... well, I am now working to hide the math under the rug so designers do not have to be exposed to those pesky numbers..." and then memes about math and a comment about being a Hollywood villain.
I welcome change in WCAG, I love science and these advancements could be really awesome if they work. But the communication here is really concerning.
ericwbailey
commented
2 years ago I am not the focus here,_
You are the person working on the algorithm, and I am asking for a process to be outlined to review the algorithm.
This is not a personal attack, it is a request for independent verification. Information you have publicly shared of your own volition is an important part of the overall context, as it affects perception of your credibility. My perception of this information led to skepticism, which led to this issue.
To reiterate, I ask because this will affect all visual content on the internet.
All questions or issues with the APCA project should be made in the appropriate repo.
I am asking if and how the Silver WG will test your work to verify its efficacy, hence posting it on the Silver GitHub issue tracker. Had someone else been working on an update to the WCAG 3 color contrast algorithm created the same sense of skepticism, I would have done the same.
Thank you to all participants for your collective patience while I work through other comments.
ericwbailey
commented
2 years ago
- Problems with the current 2.x approach/algorithm. @Myndex has provided a great deal of evidence here! Personally, I would also like something (1) a little bit more oriented to the layperson (which might describe your audience), and (2) less pejorative when describing the status quo.
- Your analysis, good or bad, of how APCA addresses those defects.
- Your prognosis for APCA.
@bruce-usab I like this, thank you for suggesting it! I also think it is an excellent stepping stone towards independent verification, with someone who is not a layperson eventually also being able to give comment.
And even if you conclude that you cannot endorse APCA, that still gets the world closer to something better than what we have with the 2.0/2.1/2.2 metric. I do not know of anyone asserting that the current 2.x approach is sufficient for WCAG3.
Would the Silver WG be open to an alternate algorithm? The A11Y Project has some funding, and we'd love to sponsor one that is open source and governed by a code of conduct such as our own.
ericwbailey
commented
2 years ago Nevertheless, peers have already been involved. Studies and live interactive experiments have been public facing for years.
Are these peers and experiments you have been conducting, or are they ones mentioned in your citations? If they are in the citations, how do they specifically map to what portion of the APCA's development?
Eric Bailey brought up issues that require discussing private, sensitive, privileged, and or proprietary information that is not for public discussion, period.
What specifically cannot be discussed? You can address NDA concepts in a way as to not violate them, and doing so would help my perception that you're purposely obfuscating.
The researchers that are performing peer review and analysis will come forward when they are ready.
A timeline, as well as who is performing these reviews would be helpful.
Links to links of publicly accessible information I have provided.
I would like to note this content is self-referential. Which, again, drives my ask for a process for independent verification.
Myndex
commented
2 years ago Hi Alex @Alextait1
Hi all. I'm an independent accessibility consultant and not involved in the w3 working groups. But reading this thread alone I'm disheartened by what's here.
Hi Alex, thank you for commenting — I am disheartened by your comment, but I am glad you made it. My entire world has been turned upside down over the last couple days and I am still recovering from the shock and stun of it all. I don't know if you are friends with anyone involved, but for the purpose of this message I want to (and will) assume you are an impartial third party.
EDIT: removed references to my misunderstandings and personal communication deficits.
The project is in a public beta stage, and I need people that are _not required to follow WCAG2 (for some govt contract for instance), to work with the tools and provide feedback to shake out any issues. This is in concert with the planned larger scale empirical studies that are way behind (due to COVID, my health, other factors).
In order to get excited and interested volunteers, I've been on social media, speaking to a narrow niche, creating awareness in the niche of accessibility and designers, particularly those familiar with accessibility, and developers. As other peers have evaluated, played with, written about, etc. there was rising interest, and this in turn brought in more individuals wanting to contribute and work with, all of which is needed for a project of this magnitude to succeed.
And all of that has to be done before the "peer review" — this is not the time for "peer review" this is the time for "beta testing" and study. If I over reacted, it is because the stakes are too high to allow internet rumors to gain traction.
The Internet destroyed the print industry. 20 years ago there was a magazine stand near my home in Hollywood that was a block long & carried (1000?!) magazines — today it's gone. When the Internet came, it slowly started shrinking as did all of the bookstores. By some estimates reading is down 40% over the last two decades. And a society that does not read is a society that is doomed to oblivion. We've seen it in the past; let's not repeat that mistake in the future.
It wasn't just the destruction of the print industry, reading is functionally harder on many devices and screens. Part of the problem identifies the WCAG2 contrast guidelines as "not fit fur purpose" to quote one reviewer (not myself). I'll say that 1.4.3 is not only not helpful for accessibility—it doesn't even work for people with standard vision. I cover my background in thread 695, but here I'll just mention that I've spent the last nearly 3 years of my life on this research project focused toward fixing the critical readability problem that faces the world. APCA is one part of the solution.
I welcome change in WCAG, I love science and these advancements could be really awesome if they work. But the communication here is really concerning.
Just for the record, the need for contrast is far beyond the web, and the scope of APCA is also more than the web. To be clear, I am donating the essential IP for WCAG_3 for web use. But I am also not "just" doing that. There are a lot of related issues beyond contrast that need to be corrected for readability, if not for web text, also for apps, and.... stay tuned, there will be more to come on this next year.
I have no idea if this message is/was the way to address this situation. I hope that it is, I'm exhausted, and I have software to still write. I do sincerely appreciate that you brought these issues up as an impartial observer. I'm trying to think of a funny punch line to end this but I should probably skip the line and just buy a beer.¹
Thank you
Andy
¹ that was supposed to be a "funny punchline"
alastc
commented
2 years ago This thread was locked due to comments that included personal attacks. The core question (what is the process for validating an updated contrast algorithm) is reasonable, so this issue will remain open until a response is drafted & approved.
However, we would like to encourage participants to edit their comments to remove references to people. Like any WCAG update, no guideline should rely on one person, and there is a plan to find experts to review the new contrast guideline at an appropriate point. Given that, it is not appropriate to focus on particular people in GitHub issues.
ericwbailey
commented
2 years ago This thread was locked due to comments that included personal attacks.
@alastc With the thread unlocked following my edits, I wish you had named me directly. I say this because of the named individual's behavior of on and offsite harassment led me to believe it could have been directed at him.
I am not questioning the need for an updated color contrast algorithm. I am questioning how we test that the algorithm that gets chosen is effective.
I would like to reiterate my previous comment:
This is information that you have voluntarily shared, cross linked to, and mentioned on the public internet that was accessible via a cursory search. By your definition, investigative journalism is doxxing.
In investigating the invited expert to learn more about their background (due, in part, to my initial excitement), I was led down a path that generated an extreme amount of skepticism. This necessitated the need to question the greater context in which the algorithm is being developed. Namely my big-picture concerns are observed behaviors of:
I am a strong believer in the the Feynman technique of learning for complicated and niche subjects such as these. From my observations so far, I see the opposite.
Like any WCAG update, no guideline should rely on one person, and there is a plan to find experts to review the new contrast guideline at an appropriate point.
If this is the case, I would ask for this thread to be closed to continue the conversation in https://github.com/w3c/silver/issues/272, as this is now a duplicate issue.
I would also like to ask for clarification and detail about what Andrew Somers' plan is for said research be addressed in that thread, given my level of skepticism.
I say this comment that as an individual I have been continuing to performing remote qualitative and quantitative research in a professional capacity during Covid. Transparency about the specific barriers preventing research from moving forward would be helpful to remove my view of evasive behavior.
I would also like to note that my perception of "no guideline should rely on one person" is that this has been one single individual pushing for change with no oversight, which helped inform the creation of this issue. If this is in error, I would like to be linked to supporting information.
I want this to be right, I really do. I also acknowledge that the stakes are incredibly high. It is so very vital the W3C can move forward on this with confidence.
a plan to find experts to review the new contrast guideline at an appropriate point
Given that there is now far more empirical evidence for this subject, as well as existing perceptual contrast algorithms derived from work, that I would like to reiterate this comment by @frankelavsky:
In addition many researchers (myself included) would love to contribute to projects that have such broad impact, especially in situations where there is still work that needs to be done.
It would help to assuage my fears a great deal if something like a working group was set up to help guide the new color contrast algorithm, where work can be done in a transparent way that fits the W3C's preferred workflow. Guidance on work in flight would be especially helpful, in that it decreases risk by uncovering (potentially critical) issues only at the end of the process.
I would also like to say feedback from an accredited, published researcher on work in flight would be very welcome, and The A11Y Project is more than willing to help fund it with the the Silver working group's blessing.
alastc
commented
2 years ago Hi Eric,
With the thread unlocked following my edits, I wish you had named me directly.
The thread was unlocked so everyone could edit their comments. It wasn't your original comments directly, but focusing the topic on one person (even in good faith) did seem to prompt a slide into personal attacks.
NB: I'm not sure if @Myndex can edit his comments at the moment due to a suspension from W3C activities/spaces pending an ombuds process and review of CEPC issues.
In general, we (the AG chairs) ask that the focus of github threads is kept to technical / process points. Particularly in a written medium it is too easy for people to take things the wrong way, and keeping to the core topics reduces the likelihood of threads sliding into personal attacks.
There are other avenues if you wish to discuss or report someone's behaviour. My email address is on the front of WCAG 2.1, you can find the group-chairs email on the WAI page, and my twitter handle is the same as my github one (DMs are open).
My next comment will be a draft response to the original question.
alastc
commented
2 years ago Draft response:
I request the working group outlines a process by which specialists can be identified who can vet the APCA’s effectiveness.
There are several strands to the contrast work, including:
Myndex works on all these strands and sometimes it is hard for people not spending as much time on it to differentiate between them.
Anything that makes it to the candidate recommendation stage of WCAG 3 will need to:
Given the specialism required for understanding the contrast formula and human perception, the AG chairs & Silver TF facilitators will also reach out to other people in the field who could review the work in detail.
Given the time required to review this type of thing, it would be best to carry on iterating on the WCAG 3 guidelines using the contrast formulas until they are in an understandable and stable state.
At that point it should also be possible to setup independent studies to verify that the new approach is a reasonable proxy to human vision (for people with disabilities), and better than the previous approach in WCAG 2.
The offer of help in performing research is very welcome, and when we get to a stable point we will reach-out to start that process.
ericwbailey
commented
2 years ago @alastc Thank you for the clarification and insight on both accounts, as well as your patience with this issue. Please definitely do keep me in mind for what assistance I can provide with research and independent review.
Myndex
commented
2 years ago One difficult lesson from the last few years with this organization is that of recognizing my personal communication deficits. Failing to "read the room" might be one way to put it. Another though is that sometimes I misunderstand someone's communication, be it literal or subtextual. At the same time, I've come to understand that I am not presenting information in a beneficial or understandable way, or I otherwise communicate in a way that results in misunderstandings or being misunderstood.
If I am misunderstanding others, and they are misunderstanding me, then we have a sustained chain reaction of misunderstanding, or an ouroboros in a form of feeding frenzy....
I use ouroboros intentionally and ironically in service of a point. Ouroboros as an ancient gnostic/alchemist symbol is stated as "expresses the unity of all things, material and spiritual, which never disappear but perpetually change form in an eternal cycle"(Brittanica) representing an eternal rebirth and renewal — a symbol not unlike the Phoenix.
But my impression of the symbology has always bent toward something more like "being one's own worst enemy; living in a vicious and unending cycle." In other words, my using it here is ironically an example of misunderstanding. So it seemed for a week in December — misunderstandings grew deeper and in all directions. If we stop to think about this momentarily, it becomes obvious that a cycle of misunderstanding can exponentially grow out of hand — more like a critical mass event than a self-eating snake.
A recent study that investigated autistic peer to peer communications has had me looking critically at my writing and other communications, and improving here is a work in progress.
My deepest concern is potential negative impact on this work. Work being done to correct the issues of readability on self-illuminated displays. To this end I did a careful review of things I had written, and discussed with others about "what went awry", where I was responding inappropriately, and how to get back on track.
I believe I see where some of the misunderstandings began, and I think in large part it was because I misunderstood some genuine and deeply held feelings by some in the accessibility community. I carelessly assumed that because APCA has been demonstrated as a major improvement for visual accessibility, that acceptance here was assured, and that I mainly needed to worry about getting the designer and developer community on board for acceptance — an important task as 86% of sites today fail WCAG 2 contrast.
But surprisingly, the design/development community is eager to move to APCA, with many already adopting APCA, at least insofar as the public beta is concerned. But in gathering support momentum from the design community, I apparently alienated the accessibility community — which was never my intent.
To this I can only apologize, deeply, for misunderstandings I had myself, and those I may have caused, but more importantly being part of the increasingly negative reactions and circular escalation that stemmed from these misunderstandings. I'm working on an article that dives deeper into this, and I would like to hear more regarding concerns that I may have overlooked, such as concerns about relations with clientele with a paradigm shift like this, or concerns about integrating a new or different workflow for conformance.
Clearly there is more work to be done, and it's a problem that needs a better understanding (at least by me) toward resolution. Again, my door is and has been open at the repo. I am as eager as ever to listen.
In an effort to clear up some of the rumors or other misunderstandings:
The APCA and the related work is a product of a research and development project following the issue thread I started in April of 2019 in the WCAG GitHub issues tracker. While much of the work has been conducted at our lab, it has also been a function of the Visual Contrast Subgroup (VCS) of Silver/W3, and the work has been under the review and oversight of the members of the VCS and Silver, one of whom is also a member of the US Access Board, as well as members of W3 and members of the public in general.
The issues and discussions tab has been open at the APCA repo since I moved the project to GitHub as open source, and I am proactive and responsive to comments and discussions there.
As it stands, there have been several independent reviews that include demonstrations of efficacy. Here are three that are in english:
Last year I suggested a wider-scope visual task force as there are many more visually related issues besides contrast that need examination in terms of guidance and standards within the W3. Subsequently, the VCS merged into a newly reformed LVTF, which had been dormant for a year or two.
There are threads in the LVTF discussions section including some evaluations and comparative review by other peers. But moreover, the efficacy of APCA is prima facie evidence. I have discussed it openly for years, and have written about it extensively, and always with clear visual examples. The evidence presented is open and easy to recreate and verify.
I've published so much regarding this topic, I can only say I do not understand the statements that I am not being open or that I am somehow "obfuscating" anything, as there is only the open evidence that I have been presenting for years. I don't know how these rumors get started, so I will just assume it falls back to me and being unable to present things as clearly as I think I am. But this has been a process; I have been focused on improving this.
Alastair had asked that I write up a one-page basic overview in plain language. I did this, using a tool that assesses readability as a guide to help keep things easy to understand..
I recently wrote these two articles covering key topics, again written for a wide audience:
My GitHub profile has a ton of resources on contrast and color.
And I have a link-tree with links to most of the key writings and resources.
I hope we can put misunderstandings by the wayside, and work for a better and more readable future.
Thank you for reading,
Andy
ericwbailey
commented
2 years ago But surprisingly, the design/development community is eager to move to APCA, with many already adopting APCA, at least insofar as the public beta is concerned. But in gathering support momentum from the design community, I apparently alienated the accessibility community — which was never my intent.
This is why I am so concerned.
Certain parts of the design community have locked onto the APCA as the new standard that can be used today, even though it is not signed off on for public use by Silver yet. It has gotten to the point where it is getting productized by the community.
The reason for the enthusiastic adoption is the design community wants to use more colors, but does not, broadly-speaking, care about the reason why those new color options exists. Given that the APCA is still in flight, this leads to one of three consequences:
Of these scenarios, 2 and 3 introduce the very real possibility of inaccessible color being utilized. As mentioned in my introductory post, I also feel these two scenarios have the very real potential to affect the public perception of the WCAG’s legitimacy. I hope my skepticism is unfounded, but I also need assurance that it is.
As it stands, there have been several independent reviews that include demonstrations of efficacy. Here are three that are in english: […] There are threads in the LVTF discussions section including some evaluations and comparative review by other peers. But moreover, the efficacy of APCA is prima facie evidence.
I also disagree about the assertion that the efficacy of APCA is prima facie, with concerns as to this previously outlined. The demonstrations that you link to discuss people's experiences working with the APCA algorithm's output, and not the algorithm itself—positive reviews of a faulty product does not mean the product ceases to be faulty.
This is why I would like to re-stress that feedback from an accredited, published researcher on work in flight would be very welcome.
Myndex
commented
2 years ago Hello @ericwbailey
This is why I am so concerned.
I am concerned because I believe based on your statements that your concern results from some misunderstandings. I recognize that I myself may be to blame for some of these misunderstandings. Let's see if we can fix this, okay?
Certain parts of the design community have locked onto the APCA as the new standard that can be used today
Do you mean the design community that does not bother to use WCAG 2 contrast at all? That design community? The design community wherein 86% of websites fail contrast today? That design community?
Can I rephrase your concern as: instead of using nothing (because so many refuse to use WCAG 2 unless forced to), a handful are eager to use APCA, a tool that has been in development for nearly three years, with the oversight and feedback of key members of the accessibility community, before being rolled out for a more public beta period, to further prove efficacy?
Part of what I am working on is not only to cure the accessibility failures of WCAG 2 contrast math, but to create an environment with a high level of adoption of the technology. It does no good to have a guideline no one is willing to use.
even though it is not signed off on for public use by Silver yet.
The first public working drafts have consensus for the purpose of the public beta period. Testing is proceeding with copious warnings and disclaimers that the APCA is in public beta and is not released code and is not affirmatively normative yet.
It has gotten to the point where it is getting productized by the community.
That specific tool is not using official APCA code, and I have been in contact with those developers to redact that plug in or bring it into compliance. See my posts on that page.
I am actually glad you brought this up, because that tool was created using code that should not be public facing, and it is not APCA. In other words, you are citing a counterfeit APCA instantiation that is not approved for any use, not even for beta use.
This is a problem internal to W3 GitHub Silver, and has nothing to do with APCA. And this is despite my efforts including three pull requests and an issue post to have that link corrected so that people would not download it and use it. One of my tasks has been tracking down developers with incorrect implementations, and this is one of the key goals of the public beta.
The reason for the enthusiastic adoption is the design community wants to use more colors, but does not, broadly-speaking, care about the reason why those new color options exists. Given that the APCA is still in flight, this leads to one of three consequences:
Eric, I've seen some of your posts like this, projecting animosity toward the design and development community, so clearly there are some strong feelings here. I would like to explore this with you at some point, as I believe there is a growing divide and misunderstanding in this area.
If I gained anything from the ruckus of last month, it's that there is a far greater divide and misunderstanding than I had ever imagined. Mea culpa in any part I had in that divide of misunderstanding, and I am looking to cure that proactively. But of greatest importance:
And everyone that has been involved here since 2019 is aware of the actual facts of how this plays out. Also false is the rumor that "APCA makes you inaccessible" — also patently false. I will address both of these now.
Chart comparing Lc60, which requires larger fonts under APCA:
Lc60 rejects many more colors than WCAG2 3:1
Please keep in mind that, in accordance with human perception (See Barten, Stevens etc), and of particular importance regarding impairments, APCA places much more emphasis on appropriate font sizes and font weights. This relates to something called "spatial frequency" and is a cornerstone of the modern understanding of human visual contrast perception.
1. My concerns are not founded and current design work using the APCA is retroactively verified upon 3.0's release. If this is the case I will use the APCA gleefully.
I'm having a hard time parsing this sentence. What additional verification are you looking for or expecting? What is it that you think about APCA that is "outside current standards"?
2. My concerns are founded and the APCA needs to be updated. Existing design work and tooling needs to be updated to reflect this, but will be hard or impossible to know who used what when. 3. My concerns are founded and a new algorithm needs to be introduced. In this scenario, the APCA has the potential to become like the Document Outline algorithm. Something that digital product makers who aren't extremely dialed into the accessibility space may mistakenly use due to preexisting hype.
Concerns 2 and 3 are without foundation, and here is why, with a breakdown of what I think you are saying, and certainly if I misunderstand you, please elaborate.
APCA is published as an npm package. Toolmakers are encouraged to use the npm so that updates are seamless.
Nevertheless, the APCA as a method for determining perceptually accurate luminance contrast (perceptual lightness contrast) is itself solid and traces to much existing visual perception science and color appearance models, in particular R-Lab, CIECAM02, CIELUV and CIELAB, and the R.W.Hunt model, and the work of the work of Barten, Stevens, Fairchild, Hunt, Luo, CIE, etc. among others.
The validation of the guidelines for readability are based largely on the work of Bailey/Lovie-Kitchen and Legge. I provide some links below that covers this in greater detail. These are listed in the various bibliographies.
If anything is to change, it is conceivable that the "levels" and the conformance points could, that is a function of Silver, not APCA.
One of the advantages of APCA is that it is designed to be both adjustable and extensible, this is a critically important feature as there are new color spaces coming out for CSS4 and CSS5 including those with very wide gamuts and high dynamic range, both of which demand a perceptually accurate contrast model, and both of which have significant implications for accessibility.
At the moment though, I can tell you that the conformance points we are presently using exceed those of WCAG 2 as I showed above, and particularly in the key zone of body-text. APCA guidelines are more in line with the ISO standards that recommend 10:1 for body text for instance.
Of these scenarios, 2 and 3 introduce the very real possibility of inaccessible color being utilized.
No, there is nothing, and no evidence of any kind, to support this assertion, as I have addressed above and I will address further below.
As mentioned in my introductory post, I also feel these two scenarios have the very real potential to affect the public perception of the WCAG’s legitimacy. I hope my skepticism is unfounded, but I also need assurance that it is.
WCAG 2 contrast math is demonstratively incorrect over an unfortunately large area of the range, and I have suggested that those problems do in fact negatively affect the perception of WCAG_2 as a whole.
This valid concern is why substantial time, effort, and due diligence have been the mantra of the Visual Contrast subgroup: to make a guideline that is unimpeachably robust and stable, even in the face of emerging technologies.
I clearly need to do a better job of demonstrating or explaining this.
Under APCA, body text is preferred to be at Lc 90, under WCAG2 that is a range of 8.9 : 1 to 16.5 : 1, this "range" is because under WCAG2, the contrast value is over-stated by a factor of about 2 when one of the colors is black vs when one of the colors is white. (4a/ff and e2/00) This is due to the fundamentally non-perceptual math method employed.
(4a/ff and e2/00) is the range that equals Lc 90.(6e/ff and ca/00) is the range that equals Lc 75.(8e/ff and ad/00) is the range that equals Lc 60.You can verify all of this yourself using the hex values listed (txt/bg) and various online calculators. I used WebAim for WCAG and the public APCA site for the Lc values.
WCAG 2 contrast ignores a lot of things like disabled controls, corporate logo images, and placeholder text, which should have some minimums defined. It's not practical with WCAG2 math due to the high level of perceptual inaccuracy. Because APCA is perceptually uniform, it's possible to make reasonable minimum standards for things like placeholder text or a disabled control, depending on certain contexts, but which do not have guidelines now.
APCA is the product of nearly three years of development and testing, but until recently you have not been involved in to any degree that I am aware of, and as a result I don't believe you have all the facts. The invitation is open, and still stands. The discussion area is open for all comments and concerns.
What concerns me Eric, is some of the statements/assertions you make seem to echo certain unfounded rumors that have formed, as they do on the internet — I am trying to demonstrate those rumors have no veracity. But I also want to ensure I am addressing any concerns of your own.
I hesitate to point you to the beginning thread 695 because it is nearly three years old, but it does show the early research and due diligence. It is issue 695 in the WCAG GitHub. Otherwise, it's useful to point out that this is a project in process, and the location of assets has moved several times.
Why APCA? A brief overview of WCAG_2 contrast issues and how APCA solves them.
What's Red & Black & Also Not Read? Do the WCAG 2 Contrast Guidelines Help Color Vision Issues?
A Contrast of Errors A look into the history of the WCAG 2 contrast guidelines, and a discussion of the proposed replacement, the APCA (Accessible Perceptual Contrast Algorithm).
But moreover, the efficacy of APCA is prima facie evidence. I also disagree about the assertion that the efficacy of APCA is prima facie, with concerns as to this previously outlined.
The term prima facie means true on the face of it, or visibly true, until refuted. I have demonstrated that through half a dozen articles and countless posts with copious visual examples. The math is not a secret. It's there out in the open. I have discussed the why and how at length. But I seem to be having difficulties getting this across. If there is a problem I would love to hear about it, which is in large part the purpose of the current public beta.
The demonstrations that you link to discuss people's experiences working with the APCA algorithm's output, and not the algorithm itself.
Roger Attill and Lisa Charlotte Muth both got into the math, Roger went so far as to write his own software analyzing it. I encourage you to read Roger's article at the very least.
—positive reviews of a faulty product does not mean the product ceases to be faulty
I don't know how to respond to comments like this, as it is a strawman, and I receive it as nothing less than derisive. It implies something that is not only untrue, but for which there is no evidence.
If you have an actual problem with APCA, if there is an actual flaw or fault, please report it, and it will be addressed. But assertions or implications that it is "faulty" with no evidentiary basis are specifically derisive and non-productive.
This is why I would like to re-stress that feedback from an accredited, published researcher on work in flight would be very welcome.
Again please see the link under the LVTF I postedas those comments are from an accredited, published researcher. I only link this one as it is a public link.
Also just FYI, peer review boards are anonymous, and the purpose is related to publishing good science in a journal, not defining efficacy.
That said, several years ago (fall 2019) I proposed that we (AGWG) form an independent review board of some notable vision researchers that are particularly well versed in these specific areas, though there is no funding, and they are normally paid consultants. I have not reached out to any of the potential candidates yet, as I am still conducting studies, which I want to spend more time with before "defending my thesis" as it were, not certain if they are interested in this as a pro-bono contribution.
You at one point mentioned the availability of some funding. I'd like to suggest that a good use of said funds would be toward forming such a board.
Upon completion of some current and pending studies, I'm expecting to be ready for an exhaustive review such as this late this year (fall) or very early 2023.
Bridge PCA is completely backwards compatible to WCAG 2, but uses APCA technology. If you pass Bridge PCA, you automatically pass WCAG 2 contrast.
APCA W3 is available as an npm package, and is integrated in a dozen or more tools WITH WCAG 2 contrast at the same time, so that comparative evaluations can be made.
There is more than contrast going on, but this is the part that is now in public beta for further evaluation. That some rumors have emerged I suppose is only inevitable. But the APCA repo has been the place to discuss anything and get to the root any any actionable issues, as well as general questions and comments.
Thank you for reading.
bruce-usab
commented
2 years ago As it stands, there have been several independent reviews that include demonstrations of efficacy. Here are three that are in english...
Those were all very well written, thanks @Myndex for sharing!
ericwbailey
commented
2 years ago Do you mean the design community that does not bother to use WCAG 2 contrast at all? That design community? The design community wherein 86% of websites fail contrast today? That design community?
This line of conversation is not the scope of my request. I am concerned about the APCA efficacy, not the design community's use of the current WCAG color contrast algorithm.
I do, however, agree that an algorithm that is more permissive in its color offerings will help with adoption. Again, this is why I am so concerned that the Silver WG can update and promote its updated color contrast algorithm with confidence.
WCAG 2 contrast math is demonstratively incorrect over an unfortunately large area of the range, and I have suggested that those problems do in fact negatively affect the perception of WCAG_2 as a whole.
Again, I am aware of the current color contrast algorithm is insufficient. This fact does not affect the efficacy of APCA.
Eric, I've seen some of your posts like this, projecting animosity toward the design and development community, so clearly there are some strong feelings here. I would like to explore this with you at some point, as I believe there is a growing divide and misunderstanding in this area.
If offsite behavior is permissible again, I would like to note that the other main contributing factor to raising this issue is my perception of your behavior of vilifying the current color contrast algorithm while simultaneously promoting the APCA.
We both agree that the design community views the current algorithm as limiting to the point of dismissal. The offering of the APCA as a replacement led to a flurry of designers using it and writing about it. I participate in multiple design communities and in under a week there were people in each community who mistakenly thought the APCA was something that was immediately available for use, and importantly, was W3C official.
The design community's adoption can be perceived as enthusiasm. That enthusiasm could also be misconstrued as efficacy, which in turn, could be used as influence for Silver's perception of its effectiveness.
I'm having a hard time parsing this sentence. What additional verification are you looking for or expecting? What is it that you think about APCA that is "outside current standards"?
"outside current standards" is your phrasing, not mine. I am saying that if the APCA can be independently verified and signed off on by an accredited individual working in this academic disipline, it is working as indented, and is a welcome update.
Concerns 2 and 3 are without foundation, and here is why, with a breakdown of what I think you are saying, and certainly if I misunderstand you, please elaborate.
They are with foundation if the algorithm is faulty.
WCAG 2 contrast ignores a lot of things like disabled controls, corporate logo images, and placeholder text, which should have some minimums defined. It's not practical with WCAG2 math due to the high level of perceptual inaccuracy. Because APCA is perceptually uniform, it's possible to make reasonable minimum standards for things like placeholder text or a disabled control, depending on certain contexts, but which do not have guidelines now.
I am aware of the current color contrast algorithm's scope, and this is nice to know, but it is also immaterial to the issue being discussed: the efficacy of APCA.
The term prima facie means true on the face of it, or visibly true, until refuted. I have demonstrated that through half a dozen articles and countless posts with copious visual examples.
Demonstration is one thing, verification is another. The bulk of the work backing up this statement is authored by you, but I am seeing next to no citations for your assertions. This communicates to me that while you may have done research in this field that there is the potential that the statements you have made may be in error. To me, this means there is the potential for the algorithm's output to be faulty.
a tool that has been in development for nearly three years, with the oversight and feedback of key members of the accessibility community, before being rolled out for a more public beta period, to further prove efficacy?
More exposure to these individuals and their contributions would be helpful for addressing my concerns. My perception of the work, and hence this issue, is based on what I have been able to research. What I was able to access in public channels led to my concerns.
In this area I am most worried about citations decoupled from assertions and self-referential citations from reference material I am no longer allowed to cite here.
Again please see the link under the LVTF I postedas those comments are from an accredited, published researcher. I only link this one as it is a public link.
I wish more of this would be public, but I also understand the process isn't generally built that way. Which is why I'm so concerned, as the public-facing pieces I see of it don't seem to add up to me.
As it stands, there have been several independent reviews that include demonstrations of efficacy. Here are three that are in english:
These articles all discuss ergonomics of working with the APCA, as well as the history of the WCAG color contrast algorithm. They do not deal with the research, and resulting math that creates the algorithm.
Efficacy is if the results the APCA algorithm generates are legible by people experiencing environmental and biological low vision conditions. I was expecting the blog post content to be a sampling of these people discussing their perception of color combinations approved by APCA, not how people use the APCA to select color combinations.
I am extremely concerned you are intentionally or accidentally conflating blog posts about working with an algorithm's output as evidence that the algorithm itself is valid. I would also like to note that this observation feeds into the larger concerns I have addressed, which only further reinforces my skepticism.
To reiterate my statement in the original issue:
I am questioning if the methodology behind the APCA is accurate, effective, and reproducible.
The paper trail I followed led to the creation of skepticism about the algorithm that is set to replace the current WCAG color contrast algorithm—the individual parts made available to the public do not seem to add up. Skepticism is a healthy part of the scientific process, and resistance to it to me only furthers suspicion.
I am not saying the current WCAG color contrast algorithm is sufficient, and I agree that it needs to be updated. I am questioning if the W3C can, beyond shadow of all doubt, move forward with it in full confidence.
@alastc has mentioned there are plans "to setup independent studies to verify that the new approach is a reasonable proxy to human vision (for people with disabilities)", which is what this issue was asking for. We appear to be at loggerheads here, but I feel like it is worth keeping this thread open until then, as that methodology has yet to be outlined.
Myndex
commented
2 years ago @ericwbailey
Hi Eric,
I have provided all the links and resources I thought would be needed to explain the current status. This includes some articles and white papers written in plain language in the spirit of Feynman, and previously linked above.
As that hasn’t worked, I will try and encapsulate them as succinctly as possible here.
EFFICACY: This term would be suitable for a drug undergoing FDA trials, but it is not correct or relevant for this scenario.
APCA: The algorithm is strongly rooted in existing CAM science, in particular M.Fairchild's R-LAB (RIT), the R.Hunt model (Kodak), the work of M. Stone (NIST, PARC), and L. Ahrend (NASA), P.Barten, and not to mention the international standards CIELAB, CIELUV, and CIECAM02.. It is a matter of prima facie evidence that it is perceptually uniform for the given application of self-illuminated monitors. The SAPC site has interactive experiments that you can try yourself to see how the functions were shaped.
WCAG_3 Guidelines: The text size and weight guidelines are directly from the seminal research of Bailey/Lovie-Kitchin, as I discuss here:Myndex/SAPC-APCA#39 and elsewhere. Also incorporated and reviewed is the extensive work of G.Legge (psychophysics of reading) and A.Arditi, and others, see bibliography.
I would like to note that the other main contributing factor to raising this issue is my perception of your behavior of vilifying the current color contrast algorithm while simultaneously promoting the APCA
This statement re-enforces my perception that our differences come from the understanding of the science noted above.
I went on record three years ago that WCAG 2 contrast math is not fit for purpose. The first post was three years ago, starting thread 695 at GitHub WCAG (which is very useful background). That was the beginning of this project, and well before I had any intention of developing anything.
That thread includes answers to the majority of your questions/issues. It is quite long but worth a read.
I do not intend to be "vilifying WCAG 2 while promoting APCA". I’m sorry if that is how it comes across, that would be a communication error on my part. My intent is to provide relevant comparisons when the topic comes up.
It would be equally accurate for me to compare WCAG 2 contrast to any of the many other perceptually accurate visual appearance models, but that would only serve to confuse the issue, as those models are not specific to readability, and they are substantially more complex and therefore harder to understand. (See the P.Barten contrast model for example).
On the other hand, APCA is specifically tuned for readability, and we have a mass of comparative examples, therefore this is the most cogent and demonstrative means of presenting the facts.
They are with foundation if the algorithm is faulty.
The underlying foundation (the perceptual science) isn’t mine, it has been long established by the science referenced above. My part has been to research the existing science and formulate a replacement algorithm and associated tool based primarily on that science. And most especially as it pertains to readability. Unlike WCAG_2 though, APCA is not a simple a priori, as we have conducted empirical studies to validate & verify, model, and ensure the shape of the curves serves the intended purpose of readability on self-illuminated monitors. The studies and public beta testing programs continue, to further validate and verify the usefulness of the methods.
The bulk of the work backing up this statement is authored by you, I am seeing next to no citations for your assertions.
I have presented all of the information publicly and openly, including bibliographies and references to the primary sources. I realize there is a lot to go through, but it isn’t necessary that you do so if you trust that others will.
I’ve been told that as part of the W3C process, other color experts will be contacted to review the work, and nevertheless it is not, and will not be, based solely on my opinion. In fact, I proactively suggested an independent peer review panel in 2019.
In the meantime though, if you do want to understand the underlying science there is quite a lot of reading to do from the links provided here and in my other several posts in this thread.
I will continue to conduct research, and independently of me, others connected to the AG group will either conduct research or contact additional people who can. Again, it is not just my work or my opinions, others have been involved from the beginning, and that will continue to be the case. From the beginning in 2019, the project has been under the oversight and guidance of the AGWG with the Visual Contrast subgroup, which involved a number of key AGWG members as has been previously stated.
Best Regards,
Andrew Somers
W3/AGWG invited Expert
Myndex
commented
2 years ago I believe there is a fairly massive misunderstanding regarding the nature of peer review, and apparently I am not capable of describing the process & status in a post in a way that is understandable. I.e., I'm not getting the facts across.
In an effort to clarify, I've created a catalog of resources relating to APCA guidelines, so that everything is all in one place: https://git.myndex.com
To be clear, this is not to imply that there won't be further review and publications, but there seems to have been a disconnect from the materials and bibliographies I've made available, and people accessing them. I don't know how else to account for the disinformation that has been propagated on the subject, as everything has been open and available since the project started over three years ago. Additional testing and studies are continuing.
ericwbailey
commented
2 years ago Again, I am raising the issue here and not on your repositories because I want independent verification that the APCA does what it claims to do.
The reason I ask is because it will affect all visible content on the internet in an international, legally binding way.
This issue arose from my perception of the work thus far, as well as the behavior of its author.
To put it plainly:
(Chair's note - the original post has been modified to reduce personal aspects.)
Barring legal threats and harassment, each of these factors could be individually overlooked. Combined, they paint a picture where skepticism makes sense.
This issue is an egoless request. When I ask for efficacy, I am requesting that Silver’s team has plans to:
APCA feels like it is being treated as an inevitability due to the amount of work it involves, but effort does not necessarily equate to effectiveness. APCA is an option. If it cannot deliver on its promise, it would be irresponsible and harmful to codify it.
alastc
commented
2 years ago There is nothing new since the last time I commented, there will be independent research and there is plenty to do before any WCAG 3 guideline is ready to use in practice.
This is a sibling issue to https://github.com/w3c/silver/issues/272 and https://github.com/Myndex/SAPC-APCA/issues/35.
I want to frame this issue as someone who isn’t a member of the Silver working group, but who has a vested interest in its success.
I have been paying close attention to the SAPC-APCA (APCA for short), in that it addresses an important gap in the current WCAG criterion. As a graphic designer by trade, I think it is incredibly important to have this area of the WCAG updated to match the current technological landscape.
I also want to acknowledge that the stakes for APCA’s success are incredibly high, in that it affects both:
The ask
Claims of scientific legitimacy must be answered by the rigor of scientific process. I request the working group outlines a process by which specialists can be identified who can vet the APCA’s effectiveness.
I understand that visual contrast is incredibly complicated. It lives in the intersection of the physics of waveforms, the biology of the eye, and cultural considerations. I also understand that vision disability may further this complication.
I do not have the specialized knowledge required to comment on these three disciplines and how they relate to how the formulates of an algorithm. However, I believe that this should not be an impediment to request clarification of process.
In my mind, these factors further reinforce the need for constructive, critical analysis of the APCA. I am very worried about confirmation bias—input from automotive or military researchers would be a good starting point, as they both possess the expertise needed to review these areas.
EDIT: Removed links to published work.
An invited expert may be convinced of the validity of their work, but that does not necessarily mean their work is partially or completely valid.
I am questioning if the methodology behind the APCA is accurate, effective, and reproducible.
I am also not saying current color contrast calculation is effective, nor am I asking if other sections of the WCAG have, or will be peer reviewed—I hope that discussions about these two subjects can be left out of this issue so as to not derail it.
The gravity of this ask is also not lost on me. Candidly, improving color contrast calculation could be one of the most impactful changes for digital accessibility on the web. This is why I am so concerned about ensuring its success.