Awareness of web accessibility is slowly growing, which is good. I remember the time when almost nobody was knowing what accessibility is in the web world. Today’s day it is changes. We think more about accessibility and want to fix the problems. This can be done manually and automatically. Just to be clear – automation tests doesn’t cover all scenarios and probably never will do. However, it helps to catch the issues that sometimes our eyes can’t see.
The software development every day brings changes and because of that it is very hard to imagine that manual tests will be done every day as it would consume quite a lot of time. To speed up this process and reduce the amount of accessibility issues it is time to think about accessibility automation tests. This is one of the reason I’ve started working on ASLint – Accessibility Validation Tool.
Should I believe in the results from automation tests?
I think yes, but there are areas where the results may not be accurate. This is because writing automation tests for accessibility is very hard. For example, you see text with black color foreground and white background. However, automation test see it differently: black text on transparent background and it is not even obvious to detect the correct state. Things gets harder when the element has
position different than
position is set to
fixed then the element is drawed out of normal flow and therefore determine background is quite difficult.
Another tests that are hard to test, for example, is if plain language is used or if color alone is used to convey content. I think I can’t say it is impossible, but it is very hard as it would need to engage quite complex algorithms to analyze content and thing gets even more complicated when we think how many languages are in the world.
But… There are also tests that you can trust in 100%. Some of the tests are relatively easy to test, e.g. missing defined
form control or headings hierarchy.
Trust the test results, but do manual tests as well before any release.
But wait, there are a few tools out there yet
Yes, indeed and you should use them interchangeably as every tool evaluate a tests in a different way. Compare results and apply fixes. You can even help me and submit an issue or idea for ASLint when you find it. I’d very much appreciate it.
Also, some of the tools are working on server side, some of them on client side. I think the advantage of using tools on client side is that it parses the content that is already parsed by the browser. Yes, you can say: I can run the browser on server as well. True, but as the single page application model is now popular it is the best way to test accessibility in real time directly in the browser right in front of you. This is because you can test also some of your features that changes the content on-the-fly and reevaluate accessibility tests immediately.
They are always and will be. That’s why I’ve been creating a list of tests (Checklist) that can’t be automated (at least at the moment), but can help you to remind you to do a manual tests.
What is good?
The good part is that you can catch the issues quite fast. At least if you’ll fix the issues reported by the tool then you’ll be on a good path.