-
201007 Jan
As you may (or may not) know, I’m an Invited Expert on the CSS Working Group at W3C. Mostly I talk about specs. But today, I’m going to talk about testing.
W3C is working on test suites for the CSS specs, and I wanted us to have more web authors involved. Many of you have been frustrated with the inconsistent levels of CSS support across browsers, and I believe collaborative testing is one of the major ways we can improve the situation.
Testing at W3C involves several efforts. The most obvious is creating the tests themselves. W3C is collecting tests to create a test suite for each spec it releases. Right now we have several browser vendors, along with some individual volunteer contributors, involved in writing tests. Because the tests are hosted at W3C, all the browser developers have access to them and will be keeping track of which tests they pass and fail.
I’d like to include more contributions from the web authoring community: you can submit tests demonstrating bugs that are frustrating you, or tests for new features you’d really like to see implemented consistently. Having your tests in the W3C test suite encourages browser developers to fix those bugs and get those new feature details right the first time.
Gérard Talbot is right now collecting tests from web authors for CSS2.1. If you have tests you’re interested in submitting, please contact him and support his efforts to include more tests from the web authoring community. More information on writing and submitting CSS2.1 and CSS3 tests can be found on the CSSWG wiki. You’ll want to update your tests to match the guidelines there before submission.
The other major area I’d like more web authors involved is reporting test results. W3C wants everyone to be able submit their test results, so that the test result data reflects the full variety of browser and system configurations out there. We also want to make the result reports useful to you. Reported well, that data could tell you which features are buggy and which ones are reliable. You could even tell under exactly which conditions they’re buggy or reliable.
Right now we have a rudimentary prototype for collecting results and reporting them, but it’s not the most straightforward thing to set up, and its output is not very useful. I’d love to have some web designers get involved and help us make a system that is easy to use and design reports that make the data it collects useful to everyone.
If you’re interested in reading more, I’ve written a couple posts on what testing at W3C means and why it’s important to the Web and on what’s missing to make it a more useful tool for authors and implementors alike.
Let us know what you think and how you’d like to help out! :)
You can skip to the end and leave a response.
-
Comments
-
01.
Great idea. I was unaware that Gérard has taken it upon himself to review test cases from designers in this way – well done, that man! However, what’s stopping authors from simply submitting tests using [email protected] as is done at the moment?
-
02.
Emil Ivanov says:Comment » January 7th, 2010 at 6:15 pm
Great, this should be permanent. I dont have time to submit bugs and testcases to all browser vendors, but submitting to one place where all can look is really good.
-
03.
However, what’s stopping authors from simply submitting tests using [email protected] as is done at the moment?
Irrational fears of mailing lists? :) For people who plan to submit many tests, or who want to get more involved, I definitely recommend joining public-css-testsuite and learning to do it yourself. (Like you, James!) But if someone has only a few tests to submit, it’s probably better for Gérard to shepherd them; that way they don’t get lost or give up halfway through the review process.
-
04.
Emil Ivanov says:Comment » January 7th, 2010 at 11:18 pm
Well this public mailing list should be easier to discover, maybe sounds weird but it should be promoted in some way. The first thing when I heard about tests on W3.org was when Microsoft contributed tests with development of IE8. Never thought that I could contribute testcase or ask something and where. Tests like ACIDs and the selector test on css3.info are popular. Something in that direction have to be done.
-
05.
[…] article on CSS3-Info, Join the CSS Quality Assurance Team, mentions collaborative testing that gathers test data from various peoples’ browsers to […]
-
06.
“Well this public mailing list should be easier to discover, maybe sounds weird but it should be promoted in some way”
Agreed, as an interim solution until a more author-centric submission system has been established.
“Tests like ACIDs and the selector test on css3.info are popular. Something in that direction have to be done.”
Popularity musn’t be a factor to determine how suitable a testsuite is. I’d recommend you read one of Elika’s articles to understand why Acid tests are far from adequate, and why the selectors test here at CSS3.info is unsuitable too.
@Elika. Since several CSS3 modules have reached CR recently, could you give us a quick update as to where the Working Group is at with establishing a testsuite for each CR module? Does the Working Group require help in the same way they do currently with CSS21?
ASIDE: it might be an idea to ascii-obfuscate Gérard’s email address in your post :)
-
07.
@James Except for css3-namespace, new CR modules all need test suites. :) And yes, contributions are welcome for the same reasons. I think the idea for the css3 test suites is to use a combination of reftests and human-verified tests–reftests where possible, human-verified where not. (Reftests are two pages that are supposed to look exactly the same. The advantage is that such tests can be automated via screenshot comparisons.) Mozilla has some reftests that could be converted for css3-background, but we probably need more. ASIDE: It’s already crawlable on the public-css-testsuite archives.
-
08.
Gaurav says:Comment » January 9th, 2010 at 7:43 am
I think the ACID tests and the CSS selector test suite are popular because they can be easily run by anyone.
Can anyone give me an example of a good, useful test suite that is also dead simple for a web user to figure out how to run?
-
09.
“Can anyone give me an example of a good, useful test suite that is also dead simple for a web user to figure out how to run?”
http://www.w3.org/Style/CSS/Test/CSS2.1/current/ :)
All tests have an intuitive pass/fail criterion which can be understood by anyone, regardless of whether or not they are familiar with the specification.
-
10.
Hello all,
I quickly have created a basic webpage for people wishing to submit testcases to the CSS 2.1 test suite:
regards, Gérard Talbot
-
11.
can anyone send me a sample test suite, so that it will helpful for me to send bugs
-
12.
@nitin Not sure what you mean. Most info about the test suite is available on the CSSWG Wiki, if that helps.
-
13.
This is a good post, but I was wondering how do I suscribe to the RSS feed?
-
14.
Hey fantasai,
Thanks for this great post.I want to ask a question regarding this-
How i will join the CSS Quality Assurance Team? -
15.
How i will join the CSS Quality Assurance Team?
-
16.
-
17.
can i get daily update regarding css3, html5 and mobile apps
-
01.
