Acid3 browser test completed, available now
Ian Hickson, the Google employee tasked with creating the next generation of acid test, has completed his work, which is now available for public consumption at its new home, acidtests.org. Unlike the first acid test, which focused on the box model, and the second acid test, which covered a broad variety of basic HTML and CSS features, Acid3 covers 100 of the nooks and crannies of HTTP, HTML, CSS, ECMAScript, SVG and XML, all through the medium of DOM scripting, a critical requirement for any modern web application. Ian Hickson is also the primary author of the HTML5 specification, which started life as a spec. called ‘Web Apps 1.0’, and as such has lots of application‐related features such as client‐side storage and enhanced forms. Ian wrote 64 of the tests, with the remaining 36 being submitted by both browser vendors and interested web developers.
Work started on the new acid test almost as soon as the IE developer team posted notification that IE8 passes Acid2. As was widely criticised around the ’net recently, it was revealed Internet Explorer 8 would now only pass the test if the server was modified to output a special HTTP header. It is not known to css3.info at this time whether the header would be required for IE8 to achieve compliance in the new test.
Firefox 2 got to 50/100 with some strange graphics, Safari 3 (Windows) made it to 40/100; IE6 just sat there looking at me rather dumbly.
has it been officiallly released? i don’t see announcement in ian’s blog or any other places. i know ian was challenging several more tests about 2 weeks ago but didn’t know it was all set and done
Ian commented that it was done in an IRC chat room about 4 hours ago, I expect an official announcement to be made within 24 hours. The most recent WebKit nightly gets 64%, a significant improvement on the released Safari 3 (which gets 40% on the Mac side too).
This race should be interesting to watch :)
Just to add to current results posting…
Opera 9.25 gets 48/100.
Latest Opera 9.5 snapshot gets 60/100.
IE7 gets 13/100.
Strangely, loading the test in a background tab in Firefox 3 (2008012704) gives me 58, whereas loading in the foreground gives me 60.
Also, on random occasions, I get 61 instead.
Actually work started months before Microsoft announced IE8 passed Acid2. I just got more motivation to work on it after the announcement.
Right now the test is in review mode. It’ll probably stay that way for some time (weeks, if I have my way).
I’m not really sure how you came up with the 64/34 number, either. :-) It’s more like 90/10, though people helped throughout.
@Peter: I don’t know what you did you your IE 6 at “Staring Dumbly”, but at a default XP SP 2 with full non-IE 7 updates, I’m at 13/100
[...] Acid3 completed. 50/100 for Firefox 2. [...]
here are my results:
IE6 13/100 - quite poor
Safari for win 3.0.4 - 40/100 - I expected more from Safari honestly
Firefox 2.0.0.11 - 51/100 - thats the winner?
wonder how Firefox 3 will do this one?
Earlier today on Windows XP with Fx 3: 57
Now on Windows Vista with Fx 3: 59
I wonder how IE8 would do in ‘IE8 mode’.
Strange to hear of these seemingly erratic results… how will we know how the browsers really fare?
Regarding the 64/36 number: Tests number 65 and onwards are commented in a way that lead me to believe they were sourced from a competition for contributions. I see now that not all are attributed to third parties. My apologies.
Seemingly erratic indeed :/ I get slightly different scores from other people on the same browser, and sometimes a different score when running the test repeatedly.
What could be the reason, some kind of AJAX race condition ? Some environement variable changing even though you’re supposed to be at default settings ? Hope this gets sorted out, we can’t afford randomness in the test. I guess now’s the time to review the test’s code…
The HTML Validator addon for mozilla at: http://users.skynet.be/mgueury/mozilla/
Shows a number of warnings for the page (from HTML tidy I believe).
First you are missing the type attribute for the stylesheet - the comment makes me think this was deliberate?
The rest of the errors look like they are from the backslash in the src for one of the script tags. Is that really illegal?? Or does the validator have a bug?
OS X - WebKit nightly (5523.10.6) - Safari Version 3.0.4
65/100
Using the latest nightly build of Firefox 3 (Minefield) i get 57/100 over four tries.
[...] CSSInfo No Tags [...]
Yes, Firefox 3 Beta 2 also got 57 over four tries, from 54 on the first try. On try 5, it dropped to 56. *shrug*
Why do new tries give different results!?
Seems like the problem have been fixed now?
Using Fx3 (2008012904); over 25 tries I got 61% 23 times, and 60% 2 times at the 4th and 18th try.
Also Opera 9.24 now gives me 48% instead of 50% like it was before, IE7 still gives me 13%.
Ariel: I suppose ACID3 works the in similar way as ACID2 does. Among other things, ACID2 contains deliberate errors to find out how the browsers behave in that situations.
Opera 9.5 build 9755 on WinXP (most recent, I believe) gives 60/100, agreeing with Lars.
I can’t test Opera 9.24 since I don’t have a default install :) FWII, I get 46/100, not 48, but I imagine that part of that is the fact that I have UserJS that modifies the DOM.
Konqueror 3.5.8 on latest Ubuntu Linux 7.10 loads the page, then crashes after a split second (attempted several times). I was curious about Konqueror, because it is the only browser I’ve tried that passes all selectors at CSS3.info’s CSS Selector Test.
I wonder how Ian Hickson could know how it should look when test is passed…
That’s ok
Randomness in the test results do not necessarily mean that the test itself is to blame, but rather that the test is exposing problems in the implementations.
[...] Hixie’s Natural Log: Come up with the best test for Acid3 Edit: Strike that, Acid3 has been completed. [...]
[...] Ian Hickson завершил работу над созданием следующей версии acid-теста (Acid 3) [...]
Browsers were probably getting different results yesterday because I was still fixing problems in the test throughout the day.
[...] parecer, el test ACID3 esta listo. Se publico hoy que Ian Hickson, ya tiene listo el test ACID3. Este test prueba capacidades que aun no están [...]
[...] CSS3.info - Acid3 browser test completed, available now [...]
Opera 9.50.9500.0 gets to 40 then crashes.
But they are bloks, Firefox makes only rows of it.
xsys, there isn’t much magic in this reference implementation: most of the test is easily reproducible using widely-supported (and valid) CSS2. For example, the text-shadow test is very simple, with no blur radius.
I guess they had tougher problems guaranteeing Acid2’s reference implementation.
[...] CSS3.INFO Tags: acid2, acid3, browser, Internet, standards, test No Comments, Comment or [...]
@Ian -
While an above commenter may have been joking, I was curious as to how you know how a page should actually render your test if the leading browsers all fail (pretty miserably). I know that you conceptually know what each element is supposed to do, but how do you review your test with the shortcomings of browser engines?
There’s a couple of techniques. One is to start with something that is simple, and easy to determine the correct handling of, and then incrementally make it more complex, updating the reference rendering at the same time. The other is to do something really complicated, and then make a separate version of the test that works around all the bugs in one browser, and then take a screenshot of that browser. I use a mix of both.
It also helps to have a really good grasp of what the standards require. :-)
this is the strictest test available. No browser passed it .:D
I’m very glad this includes SVG tests, as SVG needs and deserves more attention. That also means IE will end up having to support SVG if they want to brag about passing the Acid3 test, like they did for Acid2.
[...] Buradaki haberden öğrendiğimize göre web tarayıcıların standartlara uyumluluğunu denetleyen acid testlerinin üçüncüsü hazır.acid testleri, tarayıcıların standartlara uyumluluğunu test etmekte önemli bir kriter sayılıyor. Örneğin Microsoft, Internet Explorer 8′in standartlara ne kadar uygun olacağını duyurmak için tarayıcısının acid 2 testini geçtiğini müjdelemişti. [...]
Opera snapshot from today got 64/100, wow!
Webkit nightly from today got 69/100, wow ;)
[...] Acid3 browser test available from new Web site, acidtests.org. Acid3 addresses HTML, CSS, HTTP, ECMAScript, XML, and SVG through DOM scripting. [...]
This is what I found:
Opera Stable (9.25): 46/100
Opera 9.5b1: 59/100
Firefox Stable (2.0.0.11): 50/100
Firefox 3.0b2pre: 54/100
Konqueror Stable (3.5.8): Seg faults
Konqueror 4.0.1 (Using KDE 4.0.1): 49/100
Take the results as you wish.
Latest Webkit nightly on Windows gets 72/100 already! Woot!
[...] Acid3 browser test available from new Web site, acidtests.org. Acid3 addresses HTML, CSS, HTTP, ECMAScript, XML, and SVG through DOM scripting. [...]
WebKit r30109 nightly on Mac OS X gets 78/100 already! That about 20 more than it had when Acid3 became available.
I get 80/100 on WebKit r30118 nightly on Windows.
Nowhere near as impressive as Webkit’s result, but Firefox got up to 61/100 with today’s nightly. :p
I have a feeling this race will be shorter than the Acid2 one…
The IE team must be pretty annoyed… “Man, we just finished work on acid2 and now this!! Give us a break!”
Gotta love how the other browsers just speed ahead and … *fix* things.
WebKit up to 82/100
[...] The Acid3 Test - Pgina do Acid3, teste destinado a medir o nvel de compreendimento do CSS3 por parte dos browsers. Vale ressaltar que os browsers atuais ainda esto chegando nos 100% do Acid2… Resta esperar para ver quanto tempo demorar at que atinjam essa marca no Acid3. [...]
WebKit up to 85/100.
Opera 9.5 latest build - 65/100 with its 1% market share
ie6 12/100 with its 45% market share
humm..
I don’t know what to think of WebKits quick bugfixing for Acid3. I don’t believe that they made the old WebKit pass Acid2 but when I saw how Acid3 looked at first glance it seems the do not really implement their properties with quality.
I may be completely wrong here, but I think implemented properties and functions should work in everyday situations as well as in the test, not just only in the test.
WebKit now at 86/100
And Daniel, they do fix things properly. You can watch them work on the bugtracker (search for Acid3). They’re fully aware that just fixing it so Acid3 works defies the point of it and they actively work with the developer of Acid3 to ensure that they’re meeting the spirit of the tests.
New webkit30595 got 87/100
The few errors left are
Failed 13 tests.
Test 2: reached expectation 13 when expecting expectation 12
Test 11: when inserting <a> into Document with another child: wrong exception raised; code = 1
Test 13: collapsed is wrong after deletion
Test 23: no exception for createElementNS(’null’, ‘:div’)
Test 25: failed to raise exception
Test 26: e1 - parent element doesn’t exist after looping
Test 27: e1 - parent element doesn’t exist after waiting
Test 70: UTF-8 encoded XML document with invalid character did not have a well-formedness error
Test 73: expected: 10, got: 1 - click event handler called the wrong number of times
Test 75: Value undefined (result of expression anim.beginElement) is not object.
Test 76: expected: 0, got: 100 - Incorrect animVal value after svg animation.
Test 77: expected: 42, got: 65 - getSubStringLength #1 failed.
Test 79: expected: 34, got: 33 - SVGSVGTextElement.getNumberOfChars() incorrect
Elapsed time: 1.22s
I think they’re fixing one at a time :D
just tested now:
Firefox3beta4pre(latest) gets 66/100 and if I tried in background it reaches 65/100 and all ran under Debian GNU/LINUX Sid.
WebKit r30790 now at 90/100
WebKit r30790 on OSX gives me 89/100 the first try and 88/100 every subsequent refresh.
89/100 fails tests: 2, 11, 13, 23, 25, 70, 73, 75, 76, 79 and 80
90/100 fails tests: 2, 11, 13, 23, 25, 69, 70, 73, 75, 76, 78 and 79
Oh, just tried again and got 90/100 - test 80 passed :)
Ed, you mean
90/100 fails tests: 2, 11, 13, 23, 25, 69, 70, 73, 75, 76, 78 and 79 = 88/100 because 12 tests failed means 100 - 12 and that’s still 88 :)
Yeah, got 90 on Webkit r30790 too :)
btw: test 69 means you refreshed too many times: “69, took 41 attempts”
Yeah, Rifton, was a typo. Didn’t notice it until after I’d already made a second post and thought it’d be a bit excessive to do a third :)
lol, ed, you just did one :P
No seriously… I’m looking for the release notes of webkit… Where can I find them… I’ve looked everywhere…(I think)
http://cia.vc/stats/project/WebKit might be what you want. Not release notes, but a commit log…
I have to say WebKit has really grown on me and I’m quite keen on it now. They’re very fast at developing things, seem open to input and quite a small team.
hi all !
I have the results for IE 8!
it,s 17% sometimes 16%
The beta 1 is Available !!!
http://www.microsoft.com/windows/products/winfamily/ie/ie8/readiness/default.htm
look at the Acid 2 test, it’s really cool
WebKit r30768 on Windows still stuck at 85/100 (and has been at that for the past few nightlies). This one didn’t even launch out-of-the-box without some manual tweaking to make it work (the FindSafari.exe application seems corrupt).
[...] Acid3 test was recently released, and I ran it on a few web browsers I had installed here. Here is how they [...]
Now getting 87/100 on WebKit r30865 on Windows (had to use FindSafari.exe from r30628 to make it work though).
WebKit r30881 on Windows: Safari reaches a score of 90/100, but it crashes constantly after reaching that score, also on other websites.
WebKit r30881 on Windows: 87/100, no crash.
I can’t seem to get 90 points no matter what I try.
… time passes …
Ok, I ran Apple Software Update and it gave me an update to Safari 3.0.4 beta. When I installed that and retried Acid3 in WebKit r30881, I now got 90/100. No crash either.
How U can get so much with this Webkits? Do You individually change them somehow? I downloaded Safari 3.1.525.12 and newest webkit from Apple site. I have 76/100! WebKit 30885
You are getting 76/100 because you are still using the WebKit version that is built into Safari 3.1.
You need to run nightly build of WebKit itself. It will still look and function like Safari 3.1 as it pulls the front-end code for it (layman version) but the engine will be using, in your case, WebKit r30885. You will now get 90/100.
PS: You know you are using WebKit and not Safari as the compass icon will have a gold rim instead of silver.
That is what I was suspicing. But how to use that WebKit? I put it in the plugins directory and don’t see any change
I’m sorry for my english.
You’re English is fine.
It’s not a plugin. It’s an application. Stick it in /Applications or on the Desktop or run it from the mounted Disk Image. Just just double click it like any other application.
It’s an understandable mistake.
If you’re using a Mac, you can launch it straight out of the disk image. Haven’t tried Windows, but someone mentioned using FindSafari.exe…
Your, not you’re. Apparently my English is not fine. :D
I’m using Windows XP.
I read that this is plugin. Is this application really? It name is “WebKit-SVN-r30885.DMG”. When I double-click it starts to open Safari browser. It’s certainly not application, but I maybe in wrong.
I can’t understand anything about FindSafari.exe I tried Google but all I got is something about Java debugging and some not understandable for me crap (I suspect it’s ’bout Mac).
Please help.
P.S. How to change Safari language? My native help is very crap (unusable)
Sounds like you’ve downloaded the Mac version rather than the windows version.
Visit http://nightly.webkit.org/ and download the build under the Windows heading, currently r30881.
Based on their site, you should download that zip file, extract it and double click the file run-nightly-webkit.cmd. You must have the public Safari installed already (from apple.com/safari).
If you can get on IRC, you can chat with the developers in #webkit on irc.freenode.net and I’m sure they’ll be happy to help.
Lol. Sincerely I downloaded Mac Os version. Thank You.
Hi, I’m getting an error on Windows XP x64 when trying to start the nightly r30881 by clicking the run-nightly-webkit.cmd, like it is recommended:
The procedure entry point ?fastMalloc@WTF@@YAPAXI@Z could not be located in the dynamic link library WebKit.dll
I only can hit “ok” then, and nothing happens.
I had to copy the whole content of the archive to a nightly folder, and create a start script:
@echo off
set PATH=%CD%;C:\Program Files (x86)\Safari\;%PATH%
Safari.exe /customWebKit
This one (run-webkit-nightly2.cmd) didn’t work:
@echo off
mkdir 2>NUL “%TMP%\WebKitNightly\Safari.resources”
xcopy /y /i /d “C:\Program Files (x86)\Safari\Safari.exe” “%TMP%\WebKitNightly”
xcopy /y /i /d /e “C:\Program Files (x86)\Safari\Safari.resources” “%TMP%\WebKitNightly\Safari.resources”
dir “%TMP%\WebKitNightly”
set PATH=%CD%;C:\Program Files (x86)\Safari\;%PATH%
echo %CD%
“%TMP%\WebKitNightly\Safari.exe” /customWebKit
Also the run-webkit-nightly.cmd does not work.
But fortunately now, after I clicked run-nightly-webkit.cmd again, my prepared nightly doesn’t crash anymore.
I get 90/100 on acid3
91/100 on Webkit r31078 !!
Almost there :) Its getting smoother :)
93/100 on WebKit.
Bear in mind that the 93/100 for WebKit is due to one of the Acid3 tests being changed.
http://webkit.org/blog/161/webkit-hits-93100-in-acid3/
Getting 95/100 on webkit for windows on server 2003 as of sat the 22 of march. Looking forward to seeing it reach 100 here soon.
WebKit at 98/100.
http://webkit.org/blog/170/webkit-hits-98100-on-acid3/
Opera claims 100/100 now (not public yet).
http://my.opera.com/desktopteam/blog/2008/03/26/opera-and-the-acid3-test
Opera claims that, but where can we verify that?
They want to release that snapshot within one week, or so.
Maybe webkit will be already there then?
Opera is missing a space after the comma in: ‘test,a’.
So they are not quite there yet, but very close.
[...] o lanamento da terceira verso do teste, muitos browsers conseguiram boas pontuaes, como o Safari, tendo a nota mais alta de [...]
No unicode support in this wiki? You should avoid AddDefaultCharset http://httpd.apache.org/docs/2.0/mod/core.html#AddDefaultCharset
and set it by the meta-tag to utf-8 ;-)
WebKit/Safari now at 100/100. A fixed bug in the tests has reduced Opera to 99/100. Still minor visual issues to fix for both.
[...] It’s enlightening to watch how the browsers roll. Just yesterday the bottom two hit 100% on the Acid3 test. I think Safari did it first and then Opera - although Safari’s is available in the nightly webkit and opera’s is a private build. This is fantastic news - mad props to both teams for making this happen so quickly. [...]
Safari 3.1.1 (5525.18)
got 75/100!
Without webkit.
Last week, Webkit passed one speedtest, only managed to make it once… no more :(
Ah, got a proof of it :)
http://rifton.free.fr/Webkit.png
but what’s this happening sometimes
http://rifton.free.fr/Webkit-80.png
ae… quero ver quem combate o OPERA agora
Opera 9.5 Build 10063
got 83/100 !!! That’s Great navegation …
;)
for you - http://www.opera.com/download/
I hope that will be the end of this crazy stuff. this untolerable for the small clients. anyway updating means something getting better.
WebKit passed Acid3 fully for the first time on Friday. So it’s 100 subtests of 100 passed now. Opera 9.60 Beta scored 85.