The next generation image format for the web is not JPEG XL?
I am late to the party here, but Google looks like they casted their vote for what they want as the next, widely adopted image format for the web. It is not JPEG XL.
Format wars
If you are not familiar with the most recent image formats, there are 3.5 contenders for the hearts and minds of sentient life in the universe:
AVIF and HEIC come from video-land, they are derived formats from video specifications. JPEG XL was created as a thorough-bred image format. WebP v2 is a new version of WebP, but it appears to be purely experimental at the moment.
Throughout history, it can be odd who wins format wars, it is not neccessarily the “best” format that wins. There seems to be a media format contest at least once a decade:
- In the 1970’s, it was analog videotape formats: JVC VHS vs. Sony Betamax vs. Philips Video 2000.
- In the 1990’s, it was digital casettes: Philips’ Digital Compact Cassette (DCC) vs. Sony’s MiniDisc (MD).
- In 2000’s, it was high-definition optical disc formats: Blu-ray Disc vs HD DVD.
- In 2010’s, it was digital video formats: It was H.264 vs H.265 (patented standards that require royalty payments) versus royalty-free alternatives like VP8, VP9, and Theora that attempted to avoid patent infringement.
These contests are not box office viewing, but if you dip into the stories it can be enlightening to understand the forces that shape the world around us.
Returning to Google’s story, Google added the AVIF format to Chrome version 85 without a flag in August 2020. Later, JPEG XL was introduced to Chrome version 91 in May 2021 as an experimental feature. It is curious that AVIF went straight in as a feature, but JPEG XL was experimental!
On October 30th of 2022, Google announced that it will remove JPEG XL from Chrome for the following reasons:
Thank you everyone for your comments and feedback regarding JPEG XL. We will be removing the JPEG XL code and flag from Chromium for the following reasons:
- Experimental flags and code should not remain indefinitely
- There is not enough interest from the entire ecosystem to continue experimenting with JPEG XL
- The new image format does not bring sufficient incremental benefits over existing formats to warrant enabling it by default
- By removing the flag and the code in M110, it reduces the maintenance burden and allows us to focus on improving existing formats in Chrome
Some short words with broad vowels were propelled with spittle by some corners of the web. Then, in January 2023, JPEG XL was removed in Chrome version 109.
Cloudinary aired some thoughts on Google’s reasons for removing support:
The first statement makes sense, but many people expected that it would be resolved by enabling the feature by default, not by removing the feature entirely. This does not justify this decision at all.
As for the second point: the question is if and how the interest of the ecosystem was gauged. Since the feature has been locked behind a flag, obviously any actual deployment has been blocked — disabled-by-default features can be used for experimentation, but not for actual deployment since the bulk of the end users will not have the flag enabled. So there are no meaningful usage statistics to look at.
However, if the enthusiastic support in the Chromium bugtracker from Facebook, Adobe, Intel and VESA, Krita, The Guardian, libvips, Cloudinary, and Shopify is any indication, it seems baffling to conclude that there would be insufficient ecosystem interest.
That brings us to the third and perhaps most important point: “Not sufficient incremental benefits over existing formats.”
Of course “not sufficient” is a rather vague criterion, if it is not specified what the threshold is for a benefit to be considered sufficient. In this blog post we will take a closer look at what the benefits are, and then you can judge for yourself whether they are “sufficient” or not.
You get the idea, there are plenty of counter arguments. You can read Cloudinary’s article to see their case for JPEG XL.
Cloudinary is biased with regard to JPEG XL as it is an image format that they had significant input to. Strangely enough, Google was a contributor to both the AVIF and the JPEG XL specifications.
Google likes a rival format it helped develop, AVIF. But JPEG XL has some advantages photographers might appreciate, and the dispute could mean we’re stuck with plain old JPEG for even longer.
Apple expressed a preference in the past too. It was an early adopter of HEIC, HEIC is supported natively on apple devices since macOS 10.13 High Sierra and iOS 11, to conserve device storage space apparently. However, it was not added to Safari.
It does seem daft that video-derived formats are preferred over image formats for images on the web. WebP also emerged from a video format, VP8. The improved image formats proposed in the past have barely made it to the web: JPEG 2000, and JPEG XR. The biggest stumbling block around JPEG 2000 was that it was percieved to be patent emcumbered.
I don’t know all of the merits and demerits of the formats, and if the best format did succeed, and is succeeding now. It appears to me that there is a dichotomy in usage - photographers want great quality images and fast processing, whereas the web wants optimum compression and animated images. Its like they are rival gangs who don’t want to cross paths. I am a reluctant member of both gangs!
The dream of an universal format
I just wonder, in 2023, why can’t I take a photo on my camera in a format that has good compression and is supported by web browsers? I could upload a photo to the internet and have it be shown on a webpage without transcoding.
Captain Planet can’t be happy with all that transcoding guzzling watts here, there, and everywhere!
Or is saving bytes better? 😖
It seemed to me that WebP succeeded because it delivered the feature of existing formats such as transparency and animation, along with superior compression. The narrative was about performance ultimately - give me the same as before but with smaller files.
As Web developers, it was not a big deal in theory. We could consider a new image format as a progressive enhancement. We can use the picture element, add a source for a WebP version of the image, and a source for a JPEG or PNG of the same image. If the browser does not support WebP, it will fallback to the other format. Easy. IPSO FACTO.
<picture>
<source srcset="photo.avif" type="image/avif" />
<source srcset="photo.webp" type="image/webp" />
<img src="photo.jpg"/>
</picture>Of course, you need to produce different versions of the same image for this work. You have a build process anyway, don’t you? 😥
That’s only part of the story, working with images in responsive web design is a bit of a pain. There is a reason that there was a rise of image CDNs such as Cloudinary!
I need to research it more, but it seems that the advantages of AVIF over WebP is:
- Superior compression,
- HDR support,
- Greater color bit depth (WebP is 8-bit, whereas AVIF has 8-, 10-, and 12-bit color depths).
Jake Archibald did a deep dive in AVIF in 2020. He showed the improvements offered by AVIF based on different use cases over JPEG, PNG, and WebP. It centered on relative image quality and file sizes. He has some live demos to show the images side-by-side.
It would be great to see a similar comparison of AVIF and HEIC and JPEG XL and WebP. This type of comparison could even be done in the browser, Jake used a Preact component to handle image loading and decoding for his demos, so that AVIF and WebP work even without browser support. You could extend this I guess?
Jake said the following about JPEG XL:
JPEG-XL is an image format, rather than an off-cut of a video format. It supports lossless and lossy compressions, and progressive multi-pass rendering. It looks like the lossless compression will be better than WebP’s, which is great. However, the lossy compression is tuned for high quality rather than ‘acceptable quality’, so it might not be a great fit for most web images. But, the benefit of multi-pass rendering might mean it’s worth taking a hit when it comes to file size. I guess we’ll wait and see!
Is the opportunity gone to see how JPEG XL fares?
I wonder - can any one of AVIF, HEIC, and JPEG XL become an universal format?
The only one that mentions this as a possibility is JPEG XL. It would be great for that to happen.
Do we need to re-evaluate images on the web?
With the advent of container queries, we may need to re-evaluate. Jason Gribsy wrote an interesting article on this recently with the title On Container Queries, Responsive Images, and JPEG-XL, Jason says:
Long before we settled on responsive images syntax, many of our discussions would inevitably turn to the idea of a magical image format that would contain all of the resolutions we required. It isn’t as far fetched as it sounds. At the time, JPEG-2000 had the ability to “display images at different resolutions and sizes from the same image file.”
But the dreams of a holy grail image format went nowhere. It was rumored that JPEG-2000 was patent encumbered, and we didn’t have any other formats on the horizon that offered similar advantages.
Arguably, the biggest challenge that led to the responsive images standard in the first place was the browser’s speculative downloading of images. Jason describes it as:
When the browser first receives an HTML document, before it builds the DOM, and long before it calculates layout, a feature called the lookahead pre-parser scans the document looking for assets it can start downloading. This behavior is called speculative downloading because the browser can’t be certain that the assets that it downloads will be used.
But getting a head start on downloading assets, even if some items are downloaded by mistake, has a significant impact in web page performance. Andy Davies reports, “During their implementation Mozilla reported a 19% improvement in load times, and in a test against the Alexa top 2,000 sites Google found around a 20% improvement.” And in 2015, Ilya Grigorik found that ”~43% of image fetches are initiated by the speculative HTML scanner, which account for ~50% of transferred bytes.”
An issue for JPEG 2000 at the time was that it clashed with the speculative downloader. If you don’t know the size of the image in the page in advance, how would the browser know when to stop downloading the image?
So, the route taken was to add responsive syntax to images, the srcset and sizes attributes. Be explicit, no magic required. It works with all image formats.
Now, we have container queries and it does not get along with the speculative downloader too. If you use picture element in a container query, it is tied to the viewport, not the container! What size do we download in advance?
This is an open problem space. There is also containment and lazy loading to consider in this equation.
I don’t know if JPEG XL is a magical image format that will save the day, but if offers some possibilities that the other formats don’t. Jon Sneyers, one of the creators of the image format, describes its responsive powers below:
Especially for web delivery, it would be desirable to avoid having to store and serve multiple variants of the same image according to the viewer’s viewport width. Equally desirable is an option to progressively decode images, showing a low-quality image placeholder when only a few hundred bytes have arrived and adding more detail as the rest of the data shows up. JPEG XL ably supports both nice-to-haves.
From my basic understanding of the issues we face, it seems hasty to dismiss JPEG XL as an option. We should take a step back and ask, what is the best outcome?
The best outcome for everyone, not just the web.
Certainly as a photographer, I would like to see JPEG XL being adopted. I would like to see the environment considered in these outcomes, if it takes more processing to get a smaller file, isn’t that a bad thing too? The line of enquiry is too web performance-oriented for my taste.
The state of play
The state of play is that AVIF is in Chrome, and is partially supported in Safari and Firefox.
HEIC and JPEG XL lie forlorn in a dusty corner.
Perhaps, the jig is not up for JPEG XL and HEIC. WebP was in Chrome for 5 years before the others decided to follow suit. The format war is a war of attrition!
May the format, pure of heart, triumph and rule the seven kingdoms!
Or maybe, it does not have to be a war, can the spoils be shared and we can rule together in harmony? Can we have AVIF and JPEG XL?
Captain Planet Image Credit: Wikimedia
