Some more hot PNG and now JPG compression fun!
A while back I did a quick article about PNG compression. Looking back on it. We have to admit that it was a rather weak and lackluster approach to compression. We could easily go back and amend our newest discoveries but the discussion of compression expands into multiple formats. So lets dive right into what's going on with this site and probably every WordPress generated site in existence and why compression is all around good for you.
The compression problem.
While browsing through the file structure of WordPress we found something that is really messed up. Which is the way the WordPress GD Library handles any PNG images. To illustrate this. We will give examples off of our own site.
24-bit Png Compression:
The picture above comes from Second Life that was later passed a bunch of filters and plugins in Adobe Photoshop. This results in a 24-bit PNG image. Conventionally this is the most popular way for bloggers which are also into photography to send images across the net because not only PNG has a high compression ratio but it's lossless, meaning that pixel by pixel it's going to be perfect no matter how many times they open this PNG file and re-save it. Unless a JPG or JPEG image file which is lossy, getting more and more destructive to the image the more you save.
| File Name | File Type | File Size in Bytes |
|---|---|---|
| Riot-mosaic-1024x1024 | PNG-24 | 1714821 |
| Riot-mosaic-930x930 | PNG-24 | 1476276 |
| Riot-mosaic-930x340 | PNG-24 | 554102 |
| Riot-mosaic-688x151 | PNG-24 | 210707 |
| Riot-mosaic-630x630 | PNG-24 | 758332 |
| Riot-mosaic-609x210 | PNG-24 | 256822 |
| Riot-mosaic-465x346 | PNG-24 | 319866 |
| Riot-mosaic-465x220 | PNG-24 | 208302 |
| Riot-mosaic-446x151 | PNG-24 | 145050 |
| Riot-mosaic-300x300 | PNG-24 | 196848 |
| Riot-mosaic-80x80 | PNG-24 | 15198 |
| Riot-mosaic-36x36 | PNG-24 | 3398 |
| Riot-mosaic | PNG | 1714821 |
Whenever you upload an image into WordPress. It passes through the GD Library to make all of the image listed above. As you can see, based off of the dimensional properties it makes lower file sizes for preview mode. This is good and the way WordPress is supposed to work because instead of slamming the full 1.7Mb PNG file at you upon page load it pulls the 300x300 medium sized preview only weighing in at around 197kb in file size. If the person wants to see the full detail of the image then they can simply click on it. The 1024x1024 preview is the exact same file size as the original because the original itself was 1024x1024. This is how the WordPress GD Library is supposed to work when it comes to compressed PNG images. And by default it assumes that all PNG files are 24 bit.
2-4-8-Bit PNG Compression:
But what if the PNG file isn't 24 bit you ask? What if you spent the extra effort of using a minimalist color pallet because what you're showing does not need 16 million colors on the screen? Something as simple as a technical blog article where you're firing 1920x1080p sized images to your viewer and want to make sure that your readers get them as fast as possible? 
In this example we will pull a screenshot from Android 4.x which was purposefully dumbed down from a 24-bit PNG to a 8-bit 32 color PNG. Why you ask? The original file size was somewhere around 230kb. But if we use a limited color pallet which still gets the message across it goes down to 14kb in file-size surpassing even jpg compression! Lets see what happens when we upload this to WordPress.
| FileName | FileType | File Size in Bytes |
|---|---|---|
| Android-Advanced-1024x451 | PNG-24 | 172884 |
| Android-Advanced-930x409 | PNG-24 | 156118 |
| Android-Advanced-930x340 | PNG-24 | 115796 |
| Android-Advanced-688x151 | PNG-24 | 44719 |
| Android-Advanced-630x277 | PNG-24 | 69578 |
| Android-Advanced-609x210 | PNG-24 | 56115 |
| Android-Advanced-465x346 | PNG-24 | 43232 |
| Android-Advanced-465x220 | PNG-24 | 34557 |
| Android-Advanced-446x151 | PNG-24 | 29131 |
| Android-Advanced-300x132 | PNG-24 | 16445 |
| Android-Advanced-80x80 | PNG-8-32-color | 1335 |
| Android-Advanced-36x36 | PNG-8-32-color | 461 |
| Android-Advanced | PNG-8-32-color | 13666 |
Holy crap! The GD Library inside of WordPress just took my 1733x764 sized image at 8-bits 32 color and made the preview images up to 10 times larger in file size! So people who think they are saving bandwidth and disk-space in WordPress are getting screwed over two-fold here! This is because when WordPress resizes the image into thumbnails and it can't find a good color pallet due to dithering the GD Library throws its hands up and goes
[quote] Fuck it, Saving this image as 24-bit![/quote]
Making the whole concept of optimizing your blogs a totally pointless.
I can just WordPress plugin my way through this. Problem Solved!
I've tried many plugins out there. EWWW , Kraken , Smush just to name a few. Many of them require certain compiled binaries and libraries your local web-ISP may or may not have installed onto their server. But besides that little headache these plugins do work provided the images you're uploading are 24-bit. However they do not prevent the GD Library in WordPress to generated 24-bit thumbnails on 8-bit images. There's other plugins like ShortPixel which promises to re-compress your thumbnails but requires a commercial license to do a certain amount of image for you.
Bottom line this is a problem that is at the very core of WordPress. Plugins are only going to mask the issue not legitimately fix it.
The manual approach.
In my past blog I talked about simply downloading your WordPress content folder and passing it over a commercial PNG optimizer. But once again, you're just optimizing a 24-bit png when it should be 8-bit. 
Of course we could open every single preview file that WordPress generates and re-save it ourselves. This is actually the best way to re-do your PNG's because you get to visually see how reducing the pallet will effect your images. However, a few hundred WordPress image uploads quickly turns into a few thousand preview files. You will be spending days if not weeks just in optimization. On top of also having to pay for a really good utility like Photoshop. This produces the best results but it's unacceptable time-wise.
Semi-Automated approach:
This requires you to have a linux box such as the Raspberry Pi . Or hell, even a Pine or Orange Pi as the only thing you'll be doing is using shell commands to pull this off. You can also setup a virtual linux image for those who know how to do that. Anyway you cut it. You need linux since there's no easily accessible windows programs which will do what we are about to do.
pngquant
At the time of this blog pngquant is at version 2.7.2 which you will have to download the source from their website and compile. The versions of pngquant in the Raspberry Pi and Debian repositories are massively out of date. If you check their site there's GUI versions too but because of the complexity of selection in WordPress we'll focus on just the command-line mode. After you have downloaded the backup of your website off of your website (If you are doing this on a live website be sure and MAKE A BACKUP! Not responsible for lost or damaged data from doing this!!) You can simply change directory into your WordPress content folder and pass the commands.
find /www/your-wp-blog/wp-content/uploads/ -name '*36x36.png' -exec pngquant --ext .png --verbose --force 256 {} \;
find /www/your-wp-blog/wp-content/uploads/ -name '*80x80.png' -exec pngquant --ext .png --verbose --force 256 {} \;
find /www/your-wp-blog/wp-content/uploads/ -name '*150x150.png' -exec pngquant --ext .png --verbose --force 256 {} \;
find /www/your-wp-blog/wp-content/uploads/ -name '*300x300.png' -exec pngquant --ext .png --verbose --force 256 {} \;
find /www/your-wp-blog/wp-content/uploads/ -name '*512x512.png' -exec pngquant --ext .png --verbose --force 256 {} \;
find /www/your-wp-blog/wp-content/uploads/ -name '*1024x1024.png' -exec pngquant --ext .png --verbose --force 256 {} \;
And so on, depending on how many thumbnails your both WordPress core and/or the theme that you are using. Due to my commercial theme using LightBox it generates a LOT of images! You may find that 256 colors may even work on your 24-bit preview images which will really help with speeding up your WordPress site! Also, if the majority of your PNG images are screenshots like my site. you may want to change "--force 256" to a color pallet even smaller like 32 or 64 this is something worth experimenting in a test directory just to see if you like those images.
optipng
At the time of this blog optipng is at version 0.7.4 and highly recommends that you download the source and compile it onto your Linux box. Many repositories like the Raspberry Pi and Debian repository will not have the latest version and some if not all of the flags passed will simply not work! Since this will help out ALL of our png files in terms of compression, you can pass this command globally throughout WordPress. This will also strip the meta-data files which very few people even use. Google likes it when you strip the meta as leaving meta-information inside of your images can confuse the search engines.
cd /www/your-wp-blog/wp-content/uploads/
find . -type f -name "*.png" -exec optipng -o2 -strip all {} \;
Now if you're on a more powerful system then a Raspberry Pi.. Or! you have nothing better to do with your Pi then keep a CPU perpetually occupied for months on end then pass the following command below for slightly better compression.
cd /www/your-wp-blog/wp-content/uploads/
find . -type f -name "*.png" -exec optipng -o7 -zm1-9 -strip all {} \;
What we mean by slightly better compression is you're looking at a %0.01 to %3 improvement. Which is nothing! The -o2 flag realistically is the best option for you.
Other notes about Web ISP's
Certain Web based ISP's will allow a "Jailed-Shell" SSH mode which will allow you to pass these commands provided that said web based ISP has these binaries installed onto their box. This is rare and should not be expected for users which are purchasing their web-hosting for a provider.
Optimize those JPG too while we're at it!
PNG files have taken the limelight in this article but there's something that can be done to JPG files to make them happy with google.
jpegoptim
At the time of those blog posting jpegoptim is at version 1.4.4 . You will have to download the source and compile it as many repositories like Raspberry Pi and Debian will not have the latest version installed. Not having the latest version of this file may result in errors. Once you have compiled and installed jpegoptim you can pass the following commands in the uploads section of WordPress:
find /path/to/wordpress/uploads/ -type f -name "*.jpg" -exec jpegoptim --strip-all {} \;
This will loss-less optimize the jpg files you have on your site further by removing any meta-data that is not needed by WordPress and the rest of the net. On my site this has given a savings of 5-16 percent depending on the jpg file. You can also tell it to batch re-compress the jpg files further which may be helpful on your WordPress thumbnail and preview images.
find /path/to/wordpress/uploads/ -type f -name "*.jpg" -exec jpegoptim -m70 --strip-all {} \;
Online solutions.
Sites like TinyPNG can go to hell in my book. Fuck you panda for screwing up my PNG files using a dithering technique that I don't even get to choose. What these sites do is partially lie to you. Stating that they saved 69% on a 24-bit png file but not tell you that it simply quantized your PNG into a 256-color PNG and loosing a bit of detail! The shell approach pngquant at least gives you the level of control if you so choose over your PNG files rather then mutilating your original files and simply not telling you. There's even a few wordpress plugins that pass PNGs through this service which frankly is only adding to the confusion and bullshit out there about how to optimize PNGs without destroying them. The only thing TinyPNG is good for is if you already have an indexed PNG file. In which case it can't screw it up. However, Always check it just in case!
Waving good-bye to the animated GIF - gif2apng.exe
The animated GIF has served the internet well throughout it's years. It was a format that was easily understood by the most earliest of computers dating back to the 286 with standard VGA graphics to the Commodore Amiga sporting anywhere from 4-64 colors on a 7Mhz 68000 processor. GIF unfortunately has copyright issues which is why you see a lot of free or near free graphics programs no longer support the GIF format. Combine this with the fact that there was no such thing as a 24-bit GIF file and you can see why people are walking away from the old and dying standard. Animated png or something referred to as (APNG) is a standard that all modern browsers can now understand. PNG has a open license which makes it preferable over GIF in many community projects and PNG has a better compression ratio with the help of 7-zip style compression. Below are some examples:
Cherry MX Blue switch animation in GIF format - 135KB
Cherry MX blue switch animation in PNG format - 100KB
The savings is rather massive and from here on out this site will be using the animated png standard for all content requiring it. For those interested in converting your old GIF files to PNG head over to this guys site for the windows compiled and command line versions.
Final thoughts.
WordPress like many CMS engines is a constantly evolving process relying heavily on open-source libraries that are already out there. The GD library is great because it's something that every web provider can host and there are almost no problems implementing. Perhaps quantizing your PNG files for screen captures may be a little bit on the old school side of computing. Afterall we did come from the era of the Commodore Amiga which 95% of the time you're working anywhere between 4 to 32 colors at all times. Old computers taught you restraint and optimizing your image. We certainly don't want to go back to those days but it taught us a lot of valuable lessons. And we hope that this entry gives a little bit of insight as to how WordPress operates. And that's what server said: END OF LINE+++