The Issue with Transparent Pixels in Anisotropic Filtering

In the series of problems that I never imagined I would have to deal with when I started developing this video game, I present to you: the problem of transparent pixels in PNG.

The Graphical Basics of the Game

I draw all the elements of the game on Inkscape, a nice free vector drawing software. The images are then exported to PNG (then compressed in my personal format, but let's ignore this step for now). PNG is the most suitable format for this type of use, in particular because it supports transparency (the alpha channel of the image), which allows, when displaying an object on a background image, to see the object “blended” in the image, instead of an an image with a visible background:

This is all very neat, but then an unexpected problem arises: anistropic filtering.

Anisotropic Filtering and Transparent Pixels

Anistropic filtering is a display technique that allows objects to be resized or moved smoothly, maintaining soft edges and avoiding a pixelated effect: basically, if you resize an image or/and you move it to non-integer coordinates (at x=0.5 and y=100.3 for example), the SDL renderer locally computes weighted pixel averages. A picture speaks louder than words, here's what it looks like, left without anisotropic filtering, right with anisotropic filtering:

For a very long time, this technique did not cause any trouble. Until for one of the rooms in the game, I only displayed near-black objects on a near-black background (spoiler: it's an unlit room where you have to find how to turn on the light). And then I suddenly have this weird thing popping up:

Some sort of undesired white outline! It's quite ugly, because the illusion of a single scene is broken, it very much feels like that is a superimposition of different images.

The reason for these white artifacts is very simple: when the SDL computes anisotropic filtering, for each pixel, it gets a small neighborhood to do its local averaging computation. In 99% of cases, everything goes well. But on an edge pixel, it sometimes catches a pixel that is 100% transparent. And what are the RGB values ​​of this transparent pixel? You might have guessed it: they correspond to a white color. Let's have a look at the isolated image of the broom and its red, green, blue and alpha channels (transparency, white being opaque and black being transparent):

Obviously, in most cases we don't care, because the pixel being transparent, no matter its color, we will never see it. But there, this color is integrated into a computation for a pixel that is not completely transparent, and voilà, it creates these small white artefacts.

How do we fix this

I first tried to fix this directly in Inkscape. By the way, Inkscape's behavior changed recently: before, 100% transparent pixels were exported with a black color, which explains why I never noticed the artifacts (since, with my drawing style, the majority objects have a hard black outline). This is also why the background of the images is black on the first image of this article (apart from the mouth which has a strange color, but that comes from another thing that I will not discuss here).

For example, I tried to put a background of a certain color behind my object in Inkscape, and to put this object with a transparency of 100%. A waste of time: Inkscape's PNG export phase is akin to a “graphic rendering”, so the fact that a transparent object exists behind it is not taken into account, Inkscape “sees” a transparent pixel and the therefore exports with its default value.

I also must admit that this method would have been quite tedious in the long run. Also, for some objects, a single color for transparent pixels won't work: for example, a character's head is separated from their body to be animated differently, and their skin color needs to match at the neck junction, but the contours also need to remain black (without white artefact).

So I ended up integrating a little “repair” algorithm when I compress the game data: for each image with an alpha channel, it simply identifies 100% transparent pixels that are adjacent to non-100% transparent pixels, and assign them an RGB color averaged over the nearest non-100% transparent pixels.

In practice, it works very well, and stopping at a single closest pixel is enough to no longer disturb the anisotropic filtering algorithm, while being relatively fast to calculate.

Conclusion

As very often when developing a game from start to finish, the devil is in the details. This is typically a complex problem, as it comes from an accumulation of different sources, between the PNG standard, Inkscape's export settings, SDL's anisotropic filtering handling, etc.

I guess it would be nice if this was handled by SDL (which could logically just ignore 100% transparent pixels for anisotropic filtering, pixels whose color is obviously meaningless), but I don't know if this would be feasible in practice without sacrificing rendering efficiency.