GFPGAN vs. CodeFormer: Deep Dive into AI Face Restoration Models

In the field of computer vision, restoring degraded faces from old, low-resolution, or physically damaged photos is known as Blind Face Restoration (BFR). Unlike standard super-resolution, which simply interpolates pixels, BFR models must "hallucinate" realistic eyes, teeth, hair, and skin texture.

Two open-source models dominate this space: Tencent ARC's GFPGAN and S-Lab's (Nanyang Technological University) CodeFormer. While both produce impressive results, they use fundamentally different architectures, leading to different trade-offs in identity preservation (fidelity) and reconstruction quality.

The Core Architectures

GFPGAN: Generative Facial Prior

Published in CVPR 2021 by Xintao Wang et al. (Wang et al., 2021), GFPGAN utilizes a pre-trained face Generator (specifically, StyleGAN2) as a Generative Facial Prior (GFP).

GFPGAN uses a U-Net style design to extract features from the degraded image and injects them into the StyleGAN2 generator at multiple resolutions via spatial alignment and a channel-split attention mechanism. This allows the model to retain the overall structure of the input face while utilizing StyleGAN2's rich, high-resolution priors to synthesize missing details.

CodeFormer: Codebook Lookup Transformer

Published in NeurIPS 2022 by Shangchen Zhou et al. (Zhou et al., 2022), CodeFormer takes a different approach by framing face restoration as a code prediction task in a discrete space.

Instead of mapping continuous degraded features directly to continuous clean features, CodeFormer utilizes a discrete codebook learned via a Vector Quantized Autoencoder (VQ-GAN). The codebook contains quantized vectors representing common high-quality facial compositions. A Codebook Lookup Transformer is trained to map degraded facial features to the correct sequence of discrete high-quality codes. Because it works in a quantized discrete space, the model is highly robust to severe noise and artifacts.

Technical Comparison

Finding the Sweet Spot: Tuning the CodeFormer Weight (w)

When developers host CodeFormer on Replicate or implement it in web apps, they expose a fidelity weight parameter, denoted as w(ranging from 0.0 to 1.0). If you want to know **"how to make CodeFormer look natural,"** managing this parameter is critical:

• w = 0.0 to 0.3 (High Hallucination): The model ignores almost all texture from the input image. While it creates an extremely sharp, smooth, scratch-free face, the face will look artificial and will often resemble a different person.
• w = 0.8 to 1.0 (High Fidelity): The model maps features directly, keeping the original face structure and details intact. However, it fails to remove noise, JPEG artifacts, and heavy blur.
• w = 0.5 to 0.7 (Balanced / Natural): This is the ideal range. Setting w = 0.6 is the standard production value. It uses the codebook to reconstruct realistic eye irises, teeth, and nostrils, while retaining the overall bone structure and skin texture variations of the original photograph.

Common Failure Modes in Blind Face Restoration

AI face restoration models are not magic; they have clear structural limitations that are frequently discussed on developer forums:

• The Eyeglass Glitch: Neither GFPGAN nor CodeFormer has a robust semantic understanding of eyeglasses. They often misinterpret glasses frames as facial shadows or wrinkles. In highly blurry photos, CodeFormer will frequently fuse the frame of the glasses directly into the eye socket, creating strange circular dark patches or rendering asymmetrical lenses.
• Hats and Hairlines: Models are trained primarily on standard face crops (e.g., the FFHQ dataset). If an ancestor is wearing a historical military cap, a wide-brimmed hat, or has a complex vintage hairstyle, the generator struggles to align the prior, often smudging the cap or rendering it as an extension of the hair.
• Training Bias: Because standard face datasets contain a disproportionate number of high-quality Western photographs, early versions of BFR models exhibit demographic bias. They sometimes reconstruct East Asian, South Asian, or African faces with features that conform closer to Caucasian facial proportions. Setting a higher fidelity weight (w >= 0.65) is essential to prevent this.

Summary: Which Settings to Use?

In production environments like LeoRestorer, a hybrid approach is often used:

1. For **mildly blurry photos**, **GFPGAN** is preferred because it preserves small details like individual hair strands and unique skin spots better, maintaining a highly natural look.
2. For **severely degraded photos** (e.g. newspaper prints with half-tone dots or heavily water-damaged faces), **CodeFormer** is far superior. It ignores the dot patterns entirely, reconstructing clean eyes and sharp teeth that GFPGAN would distort.