Colorize Black and White Photos — How It Works and What to Expect

AI colorization has gone viral multiple times — first with DeOldify (Jason Antic, 2018), then with DDColor (Kang et al., 2023). The results look impressive in demo videos. Real-world results are more mixed, and understanding the limitations helps you get better output.

The Fundamental Problem

No AI model knows the actual colors of a black and white photo. A colorization model sees a grayscale image and predicts: "given what I know about the world, what colors are most likely here?"

This works well for:

• Sky: Usually blue. Models get this right 95%+ of the time.
• Grass/trees: Green. High confidence.
• Skin tones: Surprisingly good, but biased toward training data demographics. Models trained mostly on Western photos may miscolor East Asian or African skin tones.
• Uniforms: Military uniforms from known eras (WWII olive drab, Union blue) are usually correct.

This fails for:

• Clothing: A dress could be any color. The model picks the most common one from training data, which tends toward muted, conservative tones.
• Cars: A 1957 Chevrolet could be any color. The model usually picks red, blue, or black — the most photographed colors.
• Interiors: Walls, furniture, rugs — too much variation for reliable prediction.
• Eyes: Models often default to brown. Blue and green eyes are frequently miscategorized.

DeOldify vs DDColor

These are the two dominant open-source colorization models, and they produce noticeably different results.

For family photos where you want something that "looks nice" on the wall, DeOldify's warm output is often more pleasing. For historical documentation where accuracy matters more than aesthetics, DDColor's conservative approach is safer.

Getting Better Results

• Provide context when possible. Some tools let you add hints — "this is a US Navy uniform from 1944" — which helps the model pick the right palette.
• Fix damage first. Scratches and stains confuse colorization models. The algorithm may interpret a white scratch as part of the clothing and colorize around it incorrectly.
• Adjust after. AI output is a starting point. Even 30 seconds of manual hue/saturation adjustment in a free tool like Photopea can improve the result significantly.
• Compare multiple runs. Some models produce slightly different outputs each time (due to stochastic elements). Running 2-3 times and picking the best result is a valid strategy.

Ethical Note

Colorized historical photos should always be labeled as colorized. Presenting AI-colorized images as original color photographs is misleading. The Library of Congress and other archival institutions flag colorized versions separately from originals for exactly this reason.

For personal family photos, this matters less — you know it was originally B&W. But if you share colorized photos on social media or in publications, a simple "colorized with AI" note prevents confusion.