ChromoNet

A metric that an optimiser is minimising cannot be the evidence that the optimiser is right.

The question

Given one uncontrolled photograph, DECA recovers a 3D face: its geometry, an estimate of the light that fell on it, and a coarse albedo drawn from the Basel Face Model. That albedo is where the skin colour comes from, and it is built from scans that under-represent darker skin. ChromoNet asks whether that produces a measurable bias, and whether a correction grounded in skin biophysics can repair it.

The method

The albedo is re-expressed as five chromophores, the pigments that actually give skin its colour, principally melanin and haemoglobin. A per-image search then adjusts melanin and dermal haemoglobin towards a reference tone recovered from the photograph itself.

1. 01 Photograph One uncontrolled image
2. 02 DECA Geometry, lighting and a coarse albedo, used unmodified
3. 03 PredNet Five chromophores per texel
4. 04 Search Melanin and haemoglobin adjusted towards the de-lit reference
5. 05 ReconNet Corrected reflectance, rendered and scored

The de-lit reference comes from the same pass: the photograph is unwrapped onto the face, its specular highlights removed with a dichromatic model, and DECA’s shading divided out.

What was found

• Before the search
• After the search

Reference says too light n = 32

Before +3.93 L* After −1.05 L* −1.05

Reference says too dark n = 71

Before +0.87 L* After −3.80 L* −3.80

All triggered faces n = 103

Before +2.23 L* After −3.12 L* −3.12

−4 −2 0, matches photo +2 +4

The numbers, scored against the photograph

Group	n	ΔL* before	ΔL* after	ΔE before	ΔE after, shipped	ΔE after, all five
Reference says too light	32	+3.93	−1.05	5.74	3.37	2.54
Reference says too dark	71	+0.87	−3.80	9.23	8.29	7.42
All triggered faces	103	+2.23	−3.12	8.58	7.43	6.65

• The bias is real. The reconstruction renders skin too light, a median of +1.47 L* across 3,254 faces, and compresses the tone range to a slope of 0.93 where a faithful pipeline would give 1.0.
• It survives without the reference. Compared directly with the photographs, 67.8% of reconstructions come out lighter than the face they were made from. This is the study’s strongest result.
• The correction works where the reference can be trusted. On faces the reference calls too light, colour error against the photograph falls from 5.74 to 3.37 with the search as shipped, and to 2.54 when all five chromophores are free.
• It fails where the reference cannot. On faces the reference calls too dark, the renders were already within about one L* of the photograph. The reference inherits the albedo and shading ambiguity it was built to detect, and following it makes the tone worse.

What it cannot claim

The correction is not shown to work on dark skin. A single-image cheek reading cannot separate genuinely dark skin from a shadowed or warmly lit lighter face, so the supported claim is limited to renders that come out too light. An earlier training scheme, a round trip through a frozen decoder, was measured, found to cause three quarters of the bias first recorded, and abandoned. The dissertation reports all of it.

Credits

A University of York Computer Science dissertation, supervised by Dr Giuseppe Claudio Guarnera. DECA and FLAME are used unmodified. The five-chromophore skin model and look-up table recipe are from Li, Guarnera et al. (3DV 2024), and the two-network estimator design from the same group’s 3D uplifting work (EGSR 2024), re-implemented here. FFHQ is used under its research licence.