AI Finds Clues in Faces About Cancer Patients' Survival

“You look unwell” or “You look tired” are expressions that, although sometimes annoying, we have internalized, and sometimes just looking at someone is enough to realize they are not feeling well. Fatigue, weight loss, or certain changes in our health status can be reflected in the face, although we do not always know how to identify what lies behind them. Doctors have long paid attention to these signs during their consultations, but now artificial intelligence dares to go further to find information that even escapes human observation.

A study published on October 19 in The Lancet Digital Health has shown that an AI is capable of analyzing facial photographs and finding patterns related to the survival of cancer patients. The tool, developed by researchers from Mass General Brigham and Harvard Medical School, has managed to distinguish groups of patients with significant differences in their mortality risk.

This is a discovery with particularly relevant possibilities for the field of oncology: being able to obtain broader information about a person’s health status with something as simple as a photograph.

What information can a facial photograph hide?

The idea behind this research starts from something we all know. Two people, despite being the same age, do not have to look their age nor be in the same physical condition. While some maintain a younger appearance, others may show more evident signs of aging.

This difference between the age on the ID and the age we appear to be has also sparked the interest of medicine, which has been investigating for years how to measure biological aging and its relationship with certain diseases.

To explore to what extent the face reflects these processes, the researchers developed a system called FAHR-Face, initially trained with more than 40 million facial images.

From there, they created two tools: FAHR-FaceAge, which calculates an approximate age based on facial features, and FAHR-FaceSurvival, which goes further and directly looks for characteristics related to the prognosis of oncology patients.

To train the latter, 34,363 photographs of people with cancer were used, along with information about their clinical progression. Subsequently, the models were evaluated with independent data from patients treated in the United States and the Netherlands, to verify if the identified associations also appeared in people different from those whose data had been used during training.

Patients with worse prognosis showed more than three times higher risk of death

When analyzing the photographs, FAHR-FaceSurvival classified patients into four groups according to the estimated prognosis. The results showed significant differences among them: patients in the worst prognosis group had a 3.2 times higher risk of death than those in the most favorable group. This association remained even after accounting for clinical factors such as age, sex, cancer type, or certain treatment characteristics.

The team also found interesting results using FAHR-FaceAge, the system that calculated apparent age from the face. Specifically, people whose estimated facial age exceeded their chronological age by at least five years had a 21% higher risk of death. This does not mean that looking older is necessarily a sign of illness, since our appearance is conditioned by numerous factors, from genetics to sun exposure or lifestyle habits. What is interesting is that, when analyzing patients as a whole, the AI identified facial characteristics associated with their clinical progression that were not fully explained by actual age.

The AI focuses on different areas of the face depending on what it tries to find out

One particularity of the research is that the two models did not pay attention to the same regions of the face. When the AI tried to calculate age, it focused mainly on the forehead, temples, and the grooves running from the nose to the corners of the mouth, areas where the most common signs of aging usually manifest. However, when estimating mortality risk, the algorithm gave greater importance to the region under the eyes and around the nasal bridge.

The researchers identified these differences using techniques that allow visualization of which parts of an image have more weight in an algorithm’s predictions. And although it is still unknown what specific features it is recognizing in those areas, the finding is interesting because it suggests that calculating apparent age and approaching disease prognosis are tasks that require partially different information, although in this case both used facial photographs as a medium. In fact, combining the results of both tools improved the models’ ability to differentiate patient groups according to their progression.

How this technology could be used in hospitals

In oncology, knowing a patient’s general condition is fundamental to deciding how to approach their disease. Two people with the same diagnosis can be in very different situations: one may maintain good physical condition and tolerate intensive treatments, while another may present greater frailty or additional diseases that require considering other alternatives. Therefore, besides studying tumor characteristics, professionals take into account aspects such as mobility, autonomy, physical strength, and nutritional status before recommending certain treatments.

In this context, a tool like FAHR-FaceSurvival could provide additional information to the evaluations already performed in hospitals. Obtaining a photograph is simple, inexpensive, and does not require invasive procedures, so the researchers consider it interesting to study its usefulness especially in older patients or in situations where completing a clinical assessment is complicated. The possibility of using it to identify people who need a more detailed evaluation of their physical condition is also being considered, although this application still needs investigation.

But its incorporation into medical practice will largely depend on how much it contributes to improving the decisions professionals already make. Currently, there are diagnostic tests, frailty scales, and other indicators that help estimate prognosis and assess treatment tolerance. The usefulness of this AI would be to complement and complete that information to provide relevant data not easily obtained through usual procedures.

What remains to be discovered before using it in patients

Although the results are promising, important questions remain to be resolved. One is understanding exactly what the algorithm is detecting, since facial features associated with worse prognosis could reflect factors as different as nutritional status, disease consequences, or the aging process itself. It will also be necessary to verify its performance in larger and more diverse populations, since some groups were underrepresented in the data used during the research.

Added to this is an important limitation: identifying groups of patients with different risk levels does not mean knowing how long a specific person will live. The study found relevant statistical associations but has not yet demonstrated that the tool’s estimates are sufficiently accurate to guide individual clinical decisions. The researchers will need to evaluate its performance in new studies and verify if it really improves available prognostic models.

For now, the research has shown that a facial photograph contains information related to the progression of oncology patients that algorithms manage to identify. The next challenge will be to determine which part of that information is truly useful for doctors and how to leverage it to improve care for people with cancer.