Doctors Can Now Detect Heart Rate and Respiratory Signals During Real-Time Video Calls, Study Finds

Researchers have developed a method that uses the camera on a person’s smartphone or computer to take their pulse and breathing signal from a real-time video of their face. The development comes at a time when telehealth has become a critical way for doctors to deliver health care while minimizing in-person contact during Covid-19. The University of Washington-led team’s system uses machine learning to capture subtle changes in the way light reflects off a person’s face, which correlates with the change in blood flow. Then, it converts these changes into pulse and respiratory rate.

The researchers presented the system in December at the Neural Information Processing Systems conference, and the team now comes up with a better system to measure these physiological signals. This system is less likely to be triggered by different cameras, lighting conditions, or facial features, such as skin color, according to researchers who will present the results on April 8 at the Association for Computing conference. Machinery (ACM) on health, interference. and learning. “Everyone is different,” said lead study author Xin Liu, a doctoral student at UW.

“This system must therefore be able to quickly adapt to each person’s unique physiological signature and separate it from other variations, such as their appearance and environment.” The first version of this system was formed with a dataset that contained both videos of people’s faces and “ground truth” information: each person’s pulse and respiratory rate measured by standard instruments on the monitor. ground. The system then used the spatial and temporal information from the videos to calculate the two vital signs.

While the system works well on some datasets, it still struggles with others containing different people, backgrounds, and lighting. This is a common problem known as “overfitting,” the team said. The researchers improved the system by having it produce a personalized machine learning model for each individual.

More specifically, it allows searching for important areas in a video image that likely contain physiological characteristics correlated with the change in blood flow of a face in different contexts, such as different skin tones, lighting conditions and environments.

From there, he can focus on that area and measure the pulse and respiratory rate. While this new system outperforms its predecessor when it comes with more complex data sets, especially for people with darker skin, there is still work to be done, the team said.