For this post, I will be showing you a model to extract heart rate from a simple video from your iPhone. To do this I leveraged MatLab, and many scientific studies/papers and resources. I used a Matlab script instead of R here because it can be considered the “jack of all trades”. This post was designed to be a very abbreviated and high level overview of the proposed method. and is post #1 of a 3-Part Healthcare Series I am currently working on.
Heart Rate is an important vital that most people should be tracking as it has a direct correlation with your current state of health. Thus, checking your pulse can become a tool that can help millions of people get a quick picture of their health. For instance, low or frequent episodes of unexplained fast heart rates can be a signal of potential health issues. We can learn a lot about a person using this information.
Proposed Model
The model works based on two known facts:
- Every heart beat pertains to a rush of blood in the blood vessels
- Fingertips contain capillaries, which are any of the fine branching blood vessels that form a network between the arterioles and venules.
The model will involve a cellphone that will record and flash a light through your finger that will illuminate the flow of blood through it. This process is one that mimics a Pulse Oximeter.
A Pulse Oximeter is a sensor device that is placed on a thin part of the patient’s body, usually a fingertip or earlobe. When the lights enter the finger, some amount of light is absorbed by the blood and tissues depending on the concentration of hemoglobin, from which the pulse and oxygen level are determined. This is possible because a pulse oximeter passes two wavelengths (red spectrum and infrared spectrum) of light through the body part to a photodetector and can measures the changing absorbance at each of the wavelengths, allowing it to determine the absorbances due to the pulsing arterial blood alone, excluding venous blood, skin, bone, muscle, fat, and (in most cases) nail polish.
In this model, the cellphone will act as the photodetector and LED Flash will serve as the LED’s of the oximeter probe:
- First, an iPhone will be used to record a video of short duration, with the finger placed over the lens of the mobile camera.
- During this recording, the flash will turned ON, so that adequate amount of light can reach the finger for proper measurement.
- At this point, this video will be transferred on to Matlab on my computer and I will run the developed script to obtain the heart rate measurement.
Results
The following is the resulting HR from a 10 second video of a person’s finger while illuminated with flash, and transferred over to Matlab:
MEAN HR: 56 BPM
A slow heart rate, of less than 60 beats per minute, is called bradycardia. This is an example of a slight bradycardia. Bradycardia can be normal if you’re a well-conditioned athlete. For example, a patient that bicycles 50 miles a week can have a resting heart rate of 50 BPM
As you can see, we can learn a lot about a person based on this information.
Technical Overview
The script that was developed can be broken down into six components: Signal Acquisition, Signal Computation, Band-Pass Filtering, Fast Fourier Transformation, Peak Detection and Smoothing. Below each topic is partial code that was used:
Signal Acquisition:
First thing to note here is that videos captured by the iPhone are at 30 Frames Per Second (30 Hz). Second thing to note is that the human heartbeat can range as dangerously low as 40 and as dangerously high as 230 beats per minute. Famous Electronic Engineer Henry Nyquist has stated that the sampling frequency should be at least twice the highest value to be able to catch the whole range of heartbeat frequencies. If we assume that the the target signal is located in between the whole heart beat range stated above (roughly between .667 – 3.8 Hz). The iPhone, having a frame rate of 30 (30 Hz) is much higher than 3 times the amount of needed sampling frequency to capture heart rate.
dataDir = 'Fill In The Blank';
vidname = [Fill in the Blank];
video_file = fullfile(dataDir,vidname);
v = VideoReader(video_file);
Signal Computation
As stated above, we are analyzing the amount of light that passes through your finger to come up with an Heart Rate estimation. When the camera is covered with a finger with flash on, we see a static red frame with our naked eye. However, using a video processing model, we can actually see subtle variations caused by the flow of blood in the blood vessels happening under the skin. In our video processing model we will represent all the red pixels per frame as one average brightness per frame calculation. This should help make the analysis easier. It should be noted that other pixel colors can be chosen as well (Blue and Green). The red pixel is only chosen here because it appears most in the video.
frame = read(v, i);
redPixels = frame(:, :, 1); y(i) = sum(sum(redPixels)) / (size(frame, 1) * size(frame, 2));
Band-Pass Filtering
This can be easily described as a method to pass frequencies within a certain range and reject (attenuates) frequencies outside that range. The sole purpose of doing this here is to reduce the amount of noise caused by external factors in the video capture. This includes actions such as movement. At a high level, a butterworth filter was designed and cut-off frequencies were set to reflect the whole heart beat range stated above (40 – 230 bpm). It is up to you to choose the order of the butterworth filter. Keep in mind that this filter is designed to have as flat a frequency response as possible in the passband.
BPM_Lowest = 40; BPM_Highest = 230;
[b, a] = butter(2, [(((BPM_Lowest)/60)/fps2) (((BPM_Highest)/60)/fps2)]);
yf = filter(b, a, y);
y = yf((fps * max(FiltTime, CutSeconds))+1:size(yf, 2));
Fast Fourier Transformation (FFT)
At a high level, the FFT converts a signal from its original domain (often time or space) to a representation in the frequency domain and vice versa. For our use, we can will transform a signal from time domain to frequency domain. The FFT will determine the magnitude of the corresponding frequency we have obtained. By repeating this step twice every second, we are able to generate a continuous Heart Rate calculation. You can utilize the fft function in Matlab to accomplish this.
Magnitude = abs(fft(y)); % y is the signal here
Peak Detection
As well, there is a simple Matlab function to help determine peaks (findpeaks function). We use this to find the magnitude peaks in our band of interest. A peak is only determined to be a peak if it is either larger than its two neighbors or equal to infinity. We will then find the highest peak using the max function and then translate it to its corresponding frequency.
[pks, locs] = findpeaks(gain(index_range));
[max_peak_v, max_peak_i] = max(pks);
Smoothing
At this point, we have the location of the largest tone in our band of interest. Smoothing of the FFT peak is then conducted to ensure the heart rate readings look more continuous, with 1 bpm frequency resolution as seen above. The frequency that corresponds to the highest magnitude is taken as the smoothed heart rate.
[max_peak_v, max_peak_i] = max(power);
bpm_smooth(i) = 60*freqs(max_peak_i);
Conclusion
Stayed tuned for the upcoming posts in this 3 part series.