Exploring the Feasibility of Remote
Cardiac Auscultation Using Earphones
Asclepius's Processing Animation
Micro-Benchmark Results
| Impact of different ear canal environment: |
We evaluate Asclepius's performance in a dry and a wet ear canal environment by testing it before and after a shower. As shown in the Figure (a), Asclepius achieves a decent performance in a dry ear canal environment, with an average RMSE of 1.08%. The average RMSE slightly increases to 1.12% in a wet ear canal environment. The result shows that the ear canal environment less impacts heart sound detection and reconstruction. |
| Impact of different human activities: |
Next, we examine the impact of different human activities on Asclepius. In this trial of experiments, we invited a volunteer and measured her heart sound before and after a running workout (termed as “normal status” and “after running”). The average RMSE grows from 0.85% to around 1.68% before and after a running workout, as shown in Figure (b). One possible reason could be the amplified interference from lung functions -- the subject tends to take deep breaths after a running workout, which leads to higher interference to the heart sound signal. We thus suggest users do not use Asclepius after a training workout. |
| Impact of different human postures: |
We then examine the impact of different human postures for auscultation. We divide the postures into sitting, standing, and lying down and measure the RMSE of recovered PCG signals in each posture setting. Figure (c) shows that the sitting position has a smaller average RMSE (0.8%) but with a higher variance. As for the other two postures, we find the average RMSE is slightly higher than that of the sitting posture (1.0% and 1.1%). |
| Impact of noise levels: |
We then examine the impact of ambient noise levels on Asclepius's performance. We play a piece of sound at different noise levels while experimenting. The result is shown in Figure (d). The average RMSE grows slightly from 0.7% to 1.3% when the noise level increases from 45 to 60 dBSPL, not as much as we expected. One reason could be that the earphone's closure blocks most of the ambient noise from entering the ear canal. |
| Impact of video meeting platform: |
Next, we assess the impact of different video meeting platforms on PCG quality. Existing video meeting platforms adopt proprietary codecs to ensure high user quality, which may negatively affect the quality of PCG signals. In this experiment, we tested four widely used video meeting platforms: Zoom, Skype, VooV, and Webex. We select the high-fidelity mode on each platform for PCG signal transmission. The results are shown in Figure (e). Our finding shows significant variation in performance across different platforms. The VooV meeting has the worst performance, with an average RMSE of 2.8%, followed by Zoom (2.4%) and Skype (2.4%). The Webex meeting performs best, with an average RMSE of 1.7%. These results suggest a non-compression transmission is needed for remote auscultation, and we believe this is doable for these video streaming platforms through a simple patch. |
| Impact of different wearing angle: |
Lastly, we investigate the effects of various wearing angles on Asclepius. Earbuds often sit differently in the ear based on their placement. Different wearing angles could introduce varying amounts of acoustic leakage, potentially affecting the perfect coupling between the earbud and the ear canal. To study this, we categorized wearing angles into five distinct degrees and measured the RMSE of the recovered PCG signal for each. At 0 degrees, which represents optimal coupling with no acoustic leakage, the RMSE begins at 1.22%. As illustrated in Figure (f), we observed that the average RMSE gradually increases from 1.22% to 1.73% as the wearing angle expands from 0 to 45 degrees. The RMSE remains elevated at 1.66% when the angle extends to 90 degrees, possibly due to increased acoustic leakage and reduced earbud-to-ear canal coupling. However, a surprising trend emerges when the angle reaches 180 and 270 degrees: the average RMSE drops slightly to 1.30% and 1.44%, respectively. This suggests that Asclepius's performance is not heavily compromised by variations in earphone placement. |
Audio Clips from Human Subjects
| Subject1 | Asclepius Groundtruth |
|||
| Subject2 | Asclepius Groundtruth |
|||
| Subject3 | Asclepius Groundtruth |
|||
| Subject4 | Asclepius Groundtruth |
|||
| Subject5 | Asclepius Groundtruth |
|||
| Subject6 | Asclepius Groundtruth |
|||
| Subject7 | Asclepius Groundtruth |
|||
| Subject8 | Asclepius Groundtruth |
|||
| Subject9 | Asclepius Groundtruth |
|||
| Subject10 | Asclepius Groundtruth |
Audio Clips Recovered from Pork Belly-based Emulation
| Groups | Types of Pathological PCG (from UMich Heart Sound & Murmur Library) |
| G1 | Split S1, Mid Systolic Click, Spilt S2 Persistent, Spilt S2 Transient |
| G2 | Early Systolic Murmur, Mid Systolic Murmur, Late Systolic Murmur, Holo Systolic Murmur, Early Diastolic Murmur, Systolic & Diastolic Murmur |
| G3 | S3 Gallop, S4 Gallop |
| G4 | Systolic Click & Late Systolic Murmur, S4 & Mid Systolic Murmur, S3 & Holo Systolic Murmur, Mitral Opening Snap & Diastolic Murmur, Systolic Murmur & Absent S2, Eject Systolic Murmur & Trans Split S2, Split S2 & Eject Systolic Murmur, Eject Systolic Murmur & Single S2 & Eject Click |
Select each pathological PCG with one heart cycle example (Sys for Systolic, Dia for Diastolic, Mur for Murmur):
| Split S1 | Asclepius Groundtruth |
Mid_Sys Click | Asclepius Groundtruth |
||
| Spilt_S2 Persistent | Asclepius Groundtruth |
Spilt_S2 Transient | Asclepius Groundtruth |
||
| Early_Sys_Mur | Asclepius Groundtruth |
Mid_Sys_Mur | Asclepius Groundtruth |
||
| Late_Sys_Mur | Asclepius Groundtruth |
Holo_Sys_Mur | Asclepius Groundtruth |
||
| Early_Dia_Mur | Asclepius Groundtruth |
Sys&Dia_Mur | Asclepius Groundtruth |
||
| S3 | Asclepius Groundtruth |
S4 | Asclepius Groundtruth |
||
| Sys_Click& Late_Sys_Mur | Asclepius Groundtruth |
S4& Mid_Sys_Mur | Asclepius Groundtruth |
||
| S3& Holo_Sys_Mur | Asclepius Groundtruth |
Opening_Snap& Dia_Mur | Asclepius Groundtruth |
||
| Sys_Mur& Absent_S2 | Asclepius Groundtruth |
Eject_Sys_Mur& Trans_Split_S2 | Asclepius Groundtruth |
||
| Split_S2& Eject_Sys_Mur | Asclepius Groundtruth |
Eject_Sys_Mur& Single_S2& Eject_Click | Asclepius Groundtruth |
Asclepius
Comparison with HeadFi
| Hardware detection | Spectrogram recovery | Waveform refinement |
| Hardware detection | Spectrogram recovery | Waveform refinement |
|
Why not HeadFi:
The above audio examples demonstrate HeadFi is not qualified for reconstructing fine-grained heart sound used for auscultation.
The reason could be HeadFi adopts a Wheatstone bridge to cancel out the strong music interference.
The Wheatstone bridge consists of four resistors. Two of them have an identical resistance (impedance), denoted as Z. Another two resistors are replaced by the left transducer and right transducer of an earphone, respectively, which are identical as well. As the physiological signal propagates along different paths before arriving at left and right transducers, it will lead to different variations to the impedance of the left transducer and right transducer of earphones. We denote ΔZl and ΔZr (ΔZl ≠ ΔZr) as the impedance variation of the left and right transducer due to physiological activities. The output voltage signal of HeadFi can be represented by: 
The above equation reveals that HeadFi captures the difference in ΔZl and ΔZr, which inherently cancels out fine-grained signal features containing critical physiological meanings (such as S1, S2, and murmurs) . The following Figure shows the heartbeat signals captured by (a) HeadFi and (b) an FDA-approved digital stethoscope (used as the groundtruth), respectively. We observe that although HeadFi maintains the general profile of each heartbeat cycle, it cannot retain the fine-grained heartbeat features such as S1 and S2 heart sound components that are crucial to auscultation. |
UX Study 1: Feedback From Users
|
We design a questionnaire under the guidance of a UX researcher to collect user feedback and comments on remote auscultation and Asclepius. We then ask each participant to fill out the questionnaire before and after the experiment.
The following Figure summarizes the results of a few key questions, out of many.
Before the experiment, we briefly explained our technology to each volunteer and asked their opinions. It turns out that 90% of the participants are positive about this technology before trying it. Some participants even commented Asclepius as the game changer of the telemedicine services. 10% of participants either have no idea about this technology or are skeptical about this technology. The participant then tries our system and is asked to fill out the remaining part of the questionnaire hereafter. The results show that around 73% of volunteers prefer to use Asclepius than a stethoscope for auscultation; 23% of volunteers feel there is no difference between using our system and a stethoscope, and the other 4% said they have ``no idea''. On the other hand, all volunteers are confident with Asclepius's performance after they see, hear, and further compare their PCG signal recorded by Asclepius and the dedicated stethoscope. Even those 3% skeptical volunteers change their attitude on Asclepius. More importantly, over 87% of participants tell us that Asclepius motivates them to pay more attention to their cardiac health. In terms of the willingness to buy, about 80% of volunteers show interest in purchasing Asclepius at 10 dollars. The remaining 20% said they have a wearable (e.g., an Apple watch or a Fitbit) for health monitoring. But they would not give up the opportunity to use Asclepius. |
UX Study 2: Cardiologist Interview
|
The procedure:
We started the interview by introducing Asclepius to the cardiologists. Subsequently,
we inquired about the principal cardiac features prioritized during cardiac auscultation
(P1). Four cardiologists identified the S1 and S2 heart sounds and heart murmurs as paramount.
Additionally, one cardiologist replied that he would assess pericardial friction rub and the
overall heart rhythms as well.
Next, we played a 20-second heart sound signal, sourced from a healthy individual and captured by Asclepius, for each cardiologist. The cardiologists were informed that the audio clip originated from our technology (P2). When asked if they could identify any cardiac features from the signal, all five cardiologists confirmed the clear presence of S1 and S2 heart sounds in nearly every heart cycle. In P3, we replayed the heart sound signal from Asclepius, immediately following it with its stethoscope counterpart to enable a side-by-side comparison. After playback, we sought feedback on any perceptible disparities between the two recordings. While three cardiologists identified no differences, two pointed out some nuances. Their feedback included remarks such as, ``Asclepius's recordings are somewhat less crisp compared to those from the stethoscope'', ``The PCG sounds from Asclepius seem to have a certain echo or reverberation quality to them''. Nevertheless, the cardiologists concurred that these nuances don't critically impact their diagnostic process. In P4, we adopted the protocols from P2 and P3, presenting the cardiologists with heart sounds from a pathological patient. Our aim was to evaluate their assessment of pathological features recorded by Asclepius. These pathological sounds, inclusive of heart murmurs in the systolic period, were sourced from a public heart sound dataset and captured by Asclepius through the pork belly experiment. Upon listening, all five cardiologists were able to identify the clear presence of S1 and S2 sounds, as well as the heart murmurs. Next, we employed the comparative approach from P3, playing back heart sounds from both Asclepius and the stethoscope for the cardiologists' evaluation of potential discrepancies. Four of them found no perceptible differences between the two recordings. The fifth, however, noted variations in the energy distribution among the components. He pointed out that, in certain heart cycles, the intensity of the S1 and S2 sounds seemed somewhat diminished in comparison to the murmurs. Concluding the session, we engaged in dialogue with the cardiologists to gather feedback on the advantages and drawbacks of our technology. Detailed reflections are accessible in the following: |
Configuration of earphones
| Headphones | Type | Transducer | Price (USD) |
| ATH M20X | Over-ear | Dynamic | 41 |
| Hifiman HE400SE | Over-ear | Planar | 99 |
| Monolith M565C | Over-ear | Planar | 157 |
| Razer Kraken Ultimate | Over-ear | Neody-mium | 65 |
| Surface Headphone2 | Over-ear | Dynamic | 299 |
| Bose QC 35 II | Over-ear | Dynamic | 299 |
| SONY MDRZX110 | On-ear | Dynamic | 20 |
| Grado SR80 | On-ear | Dynamic | 100 |
| ATH EW9 | On-ear | Balanced Armature | 106 |
| JBL TUNE110 | In-ear | Dynamic | 12 |
| ATH ANC33iS | In-ear | Dynamic | 29.9 |
| Thinklabs Earbud | In-ear | - | - |
We will keep the table updated with future experiments on other earphones.