Introduction
Let’s start by having a quick review of what is meant by an ECG.
An Electrocardiogram (ECG) is a non-invasive test which uses electrodes placed on the chest and the limbs to detect the electrical changes on the skin which are generated by a heart beat. These electrical changes are generated by the cells within the right atrium of the heart called the Sinoatrial Node (SA node) and its function is to cause the atria and then ventricles to contract.
For this article, we will focus on a few key features of ECGs. The first is the different types, the most common being the 12-lead, 6-lead, 3-lead and 2-lead versions. The number denotes the number of angles the electrical signal within the heart is measured. The second is the different scenarios in which they are used. You have your Resting ECG, a Stress ECG (typically performed whilst exercising on a treadmill or an exercise bike) and a Holter ECG, which is a portable ECG device which records your ECG over a period of time normally 24 hours. Resting is almost always done using a 12-lead ECG. To amplify the electrical signal a conductor is added, which is the gel that’s applied, and a clean connection is ensured by removing dirt and body hair.
The clinical significance of an ECG is to determine how the heart is functioning. It measures heart rate and rhythm to provide information about the possibility of cardiac issues, one of the most significant being Atrial Fibrillation (AF), a condition where the top two chambers of your heart do not function properly leading to complications such as the formation of a blood clot which can move towards the brain and cause a stroke. The term fibrillation describes the irregular contraction of the heart, an uncoordinated rhythm. Studies suggest that 1 in 4 cases of AF are intermittent, meaning that people don’t even realise they have it, which makes a wearable ECG potentially invaluable for early detection. This all sounds good in theory, but with all things scientific, we need to critically and objectively analyse if this works and more importantly if it is useful.
An innovation in technology
The first smart watch ECG came into the market back in 2018, in the form of the Apple Watch Series 4. The basic technology relies on a 1-lead ECG which is measured on the wrist of the arm, with most smart watch models prompting you to touch an area of the watch (such as the crown or bezel) with the finger of your other arm. By using both arms you can essentially see the electrical changes within the heart from right to left. The term innovation is applicable, as the electrode on the back of the device coupled with the software is effectively a functioning and portable ECG, which you own!
Are smart watch ECGs accurate?
Without getting too technical, the term “Accuracy” for a diagnostic test is determined during a validation study. It is mathematically derived from the study’s True Positive (TP), True Negative (TN), False Positive (FP) and False Negative (FN) rates. The calculation is as follows:
Diagnostic Accuracy = ((TP+TN/(TN+TP+FP+FN))
I explain this to provide a brief overview of how accuracy is measured, thereby providing some information relevant to the tagline of this article, but the much more clinically useful measurements of a test’s effectiveness are the Positive Predictive Value (PPV) and Negative Predictive Value (NPV) which define how probable a test diagnosis is to be genuine in practice. A PPV is a measure of how likely a positive result is to be real and the NPV tells us the opposite, and because they factor into consideration the disease prevalence (total number of cases of a disease divided by the total population), it becomes the much more clinically meaningful metric.
Back to the question. Are smart watch ECGs accurate? That can only be answered by data, but not just any data, it has to be a significantly large number to be statistically viable and applicable to the general population. When Apple launched the Series 4 back in 2018 they released the details of two studies which led to the United States Food and Drug Administration’s (FDA) seal of approval. The numbers weren’t huge, with one trial testing on a sample set of 588. Statistically speaking, that number is not representative of a population, doesn’t even come close. What’s more it was widely criticised by a lot of the medical community for numerous reasons, one of which was the lack of transparency of the data and the absence of peer review. How it stood up to the FDA stringent validation criteria is another story. The main problem with small numbers isn’t the True Positive rate, at least not for this study, it’s the number of False Positives. The rarer the medical condition, the worse the ratio of False Positives to True Positives ultimately resulting in a misleading accuracy. This is why a substantially large number of participants is required to achieve a more accurate reflection of the general population. Simply put, smart watches may be accurate, but they may not, as the data is insufficient for any real scientist to conclude and apply to the general population.
Now I’ve scoured a number of scientific publication sites and I can’t really find anything with a significant sample data set, and the only notable sample set belongs to a study published by Stanford Medicine and endorsed by Apple to determine whether a mobile app that uses data from a heart-rate pulse sensor on the Apple Watch can identify AF. The study enrolled staggering numbers of greater than 400,000 participants and is a data collection marvel in its own right. All sounds great right? But this was a study for people who already have an apple smart watch. What is interesting is the fact that it wasn’t even the Series 4! It was 1, 2 and 3 and used optical technology to detect heart rate irregularities. The main problem with this is that the technology doesn’t function in the same way, so can’t really be used to determine if modern day watches can detect the same events. Smart watches with electrodes are probably better let’s admit, but we have no idea.
A good trial is randomised, now think to how many people own an Apple watch, I personally do not, and although I come across a lot who do, I come across equal numbers of people who do not. Multiple factors in play here, with one being economics, Apple watches are not cheap, and another being age. Generally users of high-end smart watches like the Apple Series tend to be young (relatively, I’m talking under 65 at least) who can afford it. This is reflected in the studies age brackets with the >65s comprising a minuscule 6% of the study’s cohort. Hence population (data) bias.
The other issue with the study was the absence of a follow up with the participants who did not have recordable events, which means we don’t truly know the false negative rate. Even the 0.5% who received an alert (which is approximately 2000) and were sent an ECG patch to follow up only 21% returned the patch (79% clearly kept it as a souvenir for their bookshelf!). The most interesting thing about this 21% is only a third (33.33%) conclusively had AF. Again, the issue is lack of data, not validity of data, that 79% is still unaccounted for.
The takeaway from this long winded dissection is that I cannot, and anyone else shouldn’t, make a claim (either way mind you), to the accuracy of a smart watch detecting an AF event. There are of course many more smart watches on the market now, with more studies conducted, but again, the numbers are too low for me to confidently to comment on accuracy. If you are interested in these studies, one of the best reads for me was this one, not because the sample set was large, but more because it is a well designed clinical validation.
Overdiagnosis
With all of the above taken into consideration, let’s say that a study comes along and fulfils every criteria for statistical analysis and proves the ECG function to be accurate. The next question is what do we do in a clinical setting for those who have detectable cardiac events.
We will use me as an example. A male with no history of cardiac issues, who’s in his late 30s and extremely active and a vegan, so that checks all the boxes of a healthy lifestyle, although I am a massive fan of vegan biscuits! My watch detects a cardiac event, AF on one occasion, but I’m asymptomatic, meaning that the only clinical indication of this issue is the data. I go to my clinician with the PDF report that the accompanying software has generated. They can’t ignore it, as they have the same opinion as me regarding accuracy of the result, they don’t know either way if it is accurate or not so a follow up has to be done. A resting ECG is performed, let’s say it doesn’t find anything. Does the clinician still follow it up, or chalk it up to a false positive. Since I’m fairly young, it will probably be the latter, for extremely cautious clinicians, it will be the former. I’m given a Holter monitor, it detects minimal AF, meaning it is occasional, but not permanent. For my age group, or for any age group for that matter, the minimal time to be in AF which leads to a stroke or heart attack is unknown. Again it’s an insufficient data issue. Clinicians will not treat a non-permanent AF because the medication used to prevent the risk of strokes and heart attacks is anticoagulants, which carry their own associated risks of excessive bleeding. Instead, they will council their patients, and present every option before touching potentially counter-productive medications. Let’s pretend I am a bit of a hypochondriac and can’t let it go, I go back to my clinician and say I want it treated. The clinician won’t just prescribe medication on a whim, instead they follow up with more diagnostic investigations such as a Computed Tomography scan. It shows up with a undetermined finding (fairly common), which then leads to a Magnetic Resonance Imaging scan, which does the same. Does the clinician then order a Coronary Angiogram for an asymptomatic patient? How far down does this rabbit-hole go? Where does it end? Ignorance isn’t bliss, but I was conducting my life in a perfectly fine and happy manner before this reading came along. I’m now a patient being treated for a heart condition which had absolutely no bearing on my life and may have never have a significance on my longevity. You see where I am going with this.
Summary
When trying to determine if a diagnostic test is accurate, one must critically evaluate the data and apply logical reasoning to its interpretation. Do not take anything at face value, especially study data which is designed as a marketing tool. Similarly, look at the key factors which may affect the interpretation of test data. For the smart watch ECGs, this will be the insensitivity which is inherent to the 1 -lead detection, and is by far the greatest restriction in the sensitivity of data acquired from a smart watch ECG reading. Next consider the concept of overdiagnosis. Yes the diagnosis of silent cardiac issues is amazing and would be of life-saving value for those who suddenly drop dead of a cardiac issue which they did not know they had; but with these individuals there will be much more clear-cut evidence upon the 12-lead resting ECG. Treatment can be applied and lives can be saved. For those with occasional cardiac events, the diagnosis pathways and treatment are not always justified (the strain to an already overwhelmed public health system doesn’t help those who genuinely need the service and are at higher risk) and will increase morbidity and impact life decisions. Is it worth it? For the former group with silent and life threatening issues, yes, it could absolutely save their life, for the latter, I would say no.
This article, like any scientific analysis, looks at all variables objectively, and therefore may have come across somewhat negative. That was never the intention, I genuinely admire this technology and meant what I said by it being an innovation. I also believe, albeit optimistically, that smart watch ECGs are a lot more accurate than our current understanding, it’s just that I can’t scientifically make that claim.
Would I get a smart watch and use it for the ECG function? Probably not but that’s because my sceptical scientific brain prohibits me based purely on insufficient data and the fundamental lack of information about cardiac issues for under 65s. Would I recommend you buy one for your Nan? Yes, why not? If you can afford it. There is a lot more know about cardiac issues for this cohort, and they tend to be in permanent AF leading to a much higher risk of strokes and heart attacks, detection is paramount for this demographic.
Further information
If you are concerned about your heart health, for example those who have a family history or belong to a high- risk age group, the much more prudent approach is the classic one. Talk to your GP, allow them to conduct a proper assessment which will include a 12-lead ECG and other tests such as a blood lipid analysis, all of which contribute to a much more efficient and valid diagnosis/clearer clinical picture. Here at Avicenna Health, we are led by GP who has a special interest in Cardiology and worked as a Paediatric Cardiology Registrar in Great Ormand Street Hospital and the Royal Brompton, to name but a few. Do not self-diagnose, do not concern yourself with ifs , always be critical of data and seek out the proper channels should any genuine concerns arise.