Algorithmic aversion vs. appreciation
Gaube et al use an interesting method to confirm that physicians with high domain expertise display algorithmic aversion.
In this study, physicians received chest X-rays and diagnostic advice, some of which was inaccurate, and were asked to evaluate advice quality and make diagnoses. All advice was generated by human experts, but some was labeled as coming from an AI system. As a group, radiologists rated advice as lower quality when it appeared to come from an AI system; physicians with less task-expertise did not.
Diagnostic accuracy was significantly worse when participants received inaccurate advice, regardless of the purported source. This work raises important considerations for how advice, AI and non-AI, should be deployed in clinical environments.
Can Technology increase COVID-19 vaccination rates?
This question is raised in a Lancet editorial.
It’s the classical “Can AI fix it”-question.
It’s probably going to take a lot more than a fancy algorithm to combat vaccine hesitancy.
Mislabeled Test Data can destabilize Machine Learning Benchmarks
An interestring new study by Northcutt et al shows that mislabeled test data is pervasive in ML benchmarking trials. By correcting test data labels, ML benchmarking can better predict real-world performance.
Small increases in the prevalence of originally mislabeled test data can destabilize ML benchmarks, indicating that low-capacity models may actually outperform high-capacity models in noisy real-world applications, even if their measured performance on the original test data may be worse.
Assessment of Diagnosis and Triage (study)
A study of 5000 predominantly white americans, shows that a patient’s internet search of a condition doesn’t really affect the overall triage or diagnostic accuracy.
Here’s some quotes from the article in JAMA:
Each day, millions of people worldwide who are confronted with new medical symptoms turn to the internet before seeking care to understand why they are ill, whether they should get care, and where they should get care.
Despite its ubiquitous use, the benefits and harms of an internet search for health information are poorly understood. Previous research has been largely limited to observational studies of internet search behavior and may lack a criterion standard.
As the case acuity became more serious, respondents reported more anxiety
Mean internet search time was 12.1 (95% CI, 10.7-13.5) minutes per case
Improved diagnostic accuracy was found (49.8% vs 54.0%; difference, 4.2% [95% CI, 3.1%-5.3%]; P < .001).
Microsoft invests billions of $$ in Health AI
They’ve failed in digital health before, but that doesnt seem to stop them from trying to get a foot in the door.
From Wired:
Siri and Alexa may understand your grocery list, but Pessin says Nuance has spent time and effort developing technology that grasps the specialized language of medicine; that won’t be easy for other companies to replicate, he says. “That process of getting the system, the AI engine, to understand the medical jargon that’s important and difficult—I think that’s the real gem.” Pessin says Nuance may be able to develop parallel systems for other industries with their own jargon, such as law.
10 dangers of Digital Health
This is good list of dangers. I believe points 1 and 7 are most urgent:
- Regulating adaptive A.I. algorithms
An adaptive artificial intelligence (A.I.) is one that can adapt itself on-the-go based on new information it receives. In a clinical environment, it would, for example, be able to recommend blood tests more frequently in a population that has a high prevalence of diabetes. And if it sees that people in this population also have a tendency to develop cardiac issues, it will also recommend a cardiovascular evaluation.
However, such algorithms rely on existing medical data which is fraught with inherent biases. By evolving on such data, adaptive A.I. will only reinforce those harmful biases such as discriminating based on one’s ethnicity and/or gender. To mitigate those risks, regulatory authorities must adopt new regulatory approaches to ensure that equitable adaptive A.I. algorithms are employed in healthcare facilities.
- A.I. not tested in a real-life clinical setting
We regularly hear news headlines of an A.I.’s incredible prowess in the healthcare setting. But in many cases, these results are obtained from laboratories using selected datasets or ideal settings which aren’t totally reflective of actual clinical environments. Google learnt it the hard way.
Medical researchers from the company touted the ability of its A.I. tool to screen patients for diabetic retinopathy (DR) from images with 90% accuracy. DR is a major cause of vision loss worldwide, and detecting the condition earlier can prevent complications. But Google’s A.I. didn’t fare so well in practice as it did on paper. When used by nurses in a hospital in Thailand, they encountered several issues. Sometimes they had internet connection troubles; at other times the quality of the scan didn’t meet a certain threshold, so the A.I. simply didn’t give a result. And on some occasions, nurses even had to spend extra time editing some of the images the algorithm didn’t want to analyse.
The Tech Giants that bet on Health AI and Telemedicine
From the Medical Futurist:
The Foundations of AI are riddled with errors
Basically, “shit in - shit out”, but used to drive your car, identify a face or, in the future, help set a diagnosis.
In the competition, a method called deep learning, which involves feeding examples to a giant simulated neural network, proved dramatically better at identifying objects in images than other approaches. That kick-started interest in using AI to solve different problems.
But research revealed this week shows that ImageNet and nine other key AI data sets contain many errors. Researchers at MIT compared how an AI algorithm trained on the data interprets an image with the label that was applied to it. If, for instance, an algorithm decides that an image is 70 percent likely to be a cat but the label says “spoon,” then it’s likely that the image is wrongly labeled and actually shows a cat. To check, where the algorithm and the label disagreed, researchers showed the image to more people.
Such errors might lead machine learning engineers down the wrong path when choosing among different AI models. “They might actually choose the model that has worse performance in the real world,” Northcutt says.
Outgrowing Software
Benedict Evans argues that software has already «eaten the world». Software comes and changes markets, but in the end it is people in book, music etc. industry that have to resolve futures challenges - not computer scientists. The same therefore goes for medicine and digital health.
Ten years ago Marc Andreessen wrote an article in the WSJ called ‘Software is eating the world’, arguing that there was a fundamental shift in the role that software plays in the economy. In the past, IBM, Oracle or Microsoft sold technology to other companies, as a tool. They sold computers and software to GE, P&G and Citibank. Now there’s a generation of companies that both create software and use it themselves to enter another industry, and often to change it.
