technologyreview | No one really knows how the most advanced algorithms do what they do. That could be a problem.
In 2015, a research group at Mount Sinai Hospital in New York was
inspired to apply deep learning to the hospital’s vast database of
patient records. This data set features hundreds of variables on
patients, drawn from their test results, doctor visits, and so on. The
resulting program, which the researchers named Deep Patient, was trained
using data from about 700,000 individuals, and when tested on new
records, it proved incredibly good at predicting disease. Without any
expert instruction, Deep Patient had discovered patterns hidden in the
hospital data that seemed to indicate when people were on the way to a
wide range of ailments, including cancer of the liver. There are a lot
of methods that are “pretty good” at predicting disease from a patient’s
records, says Joel Dudley, who leads the Mount Sinai team. But, he
adds, “this was just way better.”
At the same time, Deep Patient is a bit puzzling. It appears to
anticipate the onset of psychiatric disorders like schizophrenia
surprisingly well. But since schizophrenia is notoriously difficult for
physicians to predict, Dudley wondered how this was possible. He still
doesn’t know. The new tool offers no clue as to how it does this. If
something like Deep Patient is actually going to help doctors, it will
ideally give them the rationale for its prediction, to reassure them
that it is accurate and to justify, say, a change in the drugs someone
is being prescribed. “We can build these models,” Dudley says ruefully,
“but we don’t know how they work.”
Artificial intelligence hasn’t
always been this way. From the outset, there were two schools of thought
regarding how understandable, or explainable, AI ought to be. Many
thought it made the most sense to build machines that reasoned according
to rules and logic, making their inner workings transparent to anyone
who cared to examine some code. Others felt that intelligence would more
easily emerge if machines took inspiration from biology, and learned by
observing and experiencing. This meant turning computer programming on
its head. Instead of a programmer writing the commands to solve a
problem, the program generates its own algorithm based on example data
and a desired output. The machine-learning techniques that would later
evolve into today’s most powerful AI systems followed the latter path:
the machine essentially programs itself.
At first this approach
was of limited practical use, and in the 1960s and ’70s it remained
largely confined to the fringes of the field. Then the computerization
of many industries and the emergence of large data sets renewed
interest. That inspired the development of more powerful
machine-learning techniques, especially new versions of one known as the
artificial neural network. By the 1990s, neural networks could
automatically digitize
handwritten characters.
But
it was not until the start of this decade, after several clever tweaks
and refinements, that very large—or “deep”—neural networks demonstrated
dramatic improvements in automated perception. Deep learning is
responsible for today’s explosion of AI. It has given computers
extraordinary powers, like the ability to recognize spoken words almost
as well as a person could, a skill too complex to code into the machine
by hand. Deep learning has transformed computer vision and dramatically
improved machine translation. It is now being used to guide all sorts of
key decisions in medicine, finance, manufacturing—and beyond.
