Researchers at MIT and Massachusetts General Hospital have recently carried out a study investigating the possibility of using deep reinforcement learning to control the levels of unconsciousness of patients who require anesthesia for a medical procedure, per a report. The paper, set to be published in the proceedings of the 2020 International Conference on Artificial Intelligence in Medicine, was voted the best paper presented at the conference.
Schamberg and his colleagues developed a deep neural network and trained it to control anesthetic dosing using reinforcement learning within a simulated environment. They specifically focused on the dosage of Propofol, a medication that decreases people’s level of consciousness and is commonly used to perform general anesthesia or sedation on patients who are undergoing medical procedures.
The researchers trained a neural network using the cross-entropy method, by repeatedly letting it run on simulated patients and encouraging actions that led to good outcomes. Then, they trained the neural network on simulated patient data, which was generated based on pharmacokinetic/pharmacodynamic models with randomized parameters. This ultimately allowed them to account for numerous patients with varying characteristics and features. They ran a series of training trials, using the ‘cross-entropy’ method. During these trials, the neural network gradually learned to map an observed anesthetic state to a probability of infusing a fixed Propofol dosage, added the report.
When they evaluated their model’s performance, the researchers applied a deterministic policy that transforms the probability of infusing a fixed Propofol dosage into a continuous infusion rate. Overall, their neural network achieved remarkable results, outperforming a proportional-integral-derivative (PID) controller, which has previously been used to determine ideal doses of anesthesia.
The two primary advantages of this approach are its ability to scale the clinical variables included in the observation and the deep network’s continuous relationship between the input variables and the recommended dosage, stated the reseach. In the future, the deep neural network-based model devised by this team of researchers could assist anesthesiologists in identifying the ideal dosage of Propofol for individual patients and achieve different levels of unconsciousness. Nonetheless, the model has so far only been tested in simulations, so before it can be applied in real-world settings it will need to undergo a series of clinical trials with real patients.