A new study conducted by scientists has revealed that the capabilities of artificial intelligence in medicine are higher than expected. The differences were especially more noticeable in the initial diagnostic phase.
Medianews.az, citing Lent.az, reports that the research was carried out under the leadership of specialists from "Harvard Medical School" and "Beth Israel Deaconess Medical Center." The goal was to comparatively evaluate the language models developed by OpenAI with the work of doctors.
In one of the experiments, a study was conducted on 76 patients admitted to the emergency department. The diagnoses made by two practicing therapists were compared with the results presented by OpenAI’s o1 and 4o models. Later, the diagnoses were evaluated by two other doctors who did not know the source of each diagnosis.
According to the scientists, at every stage of diagnosis, the o1 model either showed better results or performed at the same level as the doctors. Especially during the initial stage, when there was little information about the patient, the superiority of artificial intelligence was more clearly felt.
The study noted that no specially processed data was provided to the models in advance. Artificial intelligence used only the information available in electronic medical records.
Based on the results, the OpenAI o1 model made accurate or very close to accurate diagnoses in 67% of the cases. For comparison, one of the participating doctors achieved an accuracy rate of 55%, and the other 50%.
One of the leading authors of the study, Arjun Manrai, head of the artificial intelligence laboratory at Harvard, stated that the model outperformed both previous artificial intelligence systems and the average performance of doctors across almost all criteria.
Nevertheless, experts consider a cautious approach to be essential. Adam Rodman noted that patients still rely more on doctors for life-critical decisions.
Emergency physician Kristen Pantagani emphasized that to properly assess the capabilities of artificial intelligence, it is necessary to compare it with doctors working specifically in the relevant specialty field.
It should be noted that artificial intelligence has recently attracted attention not only in medicine but also in other fields. For example, it was also used to discover previously unknown geoglyphs with faces in the Nazca Desert area of Peru. This research was carried out by an international team led by Masato Sakai.