Scientists from Jagiellonian University explain inconsistent results of COVID drug trials and propose solution

In their paper published in Studies in History and Philosophy of Science (https://doi.org/10.1007%2FS40656-022-00532-9) the researchers from the also say that the premises for starting clinical trials were of poor quality and propose to pay more attention to negative, not positive, mechanistic evidence.

As reported in the release of the Jagiellonian University Interdisciplinary Centre for Ethics, the SARS-COV-2 pandemic was one of the greatest threats to public health in recent years, and its effects were felt by millions of people worldwide. Now, after over 2.5 years of its duration, we already have vaccines that reduce the risk of severe course of infection and, to some extent, also the risk of transmission. We also have a certain repertoire of drugs that contain chemical molecules specially developed to prevent coronavirus replication by introducing a mutation into its RNA.

However, in 2020, humanity did not have any effective drug to improve the course of the disease or limit the spread of the virus. The development of new therapies is very time -consuming and therefore it seemed promising to check whether any of the already used drugs would prove effective.

The university’s Dr. Maziarz explains that attempts to repurpose a drug usually start with understanding how the mechanism of action of a given substance can affect the virus replication process.

Such studies are most often done by infecting cell cultures with a given virus, and then treating them with specific chemicals that can potentially slow its multiplication. Only the positive results of such tests qualify a given drug for animal trials models, and then - also in the case of satisfactory results - clinical trials involving people.

Due to the situation of the pandemic and the urgent need to discover a COVID-19 drug, clinical trials started immediately after obtaining positive laboratory test results or only on the basis of theoretical analysis of the virus replication mechanism and the action of the drug.

In addition, due to the rush clinical trials were often not properly coordinated and, for example, several different research groups began testing the same substance. In many cases such tests had inconsistent or even contradictory results. For example, some of them indicated that the drug reduced mortality or time of infection, while others showed that the same drug had no effect on COVID-19.

Dr. Maziarz says that publishing positive results first, and then research reports that contradicted them, for example, contributed to the American Food and Drug Administration (FDA) issuing a conditional approval for the treatment of COVID patients with hydroxychloroquine (an approved malaria drug ) and then withdrawing that approval.

Remdesivir shared a similar fate. Even in Poland patients in intensive care units were treated with this drug (developed as a therapy against the Ebola virus), but despite the initially positive results, the therapy that costs over 2,000 euros per one patient proved to be ineffective in further studies.

Until now, the different results were explained by the fact that some studies used worse methodology (for example they were not randomised), contained computing (or statistical) errors or used questionable research practices to prove the effectiveness of the tested drug. However, the analysis carried out by researchers from the Interdisciplinary Centre for Ethics of the Jagiellonian University and the Institute of Philosophy indicates that among both positive and negative results there were methodologically correct trials with sufficiently large samples, and studies burdened by errors or attempts at fraud.

It turns out that the very number of trials meant that one should several dozen false positives (reporting statistically significant differences between the treatment and control groups despite the actual ineffectiveness).

Under normal circumstances, new drugs are introduced to the market after obtaining positive results in two large clinical trials. However, in the case of COVID-19, in the absence of other therapeutic options, drugs were approved as soon as one study reported a positive result.

Dr. Maziarz said: “In this case, researchers and drug approval agencies (such as FDA or EMA) should statistically control the likelihood of false positives for all trials by reducing the level of statistical significance. This solution was not used because it would involve a greater chance of a false negative, i.e. not detecting a treatment that is an effective COVID.”

The solution proposed by the scientists from the Jagiellonian University is a completely different approach to theoretical reasoning and laboratory research, shifting focus to attempting to understand why a specific drug cannot be effective (this is called negative mechanistic evidence).

Until now, a large role was attributed to positive mechanistic evidence: if a drug was blocked the multiplication of SARS-CoV-2 in laboratory cultures, it was considered a potential candidate.

According to Dr. Maziarz, the problem with this approach was that the effect of the drug in the human body (in vivo) could be completely different than in in vitro conditions due to the enormous amount of potential interactions.

Therefore, a definitely better approach, proposed in the publication by Kraków researchers, is to focus on looking for mechanisms that may indicate that a given drug will not work in vivo, and not to continue to discover more substances that work in controlled cultures. The idea is to focus on negative, not positive, mechanistic evidence.

The authors of the study believe that with this approach it would be possible to avoid many unnecessary clinical trials and false hope.

For example, in the case of hydroxychloroquine, preliminary in vitro tests were carried out on the Vero line. Later tests showed that the viral infection of this line takes place differently than human lung cell infection. Thus, choosing the right model for research, one that would better reflect the place of viral infection, could provide negative mechanistic evidence that would cancel unnecessary clinical trials.

The report’s authors said: “Scientists should therefore strive to search for negative mechanistic evidence and only when it is not found, and positive is, proceed with clinical trials.”

The university’s Dr. Stencel added: “The philosophical Occam's razor principle says that +entities should not be multiplied beyond necessity+. Translating it into the language of clinical trials: We should not assume that the drug works in more different systems than tested. Especially when these systems are much more complex. Focusing on negative mechanistic evidence should help follow this principle.”

PAP - Science in Poland, Katarzyna Czechowicz

kap/ agt/

tr. RL