To be an effective tool for fighting the COVID-19 pandemic, a vaccine will need the confidence of the public, including the doctors, nurses and pharmacists who administer the vaccine. Following part one of our series, here are frequently asked questions based on conversations I’ve had with many people about clinical trials and vaccine development.

What is a vaccine and how does it work?

A vaccine is a substance that stimulates a person's immune system to develop antibodies that produce complete or partial immunity to a specific disease. Vaccines prevent 2-3 million deaths per year. Unlike most medicines, vaccines don’t treat disease, but they can prevent it from spreading and making people sick. Many vaccines are made by taking an attenuated (or weakened) strain of a virus at a level that’s too low to cause disease, but high enough for the body to create an immune response. Some newer potential vaccines, however, use different technology that doesn’t involve direct use of the virus.

Since vaccine development usually takes years, how are scientists able to move more quickly now while doing responsible research?

COVID-19 has ushered in an unprecedented level of global collaboration and investment. Researchers mapped the virus’ genome just a month after the initial outbreak, giving vaccine developers a major head start. Additionally, advancements in science and technology mean that scientists are able to use new methods of vaccine development, like RNA vaccines.
RNA gives cells in the body instructions for creating proteins (the building blocks of DNA). Two vaccine candidates use RNA to enter a cell and stimulate the production of the SARS-CoV2 virus, which causes COVID-19 disease, creating an immune response.

Another reason why vaccine development can be slow is that funding streams can be hard to secure. Enormous amounts of funding from both public and private sources have moved the science along at unprecedented speed. U.S. Federal funding for COVID-19 vaccine development helps remove this burden. Other measures the Department of Health and Human Services (HHS) are taking include testing multiple vaccine candidates simultaneously and putting advanced manufacturing capacity in place to scale up production in the event the vaccines are found safe and effective. Normally, it would be prudent to await the trial results before scaling up to avoid the enormous investment with no assurance of success.

The issue of safety is on the minds of most people when vaccines are considered. There is no way around the fact that we will not have long-term safety data when the vaccine is initially distributed should one or more of the trials meet the criteria for EUA. Accordingly, the FDA will need to use its best judgment with limited followup. The efficacy side of the equation will be more clearcut. This means that post-market followup will be critical to fill in the needed information and the rollout of the vaccine will need to be targeted at high-risk groups in whom the benefits most likely outweigh the known and potential risks.

In order to earn public trust in COVID-19 vaccines, it will be important for findings and data from the research process to be open and transparent to the scientific community.

Why is it important for trial participation to be as broad as possible?

All vaccines and treatments must undergo testing, often involving thousands or tens of thousands of people, to ensure safety and effectiveness. Because COVID-19 is a new, global disease, it’s even more important to make sure these trials represent many types of people — including different age groups, racial and ethnic backgrounds, and health histories. The balance of benefits and risks should guide the choice to use a particular treatment, and this balance is not uniform in all people. Therefore, the recruitment of a broad population enables some assessment of the spectrum of benefits and risks.

For COVID-19, older people and those with comorbidities (especially multiple comorbidities like heart disease, chronic lung disease, diabetes and obesity) are at higher risk and may have more to gain from the vaccine, but these same people may be at higher risk of complications. In the United States, COVID-19 is also disproportionately impacting people of color and those in lower-income demographics. It’s critical that trials recruit people from these communities to ensure vaccines can benefit those that need it most or that it can be avoided if risks outweigh benefits.

What do we know about the vaccines so far?

The progress of candidate vaccines is well-covered by this NY Times website. Four vaccines made in China and one vaccine made in Russia are already in clinical use, but none of these have passed the U.S. FDA’s criteria. FDA requires that a vaccine demonstrate 30% effectiveness compared to placebo at the lower limit of the confidence interval. This means that the observed effect in a successful trial of any vaccine will be at least 50% more effective than placebo (thus the lower confidence interval falls above 30%).

Currently there are 27 vaccine candidates in Phase I evaluation, 14 in expanded safety trials and 9 in large-scale definitive clinical outcome trials. We’re still learning about the level of immunity that the COVID-19 vaccine(s) will produce, but it’s likely that the first vaccines will be most effective at preventing severe COVID-19 disease. The first set of vaccine candidates may require “booster injections” that people will likely need to receive a few months after their first dose.

What do we know about the trials?

The three trials that are most advanced in recruitment have published their protocols. This table from Eric Topol’s Twitter feed gives some relevant details. While the trials are more alike than different, each trial takes a distinct, nuanced approach to the sample size, the randomization allocation and the statistical analysis. This is not at all uncommon in clinical trials as there are literally hundreds of decisions that must be made to design a protocol. The critical point is that the FDA is overseeing the fabric of trials to make sure that the findings can be placed into context as decisions are made about marketing and labeling.

- Robert M. Califf, MD, MACC, Head of Clinical Policy and Strategy for Verily and Google Health

Vaccines prevent 2-3 million deaths per year. Unlike most medicines, vaccines don’t treat disease, but they can prevent it from spreading and making people sick by producing complete or partial immunity to a specific disease. And unlike medicines, a vaccine’s success is not based on efficacy and safety alone—public trust is paramount. Scientists estimate at least 60% of the population would need to have significant immunity in order to achieve “herd immunity” (where enough of the community is immune to prevent person-to-person spread).

In response to COVID-19, many in the global healthcare and life sciences industry are focused on developing vaccines as a powerful tool for stemming the pandemic. While vaccine development usually takes years, scientists and public health authorities are moving faster than ever before—at “warp speed”—with the goal of developing a safe, effective vaccine in less than a year.

Clinical trials for multiple vaccine candidates are enrolling participants at many sites across the world, with major participation in the U.S. The U.S. is not only the source of major biomedical innovation, but unfortunately we’re one of the few countries producing large numbers of infections, a critical factor in researching the efficacy of a vaccine candidate.

Growing mistrust of vaccines represents a serious challenge. Especially in light of the speed of the research and development timeline, scientific transparency and increased public awareness of the clinical research process is critical. The Emergency Use Authorization authority of the FDA enhances concerns, because vaccines can be approved based on potential benefits and risks as well as proven benefits and risks. In usual times, the benefits and risks must be proven rather than “potential.” While the enormous mortality, illness and economic losses from COVID-19 justify this emergency authority for the FDA, public confidence becomes even more important.

How the clinical trials work

The FDA has determined that a viable COVID-19 vaccine must reduce new infections by at least 50% compared to a placebo in the trials. The trials are set up in a “classical fashion,” with all the standard division of responsibility and safeguards needed to allow researchers to draw valid scientific conclusions. The research occurs in phases, starting with pre-clinical research, then studies involving animal testing (Phase 0), followed by human trials focused on preliminary evidence of safety and efficacy (Phase I), and then by larger trials for safety and efficacy (or how effective the vaccine is at stimulating the immune system) (Phases II and III). A vaccine that fails at any of these stages cannot move forward. All of the developmental phases culminate with a large, pivotal Phase III Trial that is intended to prove safety and efficacy.

Another critical element to achieving scientific validity is organizing the trials to ensure that bias—intellectual, financial or political—is prevented from leading to untrue results, putting both trial participants and future patients at risk. One way we reduce bias is by ensuring that trial participant assignment to active vaccine or placebo is randomly allocated, and that researchers and participants are “blinded.”

This means participants and researchers are unaware of which individuals have received the true vaccine or the control placebo until the study is finished. There are mechanisms to unblind individuals whether they receive a true vaccine or placebo to protect them, especially if they experience severe side effects that could be caused by the experimental vaccine. Research sites, led by experienced and qualified physicians, are responsible for getting informed consent, administering the vaccine or placebo, collecting data while monitoring participants’ well-being and attending to the needs of participants should health issues arise during the trial.

Researchers and participants knowing who has the vaccine or the placebo is only one of many possible sources of bias. The Food and Drug Administration (FDA), National Institutes of Health (NIH) and other trial experts have learned how to guard against these biases by designing the trials correctly.

Who’s involved?

In addition to trial design, there are numerous entities who play a role in ensuring the validity of trials. Here is a list of additional entities in place for planning, conducting, analyzing and interpreting COVID-19 vaccine trials:

Each manufacturer, which could be a pharma or biotech company, that has moved a vaccine candidate through the first few phases is known as the “sponsor” of the trial. Each sponsor is paired with a clinical research organization (CRO) or uses an internal equivalent of a CRO for trial operations.
  • What they do: CROs directly coordinate the work of the research sites in each trial and manage the trial database. The CROs are legally considered to be agents of the sponsor.
  • Access to data: The trial sponsors handle regulatory affairs and they are responsible for distributing and marketing should the vaccine prove safe and effective, but they have no knowledge of the trial results until they are unblinded for primary analysis. This helps assure that there is no manipulation of the trial results for financial or reputational gain.

Each trial has a Data Monitoring Committee (DMC), which is a standard practice for major trials with clinically important outcomes.
  • What they do: The DMC consists of experts in infectious disease, vaccine trials, biostatistics and ethics. The group receives reports from the Data Coordinating Center at the CRO that enables it to track the progress of the trial and to evaluate unblinded data at regular intervals.
  • Access to data: This group is responsible for monitoring the trial and data so no bias can creep in.The DMC analyzes trial data at pre-specified intervals, ranging from after ⅓ and ⅔ of total infections have occurred to ¼, half, ¾ through the number of infections, to assess efficacy of the vaccine. The DMC could recommend stopping the trial at one of these intervals for either a large treatment effect or a finding of futility (no chance of showing the needed treatment effect). The primary factor in the DMC decision is based on a prespecified statistical plan. At these stages, a much larger effect must be observed to declare efficacy and stop early than is required at the end of the trial (commonly known as a “statistical penalty”). These decisions are not just formulaic—the committee must consider all information at hand. The DMC is sworn to confidentiality until the trial is over.

Institutional Review Boards oversee the ethical conduct of each trial. There are both central IRBs paired with each trial, to monitor the overall trial, and local IRBs at every research site to oversee the local conduct of the research.
  • What they do: The IRB oversees the conduct of researchers and assures that processes are in place to protect participants.
  • Access to data: The IRBs are made up of experts in clinical research and ethics with no vested interest in the trial results—their responsibility is to the research participants. They do not have access to the data during the conduct of the trial, except for reporting of adverse events.

The FDA Vaccines and Related Biological Products Advisory Committee (VRBAC) reviews applications from the manufacturer prior to consideration of approval. VRBAC, like all FDA Advisory Committees, is only advisory; it does not make the decisions. In preparation for Advisory Committee meetings the Center for Biologics Evaluation and Research (CBER) staff analyzes the data independently of the manufacturer to provide perspective for public consumption so that the committee can discuss issues that might affect the final decision.
  • What they do: This committee includes experts in the field, representing medical, scientific, statistical and patient perspectives. They have a public deliberation about candidate vaccines and the trials that evaluate their safety and efficacy. The FDA full-time staff is responsible for making the final decision about whether the vaccine can be marketed and distributed, not the advisory committee. FDA does not commonly make a decision that is different than Advisory Committee recommendations, but it does happen.
  • Access to data: The FDA will have access to all data including the raw data, at the end of the trial. VRBAC and the public will have access to reports from the data prepared by the FDA and the sponsors.

The Centers for Disease Control and Prevention Advisory Committee on Immunization Practices (ACIP) includes public health and medical experts.
  • What they do: ACIP advises the CDC about the use of vaccines in the U.S. civilian population.
  • Access to data: ACIP will have full access to the data and conclusions from the FDA and sponsor(s) at the end of the trial.

Operation Warp Speed (OWS) is a special organization across the Federal government, formed specifically for COVID-19 because we are in an official national emergency.
  • What they do: OWS officials are aware of all the operations of the trials and have accountability for the overall strategy and funding allocation, but they do not know the trial results as the studies proceed.
  • Access to data: OWS leaders are “hands off” on the data and treatment assignment to avoid a biased, overly optimistic interpretation.

The Accelerating Covid-19 Therapeutic Interventions and Vaccines (ACTIV) trial network is being operated by the National Institutes of Health (NIH) with a highly experienced and savvy group of clinical trialists who have extensive experience with NIH-funded clinical trials.
  • What they do: They are managing the COVID-19 trial operations at a high level and organize the research sites for the trials in conjunction with the sponsors. As opposed to Warp Speed, which is in place to integrate logistics across the public and private sectors, the NIH is the main organization accountable for the science of the trials.
  • Access to trial data: By design, NIH officials have no knowledge of the evolving trial results. They need to be unbiased as they officiate the design and conduct of the trials and monitor the evolving science as the trials proceed.

It is important to reiterate that the full-time CBER staff has the primary responsibility for making the decision about whether a vaccine can be marketed. The rest of this trial machinery is intended to create an unbiased operation that will yield reliable findings at the end of the trial. Almost never in the history of the FDA has a political appointee over-ruled the Center Director in a decision about an individual product. Setting a precedent for political meddling with specific FDA product decisions would jeopardize the integrity of future FDA decisions, and both Peter Marks, the Center Director and Stephen Hahn, the Commissioner have publicly committed to “staying with the science.”

From the initial design to the final deployment, the vaccine clinical trials system is configured to have layers of accountability to prevent contamination of the study and should be impervious to biased influence by political, corporate or medical ideological meddling. With so many checks and balances in place, built up through decades of experience, any effort to alter the science will become a topic for public scrutiny.

Recently, concern has been expressed about the possibility that the Secretary of Health and Human Services or the President could overrule or preempt the Center Director and the Commissioner. While this is possible as the FDA is part of the Executive Branch of government, and it is a “chain of command”, ultimately a vaccine will need the confidence of the public, including the doctors, nurses and pharmacists who administer the vaccine. I am confident scientific facts about the vaccine will be known, allowing biomedical professionals to form a consensus about the balance of benefits and risks that will drive public perception. Efforts to override the consensus of scientists and health professionals will likely not succeed because the trials will be scientifically sound, the data will be publicly available and most people will not take a vaccine without a positive assertion from their trusted health professional. Hopefully, the trials will be dramatically positive and we can all have confidence not only in the trials, but also in the decisions based on the trial results.

- Robert M. Califf, MD, MACC, Head of Clinical Policy and Strategy for Verily and Google Health