Susan G Komen  
I've Been Diagnosed With Breast Cancer Someone I Know Was Diagnosed Share Your Story Join Us And Stay Informed Donate To End Breast Cancer
Home > Understanding Breast Cancer > Is it Effective? Is it Safe? The Importance of Scientific Evidence

  


Is it Effective? Is it Safe? The Importance of Scientific Evidence

 

Someone looking for a new car might look at test results in a consumer magazine to see whether a certain car is safe and reliable. In the same way, health care providers rely on scientific evidence to see how safe and effective a complementary therapy may be.

There are many complementary therapies. The quality of the scientific evidence varies from one therapy to another. For some therapies, there is good evidence on safety and effectiveness. For many others, there is limited evidence, making it hard to draw conclusions.

Weighing the scientific evidence

While there are many ways to assess the quality of the scientific evidence, most look at: 

  • The types of studies done
  • The number of studies done
  • The consistency of findings across studies 

Together, these three basic factors form the weight of evidence behind a therapy and help answer important questions about its safety and effectiveness.

Types of studies

There are many types of research studies. Some hold more weight than others when it comes to the importance of their results. In general, randomized controlled trials are considered the best type of study for learning whether a treatment is safe and effective. Prospective cohort studies and case-control studies also can be used to assess complementary therapies.

Study quality and study size (the number of people taking part in a study) are also important factors in assessing results. The results of a small, poorly-designed randomized controlled trial may be viewed as weaker than those of a larger, well-designed cohort or case-control study.

Learn more about different types of research studies.   

Number of studies

The greater the number of studies on a complementary therapy, the more evidence there is from which to draw conclusions about its risks and benefits. Of course, the type, quality and size of the studies are important too. One large, well-designed randomized controlled trial can be more compelling than 20 small studies. In general, though, the more studies there are on a therapy, the more health care providers and scientists are able to draw conclusions about its safety and effectiveness.

Consistency of findings

How consistent findings are across studies whether or not they tend to show the same resultsis important in weighing the evidence on a complementary therapy. Evidence is more compelling when most studies have similar results. When the results from half the studies show a therapy is effective and the results from the other half do not, it's hard to draw conclusions. It's similar to asking three friends what they thought of a new movie. If all three liked it, you have evidence the movie was good. If one friend liked it, one didn't and one thought it was OK, it's hard to know what to think.

Natural Standard's grading system for complementary therapies

Susan G. Komen for the Cure licenses the content on individual complementary therapies from Natural Standard. Within each discussion of a therapy, an A through F grading system (see below) is included to assess the weight of evidence for its effectiveness. "A" indicates strong evidence that a therapy is effective. "C" indicates unclear or mixed results. "F" indicates strong evidence that a therapy is not effective. And, "B" and "D" indicate the evidence falls in between the other grades.

Natural Standard evidence-based validated grading rationale™

  • Grades show the level of scientific evidence in support of a given therapy for a specific disease or symptom.
  • Expert opinion and folkloric precedent are not included in this assessment, and are found in a separate section of each monograph ("Strength of Expert Opinion and Historic/Folkloric Precedent").
  • Evidence of harm is considered separately; the grades below apply only to evidence of benefit.
 

 

Level of Evidence Grade 

Criteria 

A (strong positive scientific evidence)

Statistically significant evidence of benefit from more than two well-designed randomized controlled trials (RCTs) 

OR

Evidence from one properly conducted RCT AND one properly conducted meta-analysis

OR

Evidence from multiple RCTs with a clear majority of the properly conducted RCTs showing statistically significant evidence of benefit AND with supporting evidence from basic science, animal studies or theory.

B (positive scientific evidence)

Statistically significant evidence of benefit from one to two well-designed RCTs

OR

Evidence of benefit from more than one properly conducted meta-analysis

OR

Evidence of benefit from more than one cohort study, case-control study or non-randomized trial AND supporting evidence from basic science, animal studies or theory.

C (unclear scientific evidence)

Evidence of benefit from more than one small RCT(s) without adequate size, power, statistical significance or quality of design by objective criteria*

OR

Conflicting evidence from multiple RCTs without a clear majority of the properly conducted RCTs showing evidence of benefit or ineffectiveness

OR

Evidence of benefit from more than one cohort study, case-control study or non-randomized trial AND without supporting evidence from basic science, animal studies or theory

OR

Evidence of benefit only from basic science, animal studies or theory.

D (negative scientific evidence)

Statistically significant negative evidence (lack of evidence of benefit) from cohort studies, case-control studies or non-randomized trials, AND evidence from basic science, animal studies or theory suggesting a lack of benefit.

F (strong negative scientific evidence)

Statistically significant negative evidence (lack of evidence of benefit) from more than one well-designed* RCT.

Lack of evidence**

Unable to evaluate due to lack of adequate human data.

* Objective criteria are derived from validated instruments for evaluating study quality, including the five-point scale developed by Jadad and colleagues, in which a score below four is considered to show poorer quality of research methods [19].

** Listed separately in monographs in the "Historical or Theoretical Uses which Lack Sufficient Evidence" section.  

 

Growing evidence

New studies continue to add to our knowledge of complementary therapies. The weight of scientific evidence on many therapies is building quickly. As more studies are published, more refined conclusions can be drawn about the safety and effectiveness of these therapies.

Complementary therapies are not recommended in place of standard medical treatments. However, over time, some complementary therapies have become accepted as an integrated part of medical care. When complementary therapies are used with standard medical treatments, they are often called integrative therapies.

See a list of integrative and complementary therapies discussed in this section

Updated 06/22/12

previous  Working with Health Care Providers: Shared Decision-Making 
  Safety of Complementary Therapies  next