> Is it Effective? Is it Safe? The Importance of Scientific Evidence
Someone looking for a new car may turn to test results in a consumer magazine to see how safe or reliable a certain car is. In the same way, health care providers and other health professionals turn to the scientific evidence to see how safe and effective complementary therapies may be.
Because CAM is such a broad field, the quality of this evidence can vary from one therapy to another. For some therapies there is good evidence on safety and effectiveness. For many others there is much less evidence, making it hard to draw any conclusions.
While there are many ways to assess the quality of evidence in the scientific and medical fields, most have the same basic starting place: looking at the types of studies done, the number of studies done and the consistency of findings across studies.
Types of studies. There are many different types of research studies, some of which hold more weight than others when it comes to the importance of their results. In general, randomized controlled trials are considered the best type of study for determining how safe and effective a treatment is. Prospective cohort studies generally follow next, followed by case-control studies. (For more on the different types of studies, visit the Breast Cancer Research section.) In addition to specific study type, study quality and study size (the number of people taking part in a study) are also quite important. The results of a small, poorly designed randomized controlled trial may be viewed as weaker than those of a larger, well-designed cohort or case-control study.
Number of studies. The number of studies examining a therapy is an important marker for the amount of evidence that exists on its benefits and risks. The greater the number of studies, the more evidence there is from which to draw conclusions. Of course, the type, quality and size of studies are important as well. One large, well-designed randomized controlled trial can be more persuasive than 20 smaller studies. In general, though, the more studies there are on a therapy, the more information health care providers and scientists have to work with when drawing conclusions.
Consistency of findings. How consistent the results are across studies—whether or not they tend to show the same thing—is another important building block that makes up the evidence on a therapy. Evidence is much more compelling when most studies have the same general results than when the results from half the studies go in one direction and the results from the other half go in a different direction. It's similar to asking three friends what they thought of a new movie. If all three liked it, that's pretty good evidence that the movie was good. If one liked it, one didn't and one thought it was OK, it's hard to know what to think.
Together, these three basic factors form the weight of evidence behind a therapy and help answer important questions about its safety and effectiveness. The discussion of individual complementary therapies in this chapter of Understanding Breast Cancer shows the weight of evidence for effectiveness with an intuitive "A" through "F" grading system (see below). "A" indicates strong evidence that a therapy is effective. "C" indicates unclear or conflicting results. "F" indicates strong evidence that a therapy is not effective. And "B" and "D" represent the range of evidence in between.
Natural Standard evidence-based validated grading rationale™
- Grades reflect the level of available scientific evidence in support of a given therapy for a specific disease or symptom.
- Expert opinion and folkloric precedent are not included in this assessment, and are reflected in a separate section of each monograph ("Strength of Expert Opinion and Historic/Folkloric Precedent").
- Evidence of harm is considered separately; the below grades apply only to evidence of benefit.
|
Level of Evidence Grade
|
Criteria
|
A (Strong Scientific Evidence)
|
Statistically significant evidence of benefit from >2 properly randomized trials (RCTs), OR evidence from one properly conducted RCT AND one properly conducted meta-analysis, OR evidence from multiple RCTs with a clear majority of the properly conducted trials showing statistically significant evidence of benefit AND with supporting evidence in basic science, animal studies, or theory.
|
B (Good Scientific Evidence)
|
Statistically significant evidence of benefit from 1-2 properly randomized trials, OR evidence of benefit from >1 properly conducted meta-analysis OR evidence of benefit from >1 cohort/case-control/non-randomized trials AND with supporting evidence in basic science, animal studies, or theory.
|
C (Unclear or conflicting scientific evidence)
|
Evidence of benefit from >1 small RCT(s) without adequate size, power, statistical significance, or quality of design by objective criteria,* OR conflicting evidence from multiple RCTs without a clear majority of the properly conducted trials showing evidence of benefit or ineffectiveness, OR evidence of benefit from >1 cohort/case-control/non-randomized trials AND without supporting evidence in basic science, animal studies, or theory, OR evidence of efficacy only from basic science, animal studies, or theory.
|
D (Fair Negative Scientific Evidence)
|
Statistically significant negative evidence (i.e., lack of evidence of benefit) from cohort/case-control/non-randomized trials, AND evidence in basic science, animal studies, or theory suggesting a lack of benefit.
|
F (Strong Negative Scientific Evidence)
|
Statistically significant negative evidence (i.e. lack of evidence of benefit) from >1 properly randomized adequately powered trial(s) of high-quality design by objective criteria.*
|
Lack of Evidence**
|
Unable to evaluate efficacy due to lack of adequate available human data.
|
Growing Evidence
CAM is a very dynamic field. The pace of new studies coming out today surpasses that of five or ten years ago, which means the weight of scientific evidence on many complementary therapies is building very quickly. What was conventional wisdom about the safety or effectiveness of a therapy two or three years ago may change as new studies are completed. As with many areas of science, the research on CAM is continuously growing and as results from new, important studies are published, more refined conclusions can be drawn.
To see the list of individual complementary therapies discussed in this chapter, click here.
Updated 03/11/09
