A recently published paper on connections between prostate cancer, nut consumption, and mortality has served as a useful example of how to evaluate the abstracts of scientific studies. All studies are not created equal and, although weaker or smaller studies still have their place and purpose, problems can arise when scientists’ reach exceeds their grasp. This prostate cancer study, for instance, is essentially negative but the abstract has been written in a way that tries to trump up the findings and misrepresents them. By going over the abstract and the warning signs therein, a better understanding of what to be on the watch for can be gleaned.
Prostate Cancer and Nuts Study: Summary
- 47,299 men from the Health Professionals Follow-up Study were the subjects of the research
- 26 years of data were obtained that tracked how incidences of prostate cancer, nut consumption following diagnosis, mortality (death) from prostate cancer, and mortality from all other causes
- There were 6,810 incidences of prostate cancer during the length of the study
- No correlation was found between nut consumption, prostate cancer incidence, and mortality from prostate cancer
- Those who consumed nuts five or more times per week had a 34% lower all-cause mortality when compared to those who ate nuts less than once per month
- The conclusion is that frequent nut consumption after prostate cancer diagnosis is associated with significantly reduced overall mortality
Breaking Things Down
For the record, there does not appear to be anything wrong with the findings themselves. The observations about mortality rates are likely accurate. However, the way this figure is stressed and presented suggests that there is an attempt at spinning negative findings into more positive ones. Make no mistake—this is a negative study. No connection between nut consumption and either the development or survivability of prostate cancer was discovered, which was the main objective of the study. To understand why the abstract’s conclusions don’t change this, consider the following:
Missing Numbers
The abstract discloses the number of participants and the number of prostate cancer cases, but not the number of deaths that occurred from all causes or from prostate cancer overall. Furthermore, no raw numbers are provided for the men who ate nuts one or less times per month or five or more times per week. Instead, all the reader is presented with is the statistic that there was 34% lower mortality. This sounds impressive, but without knowing what the numbers behind the percentage are, it is ultimately meaningless.
Subgroup Analysis
A subgroup is a section of the data that gets carved off and examined separately rather than as part of the whole. In this case, the subgroups are the “five or more servings of nuts per week” and “less than one serving per month” groups. Now, subgroup analysis does have its use and when properly employed it is a valid and useful analytical tool. However, subgroups were not set beforehand, which means this analysis was not originally intended. This sort of added-on subgroup analysis is sometimes done as a way to spin positive results out of an inherently negative study. In this case, the emphasis on those who ate the most nuts with those who ate the least with regards to overall mortality.
Plausibility
This is trickier to assess but needs to be kept in mind. The less plausible a finding is, the more likely it was the result of statistical noise. In the case of this abstract, recall how a meaningful difference in mortality was only found between the highest and lowest ends of nut consumption. The fact that no effective comparison exists between one, two, three, or four servings of nuts per week when compared to one or fewer servings per month means the plausibility of five servings reducing mortality much lower.
Bottom Line
- The presence of the above issues does not automatically mean a study is worthless or being misrepresented, but they should be considered warning flags and reasons for extra caution and scrutiny
- In the case of the nut and mortality study, the focus on subgroup analysis, lack of underlying numbers, negative findings on the main observation, and lack of significant findings from moderate levels of nut consumption suggest the results are more likely to be spurious
- While there may be innocent explanations for these oddities, their presence makes the reliability of the findings much murkier than would be preferable
Source for Today’s Article:
Wang, W., “Nut consumption and prostate cancer risk and mortality,” British Journal of Cancer, 2016; 10.1038/bjc.2016.181.