Example Of Critical Analysis Of Quantitative Cohort Retrospective Study Critical Thinking

using CASP cohort checklist
using CASP cohort checklist
The following paper consists of a critical appraisal of a journal article by Duncan et al. (2012). The Critical Appraisal Skills programme (CASP) checklist for cohort studies was used (CASP, 2013).

A. Are the results of the study valid?

1. Did the study address a clearly focused issue?
The analysed study was a retrospective cohort study because data were collected in the past and obtained from a database, it was observational (there was no intervention), and several groups exposed to several levels of the risk factors were observed (Altman, 1990). The aims of the study were: 1. To determine the effect of Systemic Inflammatory Response (SIR) on plasma concentrations of several micronutrients, and 2. To provide guidance on the interpretation of routine clinical results. In regards to these aims, the researchers successfully defined the desired risk factor as SIR. As SIR is a difficult outcome to evaluate itself, C-reactive protein (CRP) has been proposed and is now considered a sensititve and objective biomarker of inflammation (Koenig et al., 1999). Furthermore, the outcomes were also successfully identified as the plasma concentrations of several micronutrients (zinc, selenium, copper, Vitamins A, E, D, B6 and C). It is clear that the study intended to detect a harmful and dose-dependent effect of plasma CRP concentrations on plasma micronutrients concentration.
2. Was the cohort recruited in an acceptable way?
The study used a large population of samples coming from a large population of patients from throughout Scotland, which might be considered as a representative sample, and all samples were considered in the analyses, both as a whole, or as a subgroup of only the first sample of each patient. It unclear why the researchers decided to take into account and report in the methodology the use of a complete set and a subset of the sample, whilst they only reported results of the global analyses. Moreover, as the researchers state in the discussion section, there might be some variability in the results because measurements come from different patients with different conditions, different severities, and different micronutrient concentrations in blood, but there might still be some dependence of the observations within patients. The researchers claim there is independence of observations, but perhaps a cluster analysis based on patients as clusters for measurements would have been more appropriate as oppose to analysis of all measurements at once (Kaufman & Rousseeuw, 2009).
3. Was the exposure accurately measured to minimise bias?
C-reactive protein has been validated as a biomarker for SIR in several conditions (Karadag et al., 2008; Koenig et al., 1999; Visser et al., 1999)., so it is assumed that it is objectively measuring what is intended to be measured (Holland, Smith, Eskenazi & Bastaki, 2003). The researchers chose to categorise CRP plasma concentrations in six different categories of different range. It is unclear why they chose this method, and if the choice of a different set of categories would have altered the results, so the presence of bias is in doubt. Moreover, there is no description of the choice of cut-off points for these categories, either. However, all subjects were taken into account for the categorisation process and the analyses.
4. Was the outcome accurately measured to minimise bias?
The researchers also used objective and previously validated methods for measuring the outcome, so they also measure what is intended to. They make it clear that there are several methods for measuring several concentrations of the micronutrients (within red blood cells, in other tissues, or plasma concentrations). They chose one of these methods and reported the coefficient of variability for each of them, which makes the tests comparable. This study was not designed to observe disease ocurrence (e.g. micronutrient deficiency), so researchers analysed the outcomes as continuous variables instead of binary. The measurement methods were similar for all observations. Since this is a retrospective cohort study, blinding was not performed and actually not needed.
5. (a) Have the authors identified all important confounding factors? (b) Have they taken account of the confounding factors in the design and/or analysis?
6. (a) Was the follow up of subjects complete enough? (b) Was the follow up of subjects long enough?
Although the design of the study seems to be retrospective cohort study, the analyses were performed cross-sectionally, and there was no specified follow-up time. Therefore, it is uncertain whether there was enough time to observe important changes in the relationship between the risk factor and the outcome. Consequently, loss of follow-up was not considered.

B. What are the results?

7. What are the results of this study?
Bottom line results indicate an inverse relationship between the risk factor and almost all of the outcomes. Although the results show a general inverse trend between CRP concentration categories and micronutrients plasma concentrations (except Vitamin E and copper), the association measured by Spearman’s correlation coefficient was not strong. Since there were multiple CRP categories instead of a binary reduction of data, no measures of association between exposed/non-exposed were reported. Mann-Whitney U tests were performed using ≤ 5 mg/L of CRP as baseline. It is also important to notice that the researchers chose a lower-than-usual p-value to detect significance in their results (p ≤ 0.01) due to multiple testing in microarray studies (Dudoit, Shaffer & Boldrick, 2003). However, this value seems to be arbitrary, and the use of other adjustment methods (e.g. Benjamini-Hochberg) would have been a better choice (Benjamini & Hochberg, 1995; Benjamini & Hochberg, 2000).
8. How precise are the results?
There are no confidence intervals provided, but it was probably because the study variables do not follow a normal distribution, so the researchers relied on non-parametric tests and reported the Interquartile Range instead of Confidence Intervals. Taking this fact into account, results show precision only on some of the outcomes and only for high categories of CRP.
9. Do you believe the results?
The researchers successfully identify that their study does not provide enough evidence for a causal relationship between SIR and micronutrients plasma concentration. Causality is plausible and reasonable, but evidence is not substancial to prove it. In fact, according to Bradford-Hill criteria (Hill, 1965; Höfler, 2005), there is a low-to-moderate strength of association, that seems to be specific for patients with high levels of SIR (except for Vitamin E and copper), and follows a biologic gradient. However, as the authors state, there might be only an indirect effect of SIR on micronutrients plasma concentrations (e.g. patients with chronic diseases and thus high SIR lose their appetite, and thus have lower concentrations of micronutrients in blood). There is plausability, coherence, and analogy involved. The researchers report some consistency with previous studies, but state that more evidence is yet needed, especially coming from RCT –if possible-. Finally, temporality is well established, but it is still possible to invert this temporality, and wonder if low concentrations of micronutrients in blood might –by some biological mechanism- exert an effect on CRP concentrations (Basu, Devaraj & Jialal, 2006). As it was previously reported, the results might have been biased, or confounded by age, sex, race, etc.
C. Will the results help locally?
10. Can the results be applied to the local population?
Even though the results are not entirely convincing, the rationale could be applied to the local population, especially to reduce iatrogenic events.
11. Do the results of this study fit with other available evidence?
The results fit with previous evidence cited by the authors, but the study does not provide sufficient evidence to recommend important changes in clinical practice.
12. What are the implications of this study for practice?
This observational study alone does not provide enough evidence to change current clinical practice. Other studies with higher levels of evidence might help establish stronger evidence on this topic. Although it seems plausible to rely on the results of this study for clinical practice, it is left at the discretion of the practitioner until further evidence is available.

References

Basu, A., Devaraj, S., & Jialal, I. (2006). Dietary factors that promote or retard inflammation. Arteriosclerosis, thrombosis, and vascular biology, 26(5), 995-1001.
Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological), 289-300.
Benjamini, Y., & Hochberg, Y. (2000). On the adaptive control of the false discovery rate in multiple testing with independent statistics. Journal of Educational and Behavioral Statistics, 25(1), 60-83.
Critical Appraisal Skills programme (CASP): Cohort Studies Tool. (2013). Available at http://media.wix.com/ugd/dded87_e37a4ab637fe46a0869f9f977dacf134.pdf. Accessed 11 March 2015.
Dudoit, S., Shaffer, J. P., & Boldrick, J. C. (2003). Multiple hypothesis testing in microarray experiments. Statistical Science, 71-103.
Duncan, A., Talwar, D., McMillan, D. C., Stefanowicz, F., & O'Reilly, D. S. J. (2012). Quantitative data on the magnitude of the systemic inflammatory response and its effect on micronutrient status based on plasma measurements. The American journal of clinical nutrition, 95(1), 64-71.
Hill, A. B. (1965). The environment and disease: association or causation?.Proceedings of the Royal Society of Medicine, 58(5), 295.
Höfler, M. (2005). The Bradford Hill considerations on causality: a counterfactual perspective. Emerging themes in epidemiology, 2(1), 11.
Holland, N. T., Smith, M. T., Eskenazi, B., & Bastaki, M. (2003). Biological sample collection and processing for molecular epidemiological studies.Mutation Research, 543, 217-234.
Hutchinson, W. L., Koenig, W., Fröhlich, M., Sund, M., Lowe, G. D., & Pepys, M. B. (2000). Immunoradiometric assay of circulating C-reactive protein: age-related values in the adult general population. Clinical Chemistry, 46(7), 934-938.
Kaufman, L., & Rousseeuw, P. J. (2009). Finding groups in data: an introduction to cluster analysis (Vol. 344). John Wiley & Sons.
Karadag, F., Kirdar, S., Karul, A. B., & Ceylan, E. (2008). The value of C-reactive protein as a marker of systemic inflammation in stable chronic obstructive pulmonary disease. European Journal of Internal Medicine, 19(2), 104-108.
Koenig, W., Sund, M., Fröhlich, M., Fischer, H. G., Löwel, H., Döring, A., & Pepys, M. B. (1999). C-reactive protein, a sensitive marker of inflammation, predicts future risk of coronary heart disease in initially healthy middle-aged men results from the MONICA (monitoring trends and determinants in cardiovascular disease) Augsburg cohort study, 1984 to 1992. Circulation,99(2), 237-242.
Lakoski, S. G., Cushman, M., Criqui, M., Rundek, T., Blumenthal, R. S., D'Agostino, R. B., & Herrington, D. M. (2006). Gender and C-reactive protein: data from the Multiethnic Study of Atherosclerosis (MESA) cohort. American heart journal, 152(3), 593-598.
Visser, M., Bouter, L. M., McQuillan, G. M., Wener, M. H., & Harris, T. B. (1999). Elevated C-reactive protein levels in overweight and obese adults.Jama, 282(22), 2131-2135.