Skip to main content Skip to navigation

Evaluating Research with Critical Appraisal

PDF

By: Eileen Wafford, Research Librarian

Critical appraisal (CA) is an integral step in the Evidence-Based Medicine process. CA requires readers to systematically examine a research article to assess the validity of the research methodology, identify bias, and evaluate the results and interpretations. Need help with CA resources? Galter Health Sciences Library & Learning Center has you covered.

back to top
 

PICO and the Research Question

PICO is a framework designed to help you create a well-built question. Identifying the research question and PICO components early in the appraisal process makes it easier to pinpoint important concepts and gauge the potential relevancy of an article. The following questions can help you evaluate a study using PICO.

P- Population or Problem

  • What are important characteristics and demographic information about patients, population, or problem under investigation?
  • Did the authors discuss the sample size and assignment of participants?

I - Intervention

  • Did the authors explicit state the intervention (treatment, procedure, diagnostic test, or prognostic factors) of interest?

C - Comparator

  • If applicable, did the authors state the comparison treatment or mention a placebo?

O - Outcome

  • Did the authors clearly state the primary and secondary outcomes of interest and describe how they measured the outcomes?
  • Did the author addressed potential confounders?
back to top
 

Question type and study design

With the key concepts and the research question identified, check to see if the investigators used an appropriate study design for that type of question. As the study design shapes all aspects of the study, an inappropriate study design is a major indicator of flawed research.

There are several classification systems based on the Levels of Evidence hierarchy with recommendations on which types of studies are appropriate for a research question. The Centre for Evidence-Based Medicine (CEBM) presents a levels of evidence table that guides you on the types of studies that are appropriate for a question.

Like many levels of evidence hierarchies, the CEBM Levels of Evidence offers recommendations and not requirements. As readers, you must evaluate the content of the article to determine the quality of the data and reliability of the results.

back to top
 

Understanding the Results

Evaluating the results of a research study can be the most intimidating part of critical appraisal. Simply put, the results section is where the authors provide quantitative and descriptive information about the outcomes in the study.

As we appraise the results, we should see data that reports:

  • Estimates: values that convey the strength of associations or relationships observed in the investigation. These values might include the relative risk, odds ratio, or sensitivity and specificity in diagnostic test studies.
  • Inference: values such as confidence intervals, p-value, and type I or type II errors that give readers an idea of the strength of the estimates and the ability to draw conclusions (or inferences) about the larger population.
  • Adjustment: data generated by models such as stratification, analysis of covariance, multivariate models, and logistic regression which address the effects of confounding variables.

Galter Library offers many resources to help reader understand issues that may affect the transparency or reproducibility of research findings.

back to top
 

Critical Appraisal Checklists

Several organizations have developed checklists to help readers critically appraise published studies. These checklists present criteria for evaluating the methodological quality of studies, including the research question, study design, and results. View checklists provided by CEBM, the Joanna Briggs Institute, and other organizations on our Tools for Reviewers page.

Ultimately, readers must weigh the strengths and weaknesses of a study and determine its usefulness to your setting whether it is research, clinical, or something in between. The tools listed above and available on the Reporting Research and Evaluating Studies GalterGuide can aid in the critical appraisal of medical research. Galter Library also offers a class on this, so check our classes schedule for the next one on the calendar.

back to top
 

Updated: September 25, 2023