Skip to main content

A goal of the Educational Programs Committee (EPC) in choosing papers for the AAEP annual meeting is to combine the best available clinical research with clinical experience and expertise to meet the needs of our patients. Below are the criteria the EPC will use when scoring papers. The endpoint is determination of the Overall Score for each abstract.  To do this the following components should be evaluated separately and then as a conglomeration: Study Design, Study Quality, Innovation & Impact, Practicality, and Manuscript Quality.  Please bear in mind that Scientific papers, How-To papers, and Review papers will be ranked against only papers of the same type. Please read through the criteria in its entirety before beginning to evaluate your assigned papers. 

1. Study Design - multiplied by a weighting factor of 1. Only Section Facilitators Score this category

Scientific Papers - 10 point scale 

  • For papers that utilize a naturally occurring disease or condition model use Flow Chart A below. 

  • For papers that utilize an experimentally induced disease or condition model use Flow Chart B below. 

  • To assist identification of the study design category use “Questions to Assist Flow Chart Navigation” and “Definitions for Study Design Categories” below. 

  • “Other or Miscellaneous Studies” categories are considered scientific papers with their respective point assignments listed under “Questions to Assist Flow Chart Navigation” below. 

  • Reviewers should keep in mind that assessment of how WELL the study was carried out is assessed with the study “quality score." The study design score is just an assessment of what type of study was performed. 

How to or case studies manuscripts – 6 point scale (use Flow Chart A)

Review articles – Review articles do not receive a score in the study design category.   

Business Papers - Business papers do not receive a score in the study design category. 

Flowchart

2. Questions to Assist Flow Chart Navigation: 

1. Is the hypothesis or clinical condition in question being evaluated by utilizing a naturally occurring disease/condition model or with an experimentally induced disease/condition model?

  •    If naturally occurring - use Flow Chart A 

  •    If experimentally induced - use Flow Chart B 

  •    If the submitted paper does not fall into one of the above categories see the "Others Categories" below.

Flow Chart A (naturally occurring disease/condition) 

2. Are controls used? 

   If yes - you have four choices: 

  •   If the study is a comparative randomized study that is blinded - 10 points  Note: Comparative studies usually evaluate the efficacy and/or safety of a therapeutic or preventive intervention 

  •   If the study is a comparative randomized study that is not blinded - 7 points 

  •   If the study follows over time, similar groups and evaluates how certain risk factors affect a certain outcome this is likely a prospective cohort study - 8 points 

  •   If the study utilizes medical records to determine how certain risk factors affect a certain outcome this is likely a retrospective cohort or case controlled study - 6 points

If no - (controls are not used) you have five choices: 

  •  If the study is a comparative randomized study that is blinded - 7 points 

  •  If the study is a comparative randomized study that is not blinded - 5 points 

  •  If the how to description of a therapeutic or diagnostic technique is a series of more than 10 case studies - 6 points 

  •  If the how to description of a therapeutic or diagnostic technique is a series of 5 - 10 case studies - 3 points 

  •  If the how to description of a therapeutic or diagnostic technique is a series of less than 5 case studies - 1 point

Flow Chart B (experimentally induced disease/condition) 

3. Are controls used? 

If yes - you have two choices:

  • If the study is a comparative randomized study that is blinded - 8 points 

  • If the study is a comparative randomized study that is not blinded - 6 points    

If no - (controls are not used) you have two choices: 

  • If the study is a comparative randomized study that is blinded - 6 points 

  • If the study is a comparative randomized study that is not blinded - 4 points

4. Other Miscellaneous Scientific Study Categories:

a. Meta-analysis - 10 points 
b. Analytical comparisons of products - 5 points 
c. Technique or diagnostic test comparisons - 5 points (e.g. x-ray versus MRI for diagnostic accuracy of a certain condition) 
d. Pharmacokinetic studies - 5 points

3. Definitions for Study Design Categories

1. Randomized Comparative Study - These studies evaluate and compare the efficacy and/or safety of a therapeutic or preventive intervention.

2. Observational Epidemiological Studies - These studies test hypothesizes regarding risk and incidence among exposed and unexposed populations. 

 - Prospective Cohort Study - A study that follows over time, a group of similar individuals (cohorts) who differ with respect to certain factors under study, to determine how these factors affect rates of a certain outcome (that has not yet occurred). See diagram below.

Cohort

 

 - Case Controlled Study - A study used to identify factors that may contribute to a medical condition/disease by comparing subjects who have that condition (cases) to subjects who do not have the condition (controls). See diagram below. Cohort

 - Case Controlled Study - A study used to identify factors that may contribute to a medical condition/disease by comparing subjects who have that condition (cases) to subjects who do not have the condition (controls). See diagram below. 

cohort

4. "How To" Manuscripts 

  • Case Study Series - A report of the medical history and outcomes on multiple patients with a specific condition receiving a treatment or diagnostic procedure. 

  • Case Report or Expert Opinion - A report of the medical history and therapeutic or diagnostic outcome of a single patient or the description/technique for the treatment or diagnosis of a specific condition based on experience and expert opinion.

5. Miscellaneous Scientific Studies Categories (those without the utilization of a therapeutic or preventive intervention) 

  • Meta-analysis - A statistical, systematic review contrasting and combining results from different studies, in hope of identifying patterns among study results, sources of disagreement among those study results or other interesting relationships that may come to light in the context of multiple studies. 

  • Analytical Comparisons of Products - For example, these studies may compare the active pharmaceutical ingredient in various compounded products. 

  • Pharmacokinetic or Pharmacodymanic Studies

  • Technique or Diagnostic Test Comparisons - For example, evaluating the specificity or sensitivity of radiography versus MRI for diagnostic accuracy of a known condition.

6. Study Quality - 10 points multiplied by a weighting factor of 2 

Assign the various scores: 
High quality = 10 
Moderate quality = 8 
Low-moderate = 6 
Low =
Poor = 1  

Scientific Papers: 
Specific factors to consider include: 

  • Appropriate study numbers and subjects 

  • Appropriate control population 

  • Appropriate use of statistics 

  • Clearly defined hypothesis 

  • Clearly defined outcome measures/end points 

  • Appropriate and well thought out inclusion/exclusion criteria for study subjects 

  • Clarity 


How to or case studies:   
Specific factors to consider include: 

  • Clarity including use of appropriate photos, diagrams, graphs or tables,  

  • Usefulness to the practitioner 

  • Consideration of potential adverse effects/risks/safety of the treatment or procedure 

  • Case management and follow-up  

  • Clinical relevance  

  • Applicability (topics that can be widely applied should receive a higher point value than those topics with a narrow scope of use) 

Review articles: 
Specific factors to consider include 

  • Organization  

  • Clarity 

  • Completeness of literature review   

  • Completeness of reference list (text books versus peer-reviewed) 

  • Clinical relevance  

  • Take home message and conclusions 

  • Interpretation of past work, including balanced treatment of the evidence (i.e. pros and cons of research that is reviewed, discussion of WHY certain results may have been achieved)  

  • Relevance to audience

  • Importance of Subject   

Business Papers - Will be similar to “How To’s” or Review papers, so please refer to these guidelines.

7. Innovation and Impact - 10 points multiplied by a weighting factor of 0.5 

Goals of category:   
To keep unorthodox ideas unsubstantiated by data that are potentially dangerous from being disseminated and to reward papers by value related to originality of the idea and how it extends and supports current knowledge.  The potential importance of a paper should also be considered when giving a score. 

High quality = 10 Dazzling new idea that may differ from current thought about a given topic (paradigm shifting rather than extending current concepts), but is well substantiated by valid data, scientific rationale, and reasoning 
Moderate quality = 8 Good idea that is well executed and advances or extends current knowledge by adding new information (but not paradigm shifting) and is well substantiated by valid data. 
Low-moderate = 6 Paper that enhances value of current knowledge by further validation but does not necessarily contribute new information. 
Low = 4 Interesting idea that extends current knowledge, but is not well substantiated by valid data.  
Poor = 2 Potentially valuable idea that challenges the current paradigm but is minimally supported by data.  
Unsubstantiated by data = 0 Idea that challenges the current paradigm but is supported by neither valid data nor scientific reasoning and could potentially be dangerous. 

8. Practicality - 10 points multiplied by a weighting factor of 1.5 

Reviewers should consider the clinical relevance of the information presented in the paper.  Can the findings be put into practice immediately or in the near future?  Can this information or technique be used in a variety of practice settings or by a variety of practitioners?   

High = 8-10  

Moderate = 6- 7  

Low-moderate = 4-5 

Low = 1- 3 

9. Manuscript quality - 10 points multiplied by a weighting factor of 0.5  

High = 8- 10 
No or extremely minimal editing is needed 

Moderate = 6- 7 
Few minor changes including grammar, reference list format 

Low-moderate = 4- 5 
Editing may include moderate changes to style and flow, spelling, typographical errors, additional support figures, moderate translation issues 

Low = 1- 3 
Editing will require major revisions, clarifications and re-write.  Paper does not follow AAEP Submission Guidelines (No Take Home message, incorrect format, obviously did not read guidelines) 

Include in this score is quality of accompanying photographs, tables, graphs, radiographs, and images. Submitters have read instructions to authors and have followed the format.