If you're curious about what is happening with your students, programs, or outcomes, start by looking at participation, counts, rates, patterns, and changes over time. This can help you understand the landscape before asking more complex questions.

Common approaches: Counts, percentages, rates, distributions, and trends.

Keep in mind: Make sure you know who is represented in the data, how the denominator was defined, and whether the way information was collected changed over time.

If you're curious about whether two or more groups of students have different outcomes or experiences, you can compare their results to see whether the differences are meaningful or could reasonably be due to chance.

Common approaches: Group comparisons such as t tests, ANOVA, and chi-square tests.

Keep in mind: A statistically significant difference is not necessarily a meaningful difference in practice. A difference also does not tell you what caused it.

If you're curious about whether students who participate in a program have different outcomes than students who do not, you can create a more comparable group by accounting for characteristics that may differ between participants and nonparticipants.

Common approaches: Matching, propensity-score methods, and adjusted comparisons.

Keep in mind: You can account only for differences you can measure. Students may still differ in ways you cannot observe or account for.

If you're curious about which student, program, or contextual factors are related to an outcome, you can examine several factors together to see which relationships remain after accounting for other measured differences.

Common approaches: Regression models.

Keep in mind: A factor being associated with an outcome does not mean it caused the outcome. The analysis also depends on having appropriate data and a well-specified model.

If you're curious not just about whether something happens, but when it happens, you can examine the timing of an outcome and account for the fact that students may be observed for different lengths of time.

Common approaches: Survival or time-to-event analysis.

Keep in mind: You need to clearly define when the clock starts, what counts as the outcome, and how students who have not yet experienced the outcome are handled.

If you're curious about how students change during or after an experience, you can look at multiple observations of the same students or cohorts to understand patterns of change.

Common approaches: Repeated-measures and longitudinal analysis.

Keep in mind: Students may leave the study or have different numbers of observations. Repeated observations of the same student also cannot be treated as completely independent.

If you're curious about how students experience a program, what they value, what gets in the way, or what might explain a pattern in your quantitative data, talking directly with students or examining their written responses can provide context that numbers alone cannot.

Common approaches: Interviews, focus groups, open-ended responses, and qualitative analysis.

Keep in mind: Consider whose experiences are represented, whose may be missing, and how responses were selected, analyzed, and interpreted.

If you're curious about both what happened and how or why students experienced it that way, you may need to bring different types of evidence together. Quantitative data can show patterns and outcomes, while qualitative or contextual evidence can help explain what may be behind them.

Common approaches: Mixed-methods approaches.

Keep in mind: Simply collecting different types of information does not make a project mixed methods. The evidence needs to be intentionally connected to answer the question.