Case Study: Sensitivity Analyses for Threshold Effects in Categorical Engagement

---

Purpose

Likert scale what if analysis was used to gauge how definitional choices for “High Intent” impact conclusions, three scenario analyses were conducted on the transformed Likert composites:

---

1. Strict Definition

Rule: A respondent counts as “High Intent” only if:

• Their composite score equals 3 (Agree), and
• Each individual item in that composite is rated ≥ 4 on the original 5‑point scale.

Implementation:

df <- df %>%
  mutate(strict_intent = if_else(
    comp_usability == 3 & Q1 >= 4 & Q4 >= 4 & Q7 >= 4,
    1, 0))

Rationale & Impact:

• First, this rule zeroes in on truly enthusiastic respondents—those who “agree” overall and hit top marks on each question.
• Then, statistical tests (e.g., chi‑square, odds ratios) compare strict_intent rates by subgroup.
• Result: You typically see smaller High Intent rates (e.g., 47%) but larger effect sizes (e.g., OR = 2.8), since the contrast between groups is sharper.

2. Neutral‑Inclusive Definition

Rule: Anyone with a composite score ≥ 2 (Neutral or Agree) is “High Intent.”

Implementation:

df <- df %>%
  mutate(inclusive_intent = if_else(comp_usability >= 2, 1, 0))

Rationale & Impact:

• Moreover, this broader rule captures both neutral and positive attitudes, reflecting a more forgiving view of “intent.”
• Consequently, High Intent rates rise (e.g., 79%), but the statistical significance and effect sizes often shrink (e.g., OR = 1.4).
• However, this approach can understate differences if many neutrals exist in one group.

3. Continuous Modeling

Rule: Use the raw composite mean (1–3) directly in regression models rather than dichotomizing.

Implementation Examples:

• Ordinal Logistic Regression:

polr_fit <- MASS::polr(
  factor(comp_usability) ~ hospital_type + covariates,
  data = df, Hess=TRUE)

Linear Regression:

lm_fit <- lm(comp_usability ~ hospital_type + covariates, data = df)

Rationale & Impact:

• Finally, continuous models preserve the full granularity of the composite score, avoiding information loss from cut‑points.
• In practice, these models yield robust odds ratios (e.g., OR = 2.1 per one‑unit increase) and allow easy inclusion of covariates.
• Furthermore, continuous analysis reduces bias from arbitrary thresholds and enhances statistical power.

---

Implementation (whatif_scenarios.R) for Likert scale what if analysis

df <- read_csv(\"likert_masterchart.csv\")
# Scenario 1
df <- df %>%
  mutate(strict_intent = if_else(comp_usability==3 & Q1>=4 & Q4>=4 & Q7>=4, 1, 0))
# Scenario 2
df <- df %>% mutate(inclusive_intent = if_else(comp_usability>=2, 1, 0))
# Scenario 3 uses comp_usability directly

---

Results Comparison from Likert scale what if analysis

Scenario	High Intent Rate	Odds Ratio (95 % CI)	χ² p-value
Baseline	58 %	–	< 0.001
Strict	47 %	OR=2.8 (2.0–3.9)	< 0.001
Inclusive	79 %	OR=1.4 (1.1–1.9)	0.047
Continuous	–	OR=2.1 (1.4–3.2)	–

• Strict Scenario: Amplified group differences, yielding larger effect sizes.
• Inclusive Scenario: Diluted significance, showing threshold choice can flip conclusions.
• Continuous Models: Maintained robust ORs and facilitated covariate adjustments.

---

Recommendations from Likert scale what if analysis

• Pre-Register Thresholds: Avoid post-hoc definitional bias.
• Report Multiple Scenarios: Transparently show how definitions affect key findings.
• Leverage Continuous Models: Preserve full information for nuanced inference.

---

Want to explore more PhD-level case studies? Check out our Comprehensive Case Studies on PhD Statistical Analysis guide page.