17 Sensitivity analyses

Input Primary and sensitivity summary tables.

Action Apply the recorded comparison rule.

Output Status counts and paired coefficient tables.

Next Create final figures.

17.1 Comparison setup

The six reported moderator models (m01–m05, m07) were repeated under three prespecified alternatives, giving 18 sensitivity models:

Analysis	Recorded change	Fixed comparison
Minimum sample size	Keep only contrasts with $n_1 + n_2 \geq 40$	Matching primary model
No phylogeny	Remove the phylogenetic random effect	Matching primary model
Study-system effect	Add a `sys_id` random intercept	Matching primary model

Each analysis changes one feature at a time. The response, moderator definitions, priors, study-level (ref_id) effect, and MCMC settings otherwise match the corresponding comparison model. The no-phylogeny analysis necessarily omits the phylogenetic effect; the minimum-sample-size analysis necessarily uses fewer contrasts.

The environmental-change model (m08) was also refit under each alternative, but it is excluded here because the primary m08 fit did not meet the convergence criteria (max $\hat{R}$ = 1.0143; see the m08 appendix). Its sensitivity fits are therefore not reported or counted.

17.2 At-a-glance results

The table below is generated from the latest synced model summaries. “Models available” should equal six for a complete analysis. “Changed coefficients” excludes intercepts and counts a coefficient when either:

its posterior median changes sign; or
its 95% credible interval changes from including zero to excluding zero, or vice versa.

This rule creates the displayed DIFFERS and consistent labels. The paired estimates are also printed in the appendices.

Code

sens_variant_summary_table()

Sensitivity-model availability, stability, and diagnostics.
Analysis	Models available	Changed coefficients	Max Rhat	Divergences	Status
Minimum total sample size (n₁ + n₂ ≥ 40)	6	3 of 34	1.015	133	complete
No phylogenetic random effect	6	5 of 34	1.013	0	complete
Study-system (sys_id) random effect	6	5 of 34	1.022	255	complete

17.3 Output columns

Each appendix begins with the exact model change and its convergence diagnostics, then places every sensitivity coefficient beside the corresponding primary coefficient:

The tables contain two submodels:

Location (mean lnM): effects on the expected magnitude of divergence.
Scale (log-SD): effects on between-observation variability, expressed on the log standard-deviation scale.

These are raw treatment-coded coefficients, not marginal means. For categorical moderators, the intercept is the reference-level estimate and every other coefficient is a difference from that reference. For continuous moderators, the non-intercept coefficient is the slope. Intercepts are shown for completeness but excluded from the summary change count.

Rows marked DIFFERS meet one of the two screening criteria above. Rows marked consistent do not; the label does not mean the estimates are numerically identical.

--- title: "Sensitivity analyses" --- ::: {.workflow} ::: {} **Input** Primary and sensitivity summary tables. ::: ::: {} **Action** Apply the recorded comparison rule. ::: ::: {} **Output** Status counts and paired coefficient tables. ::: ::: {} **Next** Create final figures. ::: ::: ```{r setup, include=FALSE} source(here::here("Scripts", "00_packages.R")) source(here::here("Scripts", "19_sensitivity_tables.R")) ``` ## Comparison setup The six reported moderator models (m01–m05, m07) were repeated under three prespecified alternatives, giving 18 sensitivity models: | Analysis | Recorded change | Fixed comparison | |---|---|---| | Minimum sample size | Keep only contrasts with $n_1 + n_2 \geq 40$ | Matching primary model | | No phylogeny | Remove the phylogenetic random effect | Matching primary model | | Study-system effect | Add a `sys_id` random intercept | Matching primary model | Each analysis changes **one feature at a time**. The response, moderator definitions, priors, study-level (`ref_id`) effect, and MCMC settings otherwise match the corresponding comparison model. The no-phylogeny analysis necessarily omits the phylogenetic effect; the minimum-sample-size analysis necessarily uses fewer contrasts. The environmental-change model (m08) was also refit under each alternative, but it is excluded here because the primary m08 fit did not meet the convergence criteria (max $\hat{R}$ = 1.0143; see the m08 appendix). Its sensitivity fits are therefore not reported or counted. ## At-a-glance results The table below is generated from the latest synced model summaries. “Models available” should equal six for a complete analysis. “Changed coefficients” excludes intercepts and counts a coefficient when either: 1. its posterior median changes sign; or 2. its 95% credible interval changes from including zero to excluding zero, or vice versa. This rule creates the displayed `DIFFERS` and `consistent` labels. The paired estimates are also printed in the appendices. ```{r sensitivity-summary} sens_variant_summary_table() ``` ## Output columns Each appendix begins with the exact model change and its convergence diagnostics, then places every sensitivity coefficient beside the corresponding primary coefficient: - [Minimum total sample size ($n_1 + n_2 \geq 40$)](appendix/sensitivity_min_n.qmd) - [No phylogenetic random effect](appendix/sensitivity_no_phylogeny.qmd) - [Study-system (`sys_id`) random effect](appendix/sensitivity_sys_id.qmd) The tables contain two submodels: - **Location (mean lnM):** effects on the expected magnitude of divergence. - **Scale (log-SD):** effects on between-observation variability, expressed on the log standard-deviation scale. These are raw treatment-coded coefficients, not marginal means. For categorical moderators, the intercept is the reference-level estimate and every other coefficient is a difference from that reference. For continuous moderators, the non-intercept coefficient is the slope. Intercepts are shown for completeness but excluded from the summary change count. Rows marked **DIFFERS** meet one of the two screening criteria above. Rows marked *consistent* do not; the label does not mean the estimates are numerically identical.