Alternative power hex logo: a distribution of simulated estimates with a highlighted tail beyond an effect threshold

Power defined by an effect threshold

The alternative power framework summarizes how often simulated coefficient estimates meet a chosen effect threshold. Its logo represents the distribution of estimates across replications, with a highlighted tail beyond a threshold.

For a nonnegative threshold, simglm computes the proportion of estimates greater than or equal to that threshold. For a negative threshold, it computes the proportion less than or equal to the threshold. Thresholds are on the fitted coefficient scale; for a logistic model, for example, this is the log-odds scale.

This quantity describes threshold attainment by point estimates. It does not require a significance test or a confidence interval to exclude zero, and should be interpreted separately from conventional hypothesis-test power.

Simulate, fit, and summarize

Specify thresholds as a named list under power$thresholds. Names must match the fitted model terms returned by broom::tidy(). Each term can have more than one threshold.

set.seed(321)

sim_arguments <- list(
  formula = y ~ 1 + x,
  fixed = list(
    x = list(var_type = "continuous", mean = 0, sd = 1)
  ),
  error = list(variance = 1),
  sample_size = 100,
  reg_weights = c(0, 0.3),
  model_fit = list(model_function = "lm"),
  extract_coefficients = TRUE,
  replications = 500,
  power = list(thresholds = list(x = c(0.2, 0.3, 0.4)))
)

simulation_results <- replicate_simulation(sim_arguments)

threshold_results <- compute_statistics(
  simulation_results,
  sim_arguments,
  power = FALSE,
  type_1_error = FALSE,
  precision = FALSE,
  alternative_power = TRUE
)

threshold_results[
  threshold_results$term == "x",
  c("term", "threshold", "alt_power", "replications")
]
#> # A tibble: 3 × 4
#>   term  threshold alt_power replications
#>   <chr>     <dbl>     <dbl>        <dbl>
#> 1 x           0.2     0.826          500
#> 2 x           0.3     0.526          500
#> 3 x           0.4     0.162          500

Here the generating coefficient for x is 0.3. The alt_power column reports the fraction of its simulated estimates reaching each requested threshold. As the positive threshold increases, the fraction meeting it cannot increase. The 500 replications keep this example quick; use more replications when greater Monte Carlo precision is needed.

For an effect in the negative direction, specify negative thresholds for that term. A threshold of -0.2 counts estimates at or below -0.2. A threshold of zero uses the nonnegative rule and counts estimates at or above zero.

Using the framework with other models

The same threshold summary can be used with the package’s other simulation designs. See Tidy simulation for the broader workflow and Spline simulation for thresholds applied to individual spline basis coefficients.