Severity of publication bias needed to "explain away" results

Estimates the S-value, defined as the severity of publication bias (i.e., the ratio by which affirmative studies are more likely to be published than nonaffirmative studies) that would be required to shift the pooled point estimate or its confidence interval limit to the value q.

Usage

pubbias_svalue(
  yi,
  vi,
  sei,
  cluster = 1:length(yi),
  q = 0,
  model_type = "robust",
  favor_positive = TRUE,
  alpha_select = 0.05,
  ci_level = 0.95,
  small = TRUE,
  selection_ratio_max = 200,
  return_worst_meta = FALSE
)

svalue(
  yi,
  vi,
  q,
  clustervar = 1:length(yi),
  model,
  alpha.select = 0.05,
  eta.grid.hi = 200,
  favor.positive,
  CI.level = 0.95,
  small = TRUE,
  return.worst.meta = FALSE
)

Arguments

yi: A vector of point estimates to be meta-analyzed.
vi: A vector of estimated variances (i.e., squared standard errors) for the point estimates.
sei: A vector of estimated standard errors for the point estimates. (Only one of vi or sei needs to be specified).
cluster: Vector of the same length as the number of rows in the data, indicating which cluster each study should be considered part of (defaults to treating studies as independent; i.e., each study is in its own cluster).
q: The attenuated value to which to shift the point estimate or CI. Should be specified on the same scale as yi (e.g., if yi is on the log-RR scale, then q should be as well).
model_type: "fixed" for fixed-effects (a.k.a. "common-effect") or "robust" for robust random-effects.
favor_positive: TRUE if publication bias are assumed to favor significant positive estimates; FALSE if assumed to favor significant negative estimates.
alpha_select: Alpha level at which an estimate's probability of being favored by publication bias is assumed to change (i.e., the threshold at which study investigators, journal editors, etc., consider an estimate to be significant).
ci_level: Confidence interval level (as proportion) for the corrected point estimate. (The alpha level for inference on the corrected point estimate will be calculated from ci_level.)
small: Should inference allow for a small meta-analysis? We recommend always using TRUE.
selection_ratio_max: The largest value of selection_ratio that should be included in the grid search. This argument is only needed when model_type = "robust".
return_worst_meta: Should the worst-case meta-analysis of only the nonaffirmative studies be returned?
clustervar: (deprecated) see cluster
model: (deprecated) see model_type
alpha.select: (deprecated) see alpha_select
eta.grid.hi: (deprecated) see selection_ratio_max
favor.positive: (deprecated) see favor_positive
CI.level: (deprecated) see ci_level
return.worst.meta: (deprecated) see return_worst_meta

Value

An object of class metabias::metabias(), a list containing:

data: A tibble with one row per study and the columns yi, yif, vi, affirm, cluster.
values: A list with the elements selection_ratio, selection_tails, model_type, favor_positive, alpha_select, ci_level, small, k, k_affirmative, k_nonaffirmative.
stats: A tibble with the columns model, estimate, se, ci_lower, ci_upper, p_value. sval_est represents the amount of publication bias required to attenuate the pooled point estimate to q; sval_ci represents the amount of publication bias required to attenuate the confidence interval limit of the pooled point estimate to q.
fit: A list of fitted models, if any.

Details

To illustrate interpretation of the S-value, if the S-value for the point estimate is 30 with q=0, this indicates that affirmative studies (i.e., those with a "statistically significant" and positive estimate) would need to be 30-fold more likely to be published than nonaffirmative studies (i.e., those with a "nonsignificant" or negative estimate) to attenuate the pooled point estimate to q.

If favor_positive = FALSE, such that publication bias is assumed to favor negative rather than positive estimates, the signs of yi will be reversed prior to performing analyses. The returned number of affirmative and nonaffirmative studies will reflect the recoded signs.

If return_worst_meta = TRUE, also returns the worst-case meta-analysis of only the nonaffirmative studies. If model_type = "fixed", the worst-case meta-analysis is fit by metafor::rma.uni(). If model_type = "robust", it is fit by robumeta::robu(). Note that in the latter case, custom inverse-variance weights are used, which are the inverse of the sum of the study's variance and a heterogeneity estimate from a naive random-effects meta-analysis (Mathur & VanderWeele, 2020). This is done for consistency with the results of pubbias_meta(), which is used to determine sval_est and sval_ci. Therefore, the worst-case meta-analysis results may differ slightly from what you would obtain if you simply fit robumeta::robu() on the nonaffirmative studies with the default weights.

References

Mathur MB, VanderWeele TJ (2020). “Sensitivity analysis for publication bias in meta-analyses.” Journal of the Royal Statistical Society: Series C (Applied Statistics), 69(5), 1091--1119.

Examples

# calculate effect sizes from example dataset in metafor
require(metafor)
dat <- metafor::escalc(measure = "RR", ai = tpos, bi = tneg, ci = cpos,
                       di = cneg, data = dat.bcg)

##### Fixed-Effects Specification #####
# S-values and worst-case meta-analysis under fixed-effects specification
svals_fixed_0 <- pubbias_svalue(yi = dat$yi,
                                vi = dat$vi,
                                q = 0,
                                model_type = "fixed",
                                favor_positive = FALSE)

# publication bias required to shift point estimate to 0
svals_fixed_0$stats$sval_est
#> [1] "Not possible"

# and to shift CI to include 0
svals_fixed_0$stats$sval_ci
#> [1] 7.58431

# now try shifting to a nonzero value (RR = 0.90)
svals_fixed_q <- pubbias_svalue(yi = dat$yi,
                                vi = dat$vi,
                                q = log(.9),
                                model_type = "fixed",
                                favor_positive = FALSE)

# publication bias required to shift point estimate to RR = 0.90
svals_fixed_q$stats$sval_est
#> [1] 8.261336

# and to shift CI to RR = 0.90
svals_fixed_q$stats$sval_ci
#> [1] 3.406779

##### Robust Clustered Specification #####
svals <- pubbias_svalue(yi = dat$yi,
                        vi = dat$vi,
                        q = 0,
                        model_type = "robust",
                        favor_positive = FALSE)
summary(svals)
#> # A tibble: 1 × 2
#>   sval_est     sval_ci
#>   <chr>          <dbl>
#> 1 Not possible    4.17