2  Data preparation

Input data/PROCEEDv6.2_RatesDB.csv

Action Clean fields, apply filters, and add derived columns.

Output Rdata/data_clean/proceed_clean_filtered.rds

Next Calculate SAFE lnM effect sizes.

Code 
source(here::here("Scripts", "00_packages.R"))
source(here::here("Scripts", "01_paths.R"))
source(here::here("Scripts", "02_read_clean_data.R"))
source(here::here("Scripts", "03_filters.R"))
source(here::here("Scripts", "15_render_guard.R"))   # read-only: never rebuild the database

# The cleaned/filtered contrast dataset is cached as an .rds. The book READS it
# by default. Set to TRUE only to rebuild the database from the raw PROCEED CSV.
rebuild_clean_data <- FALSE

clean_path <- here::here("Rdata", "data_clean", "proceed_clean_filtered.rds")

This chapter produces the cleaned, filtered contrast dataset used throughout the book. A normal render reads the saved file. The rebuild chunk runs only when rebuild_clean_data <- TRUE.

2.1 Action: clean and filter

The steps applied when (re)building the dataset:

  1. read_proceed() - read the raw PROCEED v6.2 rates database.
  2. clean_proceed() - coerce grouping/categorical variables to factors.
  3. require_lnm_vars() - keep contrasts with the two-group summary statistics required for \(\ln M\): means, standard deviations, and sample sizes for both groups.
  4. filter_generations(max_gen = 300) - retain contrasts with a known, positive elapsed-generation value no greater than 300; missing generation information is excluded.
  5. add_derived_vars() - add n_total, log10_years, log10_generations.
Code 
dat_raw       <- read_proceed()
dat_clean     <- clean_proceed(dat_raw)
dat_filtered  <- require_lnm_vars(dat_clean)
dat_filtered  <- filter_generations(dat_filtered, max_gen = 300)
dat_lnm_input <- add_derived_vars(dat_filtered)

saveRDS(dat_lnm_input, clean_path)
cat("Rebuilt and saved cleaned dataset:", nrow(dat_lnm_input), "contrasts\n")

2.2 Check: load the saved dataset

Code 
if (!rebuild_clean_data) {
  pace_need(clean_path, "cleaned dataset")
  dat_lnm_input <- readRDS(clean_path)
  cat("Read cached cleaned dataset:", nrow(dat_lnm_input), "contrasts\n")
}
Read cached cleaned dataset: 7250 contrasts

2.3 Design coverage

Code 
dat_lnm_input |>
  dplyr::count(design, sort = TRUE)

2.4 Derived variables

Code 
dplyr::glimpse(
  dplyr::select(dat_lnm_input, n_total, log10_years, log10_generations)
)
Rows: 7,250
Columns: 3
$ n_total           <dbl> 41, 33, 41, 48, 47, 41, 37, 38, 46, 53, 52, 46, 42, …
$ log10_years       <dbl> 2.089905, 2.089905, 2.089905, 2.089905, 2.089905, 2.…
$ log10_generations <dbl> 2.089905, 2.089905, 2.089905, 2.089905, 2.089905, 2.…

2.5 Output: dataset summary

Code 
dat_lnm_input |>
  dplyr::summarise(
    k              = dplyr::n(),
    n_studies      = dplyr::n_distinct(ref_id),
    n_systems      = dplyr::n_distinct(sys_id),
    n_species      = dplyr::n_distinct(sp_ncbi),
    n_allochronic  = sum(design == "Allochronic",  na.rm = TRUE),
    n_synchronic   = sum(design == "Synchronic",   na.rm = TRUE),
    pct_has_gen    = round(100 * mean(!is.na(generations)), 1),
    pct_has_years  = round(100 * mean(!is.na(years) & years > 0), 1)
  )