Reading REDATAM files in R

Please read our article for the full context of this project (Open Access):

Vargas Sepúlveda, Mauricio and Barkai, Lital. 2025. “The REDATAM format and its challenges for data access and information creation in public policy.” Data & Policy 7 (January): e18. https://dx.doi.org/10.1017/dap.2025.4.

We start by downloading the Chilean Census 2017 from ECLAC website (link valid as of 2024-10-01):

url <- "https://redatam.org/cdr/descargas/censos/poblacion/CP2017CHL.zip"
zip <- "CP2017CHL.zip"

if (!file.exists(zip)) {
  download.file(url, zip, method = "wget")
}

Now we can extract the files:

# install.packages("archive")
dout <- basename(zip)
dout <- sub("\\.zip$", "", dout)
archive::archive_extract(zip, dir = dout)

You can use unzip() from base R, but this file in particular gave me an error. The archive package is a wrapper around ‘libarchive’ that provides multi-format archive and compression support.

The REDATAM files are now stored in the CP2017CHL directory. We can read the REDATAM dictionary file (DIC or DICX for this particular case):

library(redatam)

fout <- "chile2017.rds"

if (!file.exists(fout)) {
  chile2017 <- read_redatam("CP2017CHL/BaseOrg16/CPV2017-16.dicx")
  saveRDS(chile2017, fout)
} else {
  chile2017 <- readRDS(fout)
}

One of the many possibilities with this census is to obtain the number of houses with overcrowding. For this, the Secretary for Social Development and Family (Ministerio de Desarrollo Social y Familia) divides the number of people residing in a dwelling and the number of bedrooms in the dwelling, with the special case of adding one to studio apartments and similar units, and the result is discretized as follows.

No overcrowding \([0,2.5)\)
Mean \([2.5,3.5)\)
High \([3.5,4.9)\)
Critical \([5,Inf)\)

According to the census documentation in the previous ZIP file, this consists in dividing the variables cant_pers and p04 from the vivienda (housing) table to then discretize the result. The documentation also states that we must join the vivienda table with zonaloc (zones), area, distrito (district) and communa (municipality) to match each house with its corresponding municipality. This can be done with dplyr:

library(data.table)

# 1. Merge tables
overcrowding <- merge(
    chile2017$comuna[, .(ncomuna, comuna_ref_id)],
    chile2017$distrito[, .(distrito_ref_id, comuna_ref_id)],
    by = "comuna_ref_id"
)

overcrowding <- merge(
    overcrowding,
    chile2017$area[, .(area_ref_id, distrito_ref_id)],by = "distrito_ref_id"
)

overcrowding <- merge(
    overcrowding,
    chile2017$zonaloc[, .(zonaloc_ref_id, area_ref_id)],
    by = "area_ref_id"
)

overcrowding <- merge(
    overcrowding,
    chile2017$vivienda[
    !p04 %in% c(98, 99), .(zonaloc_ref_id, vivienda_ref_id, cant_per, p04)],
    by = "zonaloc_ref_id"
)

# 2. Calculate overcrowding metrics
overcrowding[, overcrowding := fifelse(p04 >= 1, cant_per / p04, cant_per / (p04 + 1))]

overcrowding[, overcrowding_discrete := fcase(
  overcrowding < 2.5, "No Overcrowding",
  overcrowding >= 2.5 & overcrowding < 3.5, "Mean",
  overcrowding >= 3.5 & overcrowding < 5, "High",
  overcrowding >= 5, "Critical"
)]

overcrowding <- overcrowding[, .N, by = .(comuna = ncomuna, overcrowding_discrete)]

Now we can filter for any municipality of our interest, for example:

overcrowding[comuna == "VITACURA"]

#      comuna overcrowding_discrete     N
#      <fctr>                <char> <int>
# 1: VITACURA       No Overcrowding 26752
# 2: VITACURA                  Mean   174
# 3: VITACURA                  High    18
# 4: VITACURA              Critical     9

overcrowding[comuna == "LA PINTANA"]

#        comuna overcrowding_discrete     N
#        <fctr>                <char> <int>
# 1: LA PINTANA       No Overcrowding 39163
# 2: LA PINTANA                  High  1112
# 3: LA PINTANA                  Mean  4522
# 4: LA PINTANA              Critical   497

Reading REDATAM files in R

Mauricio Vargas Sepulveda

2026-10-16