R/api_process_extract.R
wait_for_extract.RdWait for an extract request to finish by periodically checking its status via the IPUMS API until it is complete.
is_extract_ready() is a convenience function to check if an extract
is ready to download without committing your R session to waiting for
extract completion.
Learn more about the IPUMS API in vignette("ipums-api").
wait_for_extract(
extract,
initial_delay_seconds = 0,
max_delay_seconds = 300,
timeout_seconds = 10800,
verbose = TRUE,
api_key = Sys.getenv("IPUMS_API_KEY")
)
is_extract_ready(extract, api_key = Sys.getenv("IPUMS_API_KEY"))One of:
An ipums_extract object
The data collection and extract number formatted as a string of the
form "collection:number" or as a vector of the form
c("collection", number)
An extract number to be associated with your default IPUMS
collection. See set_ipums_default_collection()
For a list of codes used to refer to each collection, see
ipums_data_collections().
Seconds to wait before first status check. The wait time will automatically increase by 10 seconds between each successive check.
Maximum interval to wait between status checks.
When the wait interval reaches this value, checks will continue to
occur at max_delay_seconds intervals until the extract is complete or
timeout_seconds is reached. Defaults to 300 seconds (5 minutes).
Maximum total number of seconds to continue waiting for the extract before throwing an error. Defaults to 10,800 seconds (3 hours).
If TRUE, print status updates to the R console at the
beginning of each wait interval and upon extract completion.
Defaults to TRUE.
API key associated with your user account. Defaults to the
value of the IPUMS_API_KEY environment variable. See
set_ipums_api_key().
For wait_for_extract(), an
ipums_extract object containing the extract
definition and the URLs from which to download extract files.
For is_extract_ready(), a logical value indicating
whether the extract is ready to download.
The status of a submitted extract will be one of "queued", "started",
"produced", "canceled", "failed", or "completed".
To be ready to download, an extract must have a "completed" status.
However, some requests that are "completed" may still be unavailable for
download, as extracts expire and are removed from IPUMS servers after a set
period of time (72 hours for microdata collections, 2 weeks for IPUMS NHGIS).
Therefore, these functions also check the download_links field of the
extract request to determine if data are available for download. If an
extract has expired (that is, it has completed but its download links are
no longer available), these functions will warn that the extract request
must be resubmitted.
download_extract() to download an extract's data files.
get_extract_info() to obtain the definition of a submitted extract request.
my_extract <- define_extract_ipumsi(
description = "Botswana data",
samples = c("bw2001a", "bw2011a"),
variables = c("SEX", "AGE", "YEAR")
)
if (FALSE) {
submitted_extract <- submit_extract(my_extract)
# Wait for a particular extract request to complete by providing its
# associated `ipums_extract` object:
downloadable_extract <- wait_for_extract(submitted_extract)
# Or by specifying the collection and number for the extract request:
downloadable_extract <- wait_for_extract("ipumsi:1")
# If you have a default collection, you can use the extract number alone:
set_ipums_default_collection("ipumsi")
downloadable_extract <- wait_for_extract(1)
# Use `download_extract()` to download the completed extract:
files <- download_extract(downloadable_extract)
# Use `is_extract_ready()` if you don't want to tie up your R session by
# waiting for completion
is_extract_ready("usa:1")
}