Returns metadata about objects in a bucket. This function is a wrapper to
`aws.s3::get_bucket_df` but filters out the desired Keys. If there are more
than 1000 objects, it makes iterative calls to the AWS S3 API to retrieve the
metadata for all versions. If the function is called from an interactive
session, it invokes a data viewer (View) with the search
results.
search_datalake(
...,
bucket_name = mfe_datalake_bucket,
object_versions = FALSE,
ncores = 1
)Patterns to look for. Each argument can be a character string or a
regex pattern. If multiple arguments are passed only Keys that match all
patterns are returned. Strings are passed to coll
and it ignores whether it is lower or upper case. If you want to search
using regex construct the pattern using regex (see
examples).
Name of the bucket to connect. By default, it uses the Ministry for the Environment data lake for environmental reporting "mfedlkinput".
Logical. Whether to include object version IDs in the search
a data frame with metadata for selected objects
if (FALSE) { # \dontrun{
# return all objects
search_datalake()
# search for a word
search_datalake("temperature")
# search using regex
search_datalake(stringr::regex("^a"))
# search tidy datasets for atmosphere and climate 2020
search_datalake("tidy", "climate", "2020")
# search tidy datasets with versions for atmosphere and climate 2020
search_datalake("tidy", "climate", "2020", object_versions = TRUE, ncores = 4)
} # }