RasterDataSources.jl

RasterDataSources downloads raster data for local use or for integration into other spatial data packages, like Rasters.jl. The collection is largely focussed on datasets relevant to ecology, but will have a lot of crossover with other sciences.

Currently sources include:

Please open an issue if you need more datasets added, or (even better) open a pull request following the form of the other datasets where possible.

Retrieving data

Usage is generally via the getraster method - which will download the raster data source if it isn't available locally, or simply return the path/s of the raster file/s:

julia> using RasterDataSources

julia> getraster(WorldClim{Climate}, :wind; month=1:12)
12-element Array{String,1}:
 "/home/user/Data/WorldClim/Climate/wind/wc2.1_10m_wind_01.tif"
 "/home/user/Data/WorldClim/Climate/wind/wc2.1_10m_wind_02.tif"
 "/home/user/Data/WorldClim/Climate/wind/wc2.1_10m_wind_03.tif"
 "/home/user/Data/WorldClim/Climate/wind/wc2.1_10m_wind_04.tif"
 "/home/user/Data/WorldClim/Climate/wind/wc2.1_10m_wind_05.tif"
 "/home/user/Data/WorldClim/Climate/wind/wc2.1_10m_wind_06.tif"
 "/home/user/Data/WorldClim/Climate/wind/wc2.1_10m_wind_07.tif"
 "/home/user/Data/WorldClim/Climate/wind/wc2.1_10m_wind_08.tif"
 "/home/user/Data/WorldClim/Climate/wind/wc2.1_10m_wind_09.tif"
 "/home/user/Data/WorldClim/Climate/wind/wc2.1_10m_wind_10.tif"
 "/home/user/Data/WorldClim/Climate/wind/wc2.1_10m_wind_11.tif"
 "/home/user/Data/WorldClim/Climate/wind/wc2.1_10m_wind_12.tif"

Installation and setup

Install as usual with:

] add RasterDataSources

To download data you will need to specify a folder to put it in. You can do this by assigning the environment variable RASTERDATASOURCES_PATH:

ENV["RASTERDATASOURCES_PATH"] = "/home/user/Data/"

This can be put in your startup.jl file or the system environment.

RasterDataSources was based on code from the SimpleSDMDataSoures.jl package by Timothée Poisot.

getraster(source::Type, [layer]; kw...)

Download raster layers layers from the data source, returning a String for a single layer, or a NamedTuple for a Tuple of layers.

getraster provides a standardised interface to download data sources, and return the filename/s of the selected files.

getraster can be called with any RasterDataSource. See the docstrings of these types for the specific keywords and arguments. RasterDataSource with documented getraster usage are WorldClim, EarthEnv, CHELSA, AWAP, ALWB, ERA5, NCEP, and MODIS. All these implementations follow the template specified below.

Arguments

• source: defines the RasterDataSource and (if it there is more than one) the specific RasterDataSet from which to download data.
• layer: choose the named Symbol/s or numbered Int/s (for BioClim) layer/s of the data source. If layer is not passed, all layers will be downloaded, returning a NamedTuple of filenames.

Keywords

Keyword arguments specify subsets of a data set, such as by date or resolution. As much as possible these are standardised for all sources where they are relevent.

• date: DateTime date, range of dates, or tuple of start and end dates. Usually for weather datasets.
• month: month or range of months to download for climatic datasets, as Integers from 1 to 12.
• res: spatial resolion of the file, as a String with units, e.g. "10m".

Return values

The return value is either a single String, a Tuple/Array of String, or a Tuple/Array of Tuple/Array of String –- depending on the arguments. If multiple layers are specified, this may return multiple filenames. If multiple months or dates are specified, this may also return multiple filenames.

Keyword arguments depend on the specific data source. They may modify the return value, following a pattern:

• month keywords of AbstractArray will return a Vector{String} or Vector{<:NamedTuple}.
• date keywords of AbstractArray will return a Vector{String} or Vector{<:NamedTuple}.
• date keywords of Tuple{start,end} will take all the dates between the start and end dates as a Vector{String} or Vector{<:NamedTuple}.

Where date and month keywords coexist, Vector{Vector{String}} of Vector{Vector{NamedTuple}} is the result. date ranges are always the outer Vector, month the inner Vector with layer tuples as the inner NamedTuple. No other keywords can be Vector.

RasterDataSource

Abstract supertype for raster data collections.

ALWB{Union{Deciles,Values},Union{Day,Month,Year}} <: RasterDataSource

Data from the Australian Landscape Water Balance (ALWB) data source.

See: www.bom.gov.au/water/landscape

The dataset contains NetCDF files. They have a time dimension so that multiple dates are stored in each file.

The available layers are: (:rain_day, :s0_pct, :ss_pct, :sd_pct, :sm_pct, :qtot, :etot, :e0, :ma_wet, :pen_pet, :fao_pet, :asce_pet, :msl_wet, :dd), available in daily, monthly and annual resolutions, and as Values or relative Deciles.

getraster for ALWB must use a date keyword to specify the date to download.

Usage with getraster

getraster(source::Type{<:ALWB{Union{Deciles,Values},Union{Day,Month,Year}}}, [layer]; date)

Download ALWB weather data as values or deciles with timesteps of Day, Month or Year.

Arguments

• layer: Symbol or Tuple of Symbol from (:rain_day, :s0_pct, :ss_pct, :sd_pct, :sm_pct, :qtot, :etot, :e0, :ma_wet, :pen_pet, :fao_pet, :asce_pet, :msl_wet, :dd). Without a layer argument, all layers will be downloaded, and a NamedTuple of paths returned.

Keywords

• date: a DateTime, AbstractVector of DateTime or a Tuple of start and end dates. For multiple dates, a Vector of multiple filenames will be returned. ALWB is available with a daily, monthly, and yearly, timestep.

Example

This will return the file containing annual averages, including your date:

julia> getraster(ALWB{Values,Year}, :ss_pct; date=Date(2001, 2))
"/your/RASTERDATASOURCES_PATH/ALWB/values/month/ss_pct.nc"

Returns the filepath/s of the downloaded or pre-existing files.

AWAP <: RasterDataSource

Daily weather data from the Australian Water Availability Project, developed by CSIRO.

See: www.csiro.au/awap

The AWAP dataset contains ASCII .grid files.

Usage with getraster

getraster(source::Type{AWAP}, [layer]; date)

Arguments

• layer: Symbol or Tuple of Symbol for layers in (:solar, :rainfall, :vprpress09, :vprpress15, :tmin, :tmax). Without a layer argument, all layers will be downloaded, and a NamedTuple of paths returned.

Keywords

• date: a DateTime, AbstractVector of DateTime or a Tuple of start and end dates. For multiple dates, A Vector of multiple filenames will be returned. AWAP is available with a daily timestep.

Example

Download rainfall for the first month of 2001:

julia> getraster(AWAP, :rainfall; date=Date(2001, 1, 1):Day(1):Date(2001, 1, 31))

31-element Vector{String}:
 "/your/path/AWAP/rainfall/totals/20010101.grid"
 "/your/path/AWAP/rainfall/totals/20010102.grid"
 ...
 "/your/path/AWAP/rainfall/totals/20010131.grid"

Returns the filepath/s of the downloaded or pre-existing files.

CHELSA{Union{BioClim,BioClimPlus,Climate,<:Future}} <: RasterDataSource

Data from CHELSA, currently implements the BioClim, BioClimPlus, and Climate variables for current and future conditions.

See: chelsa-climate.org for the dataset, and the getraster docs for implementation details.

Usage with getraster

getraster(source::Type{CHELSA{BioClim}}, [layer]; [version], [patch])
getraster(source::Type{CHELSA{BioClimPlus}}, [layer]; [version], [patch])
getraster(T::Type{CHELSA{Climate}}, [layer]; month)
getraster(T::Type{CHELSA{Future{BioClim}}}, [layer]; date
getraster(T::Type{CHELSA{Future{Climate}}}, [layer]; date, month)

Download CHELSA BioClim data from chelsa-climate.org.

Arguments

• layer: Integer, Symbol or tuple/range of these. Without a layer argument, all layers will be downloaded, and a NamedTuple of paths returned. Available layers are:
  • BioClim: Integers 1 to 19 or Symbols :bio1 to :bio19
  • BioClimPlus: Includes BioClim layers and many additional layers. See RasterDataSources.layers(BioClimPlus).
  • Climate: (:clt, :cmi, :hurs, :ncdf, :pet, :pr, :rsds, :sfcWind, :tas, :tasmax, :tasmin, :vpd)

Keyword arguments

• version: Integer indicating the CHELSA version, currently either 1 or 2. Defaults to 2.
• patch: Integer indicating the CHELSA patch number. Defaults to the latest patch (V1.2 and V2.1)
• month: Integer or AbstractArray of Integer. Chosen from 1:12.
• date: a Date or DateTime object, a Vector, or Tuple of start/end dates. Note that CHELSA CMIP5 only has two datasets, for the periods 2041-2060 and 2061-2080. CMIP6 has datasets for the periods 2011-2040, 2041-2070, and 2071-2100. Dates must fall within these ranges.

Example

using RasterDataSources, Dates
getraster(CHELSA{Future{BioClim, CMIP6, GFDL_ESM4, SSP370}}, 1, date = Date(2050))

EarthEnv{Union{HabitatHeterogeneity,LandCover}} <: RasterDataSource

Data from the EarthEnv including HabitatHeterogeneity and LandCover

See: www.earthenv.org

Usage with getraster

getraster(T::Type{EarthEnv{HabitatHeterogeneity}}, [layer]; res="25km")
getraster(T::Type{EarthEnv{LandCover}}, [layer]; discover=false)

Download EarthEnv habitat heterogeneity data.

Arguments

• layer: Integer, Symbol or tuple/range of these. Without a layer argument, all layers will be downloaded, and a NamedTuple of paths returned. Available layers are:
  • HabitatHeterogeneity: (:cv, :evenness, :range, :shannon, :simpson, :std, :Contrast, :Correlation, :Dissimilarity, :Entropy, :Homogeneity, :Maximum, :Uniformity, :Variance)
  • LandCover: Integers 1 to 12 or Symbols (:needleleaftrees, :evergreenbroadleaftrees, :deciduousbroadleaftrees, :othertrees, :shrubs, :herbaceous, :cultivatedandmanaged, :regularlyflooded, :urbanbuiltup, :snowice, :barren, :openwater)

Keywords

• res: String chosen from ("1km", "5km", "25km"), defaulting to "25km".
• discover::Bool: whether to download the dataset that integrates the DISCover model.

Returns the filepath/s of the downloaded or pre-existing files.

ERA5 <: RasterDataSource

Data from the ERA5 reanalysis dataset, accessed via Google's ARCO-ERA5 cloud-optimized Zarr store.

See: ARCO-ERA5

The dataset contains hourly data at 0.25° resolution from 1940 to present (~3 month lag). Data is accessed lazily - only the chunks you read are downloaded and cached locally.

Available layers

(:t2m, :d2m, :u10, :v10, :u100, :v100, :sp, :msl, :skt, :sst, :sd, :swvl1, :swvl2, :swvl3, :swvl4, :stl1, :stl2, :stl3, :stl4, :tcc, :lcc, :mcc, :hcc, :cape, :blh, :tcwv, :tco3, :tp, :ssrd, :ssr, :str, :strd, :slhf, :sshf, :e, :ro, :lsm)

Common layers:

• :t2m - 2m temperature
• :d2m - 2m dewpoint temperature
• :u10, :v10 - 10m wind components
• :sp - Surface pressure
• :msl - Mean sea level pressure
• :tp - Total precipitation
• :ssrd - Surface solar radiation downwards
• :tcc - Total cloud cover

Usage

using RasterDataSources, Zarr

# Get cloud source reference
source = getraster(ERA5)

# Create caching store and open
store = Zarr.CachingHTTPStore(source)
ds = zopen(Zarr.ConsolidatedStore(store, ""))

# Access a variable (chunks download on demand and are cached)
temp = ds["2m_temperature"]

# Or use the layer symbol helper
varname = layername(ERA5, :t2m)  # "2m_temperature"
temp = ds[varname]

The local cache is stored at RasterDataSources.rasterpath(ERA5). Subsequent runs reuse cached chunks without re-downloading.

WorldClim{Union{BioClim,Climate,Elevation,Weather,<:Future}} <: RasterDataSource

Data from WorldClim datasets, either BioClim, Climate, Weather, Climate, or Future variables for current and future conditions.

Future variables are available for BioClim and Climate data. See the docstring of Future for available model choices and implementation details.

See: www.worldclim.org

Usage with getraster

getraster(T::Type{WorldClim{BioClim}}, [layer::Union{Tuple,Int,Symbol}]; res)
getraster(T::Type{WorldClim{Climate}}, [layer::Union{Tuple,Symbol}]; month, res)
getraster(T::Type{WorldClim{Elevation}}; res)
getraster(T::Type{WorldClim{Weather}}, [layer::Union{Tuple,Symbol}]; date)
getraster(T::Type{WorldClim{Future}}, [layer]; date, res) => String

Arguments

• layer: Integer, Symbol or tuple/range of these. Without a layer argument, all layers will be downloaded, and a NamedTuple of paths returned. Available layers are:
  • BioClim: Integers 1 to 19 or Symbols :bio1 to :bio19
  • Climate: (:tmin, :tmax, :tavg, :prec, :srad, :wind, :vapr)
  • Elevation: Only has a single layer, :elev
  • Weather: (:tmin, :tmax, :prec)
  • Future{Climate}: (:tmin, :tmax, :prec)

Keywords

• res: String chosen from ("30s", "2.5m", "5m", "10m"), "10m" by default.
• month: Integer or AbstractArray of Integer. Chosen from 1:12.
• date: a Date or DateTime object, a Vector of dates, or Tuple of start/end dates. WorldClim{Weather} is available with a daily timestep. Future data is available in 20-year intervals from 2021 to 2100.

Examples

using RasterDataSources, Dates
bio_current = getraster(WorldClim{BioClim}, res = "5m")
bio_future = getraster(WorldClim{Future{BioClim, CMIP6, GFDL_ESM4, SSP370}}, date = Date(2050), res = "5m")

MODIS{ModisProduct} <: RasterDataSource

MODIS/VIIRS Land Product Database. Vegetation indices, surface reflectance, and more land cover data. Data from ModisProducts datasets.

See: modis.ornl.gov

Usage with getraster

getraster(T::Union{Type{<:ModisProduct}, Type{MODIS{X}}}, [layer::Union{Tuple,AbstractVector,Integer, Symbol}]; kwargs...) => Union{String, AbstractVector, NamedTuple}

Download MODIS data for a given ModisProduct as ASCII raster(s).

Arguments

• layer: Integer or tuple/range of Integer or Symbols. Without a layer argument, all layers will be downloaded, and a NamedTuple of paths returned.

Available layers for a given product can be looked up using RasterDataSources.layerkeys(T::Type{<:ModisProduct}).

Keywords

• lat and lon: Coordinates in decimal degrees of the approximate center of the raster. The MODIS API will try to match its pixel grid system as close as possible to those coordinates.

• km_ab and km_lr: Half-width and half-height of the raster in kilometers (kilometers above/below and left/right). Currently only Integer values are supported, up to 100.

• date: String, Date, DateTime, AbstractVector of dates or Tuple of a start and end date for the request. Strings should be in format YYYY-MM-DD but can be in similar formats as long as they are comprehensible by Dates.Date. The available date interval for MODIS is 16 days, reset every first of January.

Example

Download 250m NDVI in the western part of Britanny, France, from winter to late spring, 2002:

julia> getraster(MOD13Q1, :NDVI; lat = 48.25, lon = -4, km_ab = 50, km_lr = 50, date = (Date(2002,1,1), Date(2002,6,1)))

    10-element Vector{String}:
    "/your/path/MODIS/MOD13Q1/250m_16_days_NDVI/47.8313_-4.5899_2002-01-01.asc"
    ...
    "/your/path/MODIS/MOD13Q1/250m_16_days_NDVI/47.8313_-4.5899_2002-05-25.asc"

Will attempt to download several files, one for each date and layer combination, and returns the filepath/s of the downloaded or pre-existing files. Coordinates in the file names correspond to the lower-left corner of the raster.

ModisProduct <: RasterDataSet

Abstract supertype for MODIS/VIIRS products.

Usage

Some commonly used products are MOD13Q1 (250m resolution MODIS vegetation indices) and VNP13A1 (500m resolution VIIRS vegetation indices). Refer to the MODIS documentation for detailed product information.

layerkeys(T::Type{<:ModisProduct}) => Tuple

Tuple of Symbols corresponding to the available layers for a given product. May issue a request to MODIS server to get the layers list, or might just read this information if the correctly named file is available.

TerraClimate{<:WarmingScenario} <: RasterDataSource

Data from the TerraClimate dataset, a high-resolution global dataset of monthly climate and climatic water balance.

See: climatologylab.org/terraclimate

The dataset contains NetCDF files with monthly data. Each file covers one year and contains 12 monthly layers. Coverage is from years 1950-present.

The available layers are: (:aet, :def, :PDSI, :pet, :ppt, :q, :soil, :srad, :swe, :tmax, :tmin, :vap, :vpd, :ws).

Available scenarios:

• TerraClimate{Historical} or just TerraClimate: Historical data
• TerraClimate{Plus2C}: +2°C warming scenario
• TerraClimate{Plus4C}: +4°C warming scenario

getraster for TerraClimate must use a date keyword to specify the year to download.

Usage with getraster

getraster(source::Type{<:TerraClimate}, [layer]; date)

Download TerraClimate data for the specified scenario.

Arguments

• layer: Symbol or Tuple of Symbol from (:aet, :def, :PDSI, :pet, :ppt, :q, :soil, :srad, :swe, :tmax, :tmin, :vap, :vpd, :ws). Without a layer argument, all layers will be downloaded, and a NamedTuple of paths returned.

Keywords

• date: a DateTime, AbstractVector of DateTime or a Tuple of start and end dates. For multiple dates, a Vector of multiple filenames will be returned.

Example

julia> getraster(TerraClimate, :tmax; date=Date(2020))
"/path/to/storage/TerraClimate/historical/TerraClimate_tmax_2020.nc"

julia> getraster(TerraClimate{Plus2C}, :tmax; date=Date(2000))
"/path/to/storage/TerraClimate/plus2c/TerraClimate_plus2C_tmax_2000.nc"

Returns the filepath/s of the downloaded or pre-existing files.

NCEP{C<:NCEPCategory} <: RasterDataSource

Data from the NCEP/NCAR Reanalysis 1 (1948-Present) and NCEP/DOE Reanalysis 2 (1979-Present) datasets.

See: https://psl.noaa.gov/data/gridded/data.ncep.reanalysis.html

Type Parameters

• C: The category of data to download. One of SixHourlyPressure, SixHourlySurface, DailyPressure, DailySurface, DailySurfaceReanalysis2, MonthlyPressure, MonthlySurface, or SurfaceGauss.

Usage

getraster(NCEP{<category>}, layer; date, dataset)

Keywords

• date: A Date or DateTime object. The year is used to select the file for daily data.
• dataset: "reanalysis" or "reanalysis2". Defaults to "reanalysis".

Layers

Example

getraster(NCEP{DailyPressure}, :hgt; date=Date(2001), dataset="reanalysis")

CopernicusDEM <: RasterDataSource

The Copernicus Digital Elevation Model (GLO-30 and GLO-90), a global digital surface model derived from the WorldDEM, hosted as Cloud-Optimized GeoTIFFs on the AWS Open Data registry (no authentication required).

The dataset is split into 1°×1° tiles. Oceanic tiles that contain no land are absent from the dataset and are returned as missing.

See: Copernicus DEM on AWS

Usage with getraster

getraster(T::Type{CopernicusDEM}; extent, res) => Matrix{Union{String,Missing}}
getraster(T::Type{CopernicusDEM}; bounds, res) => Matrix{Union{String,Missing}}
getraster(T::Type{CopernicusDEM}; tile_index, res) => String

Keywords

• res: String chosen from ("30m", "90m"), "30m" by default.
• extent: an Extents.Extent, the canonical spatial selector. All 1°×1° tiles intersecting the extent are downloaded, returning a Matrix of file paths (with missing for non-existent oceanic tiles).
• bounds: a legacy alias for extent, also accepting a Tuple of (longitudes, latitudes) bounds.
• tile_index: a CartesianIndex (or CartesianIndices) addressing tiles by the integer degrees of their south-west corner. Mostly used internally.

Example

using RasterDataSources, Extents
# 30m tiles intersecting an extent over Italy
getraster(CopernicusDEM; extent=Extent(X=(10, 12), Y=(41, 43)), res="30m")

SRTM <: RasterDataSource

The Shuttle Radar Topography Mission digital elevation model at ~90 m (3 arcsecond) resolution, served from the CGIAR-CSI SRTM 4.1 mirror as GeoTIFFs in WGS84.

The dataset is split into 5°×5° tiles spanning latitudes 60°N to 60°S. Oceanic tiles that contain no land are absent from the dataset.

See: SRTM at CGIAR-CSI

Usage with getraster

getraster(SRTM; extent) => Matrix{String}
getraster(SRTM; bounds) => Matrix{String}
getraster(SRTM; tile_index) => String

Keywords

• extent: an Extents.Extent, the canonical spatial selector. All 5°×5° tiles intersecting the extent are downloaded, returning a Matrix of file paths.
• bounds: a legacy alias for extent, also accepting a Tuple of (longitudes, latitudes) bounds.
• tile_index: a CartesianIndex (or CartesianIndices) addressing tiles directly. Mostly used internally.

Example

using RasterDataSources, Extents
# tiles intersecting an extent over Italy
getraster(SRTM; extent=Extent(X=(10, 12), Y=(41, 43)))

SLGA <: RasterDataSource

Soil and Landscape Grid of Australia at ~90 m (3 arcsecond) resolution from CSIRO/TERN. Files are Cloud Optimised GeoTIFFs in WGS84 (EPSG:4326), covering continental Australia.

Available layers

Coarse fragments (cfg) have a distinct multi-file structure; use SLGA{CoarseFragments} instead.

Use depth and component keywords with getraster:

• depth: 0-5cm, 5-15cm, 15-30cm, 30-60cm, 60-100cm, 100-200cm

  • :der only supports "0-999cm"
  • :des only supports "0-200cm"

• component: EV, 05, 95

Examples

getraster(SLGA, :clay; depth="0-5cm", component="EV")
getraster(SLGA, :clay; component="EV") # returns all depths
getraster(SLGA, (:clay, :sand); depth="5-15cm", component="05")
getraster(SLGA, :clay; depth=["0-5cm", "5-15cm"])   # returns Vector
getraster(SLGA, :der) # single-depth layer

GADS <: RasterDataSource

A global 5-degree gridded product of spectral aerosol optical depths, derived from the Global Aerosol Data Set (Koepke et al. 1997) via the NicheMapR R package.

The dataset provides climatological aerosol optical properties for a global 5-degree grid based on measurements and models of 11 aerosol components. It covers two representative seasons (summer and winter, Northern Hemisphere) and a range of relative humidity conditions.

The file gads.nc contains a single variable OPTDEPTH with dimensions:

• Longitude: 72 points (−180° to 175°, 5° resolution)
• Latitude: 37 points (−90° to 90°, 5° resolution)
• Relative humidity: 8 levels (0, 50, 70, 80, 90, 95, 98, 99%)
• Season: 2 values (0 = NH summer/July, 1 = NH winter/January)
• Wavelength: 25 bands (250–4000 nm)

See: zenodo.org/records/19246341 and github.com/mrke/Global-Aerosol-Data-Set

Reference: Koepke, P., Hess, M., Schult, I., and Shettle, E.P. (1997). Global Aerosol Data Set. MPI-Report No. 243, Max-Planck-Institut für Meteorologie, Hamburg.

Usage with getraster

getraster(source::Type{GADS})

Example

julia> getraster(GADS)
"/path/to/storage/GADS/gads.nc"

Returns the filepath of the downloaded or pre-existing file.

CRUCL2 <: RasterDataSource

Monthly mean climate for the global land surface at 10-minute (~18 km) resolution, based on station observations from 1961–1990 (CRU CL 2.0).

The file cru_cl2.nc contains the following variables with a month dimension (1–12), except elv which has no month dimension:

Minimum and maximum temperature can be derived as tmp ± dtr/2.

See: zenodo.org/records/20754689 and github.com/mrke/CRU-CL-2

Reference: New, M., Lister, D., Hulme, M. and Makin, I., 2002. A high-resolution data set of surface climate over global land areas. Climate Research, 21: 1–25.

Example

julia> getraster(CRUCL2)
"/path/to/storage/CRUCL2/cru_cl2.nc"

Returns the filepath of the downloaded or pre-existing file.

Mean

Type parameter for data sources that provide a long-term mean (or historical mean) product in addition to a climatological one. Currently used with CPCSoil:

getraster(CPCSoil{Mean})  # full monthly time series 1948–present

CPCSoil{X} <: RasterDataSource

Monthly soil moisture data from the NOAA Climate Prediction Center (CPC), on a global 0.5° grid.

Two products are available:

Long-term mean (LTM) — CPCSoil (default): 12 monthly climatological means for a selected 30-year period. Use the period keyword to select the climatology:

• "1991-2020" (default, 9.5 MB)
• "1981-2010" (3.2 MB)

Historical monthly means — CPCSoil{Mean}: Full monthly time series from 1948 to present (254 MB).

See: psl.noaa.gov/data/gridded/data.cpcsoil.html

Reference: Fan, Y. and van den Dool, H. (2004). Climate Prediction Center global monthly soil moisture data set at 0.5° resolution for 1948 to present. Journal of Geophysical Research, 109, D10102.

Usage with getraster

getraster(source::Type{CPCSoil}; period="1991-2020")
getraster(source::Type{CPCSoil{Mean}})

Examples

julia> getraster(CPCSoil)
"/path/to/storage/CPCSoil/soilw.mon.1991-2020.ltm.v2.nc"

julia> getraster(CPCSoil; period="1981-2010")
"/path/to/storage/CPCSoil/soilw.mon.1981-2010.ltm.v2.nc"

julia> getraster(CPCSoil{Mean})
"/path/to/storage/CPCSoil/soilw.mon.mean.v2.nc"

Returns the filepath of the downloaded or pre-existing file.

RasterDataSet

Abstract supertype for datasets that belong to a RasterDataSource.

BioClim <: RasterDataSet

BioClim datasets. Usually containing layers from 1:19. These can also be accessed with :bioX, e.g. :bio5.

They do not usually use month or date keywords, but may use date in past/future scenarios.

Currently implemented for WorldClim and CHELSA as WorldClim{BioClim}, WorldClim{Future{BioClim}}, CHELSA{BioClim}, CHELSA{Future{BioClim, args..}}.

See the CHELSA and WorldClim docs for implementation details.

BioClimPlus <: RasterDataSet

Extended BioClim datasets, available from CHELSA. More information on the CHELSA website: https://chelsa-climate.org/exchelsa-extended-bioclim/

Some of these are available as average annual maximum, minimum, mean, and range. Others have a single value, more like the regular BioClim variables.

They do not usually use month or date keywords, but may use date in past/future scenarios.

Currently implemented for CHELSA as CHELSA{BioClim} and CHELSA{Future{BioClim, args..}}, specifying layer names as Symbols.

See the CHELSA docs for implementation details.

Climate <: RasterDataSet

Climate datasets. These are usually months of the year, not specific dates, and use a month keyword in getraster. They also use date in past/future scenarios.

Currently implemented for WorldClim and CHELSA as WorldClim{Climate}, WorldClim{Future{Climate, args..}} CHELSA{Climate} and CHELSA{Future{Climate, args..}}.

See the CHELSA and WorldClim docs for implementation details.

Weather <: RasterDataSet

Weather datasets. These are usually large time-series of specific dates, and use a date keyword in getraster.

Currently implemented for WorldClim as WorldClim{Weather}.

See the WorldClim docs for implementation details.

Elevation <: RasterDataSet

Elevation datasets.

Currently implemented for WorldClim as WorldClim{Elevation}.

See the WorldClim docs for implementation details.

LandCover <: RasterDataSet

Land-cover datasets.

Currently implemented for EarthEnv as EarthEnv{LandCover}.

See the EarthEnv docs for implementation details.

HabitatHeterogeneity <: RasterDataSet

Habitat heterogeneity datasets.

Currently implemented for EarthEnv as EarthEnv{HabitatHeterogeneity}.

See the EarthEnv docs for implementation details.

CoarseFragments <: RasterDataSet

Coarse fragments (>=2 mm soil particles, % vol) dataset.

Used as a type parameter on SLGA: SLGA{CoarseFragments} selects the Soil and Landscape Grid of Australia coarse-fragments product, which has a distinct multi-file structure from the standard SLGA attribute layers — each depth produces 7 files (six coarse-fragment probability class maps and one dominant class map). getraster(SLGA{CoarseFragments}; depth=...) returns a NamedTuple with keys :class1–:class6 and :dominant.

Examples

paths = getraster(SLGA{CoarseFragments}; depth="0-5cm")
paths.class1    # estimated probability of coarse fragment class 1
paths.dominant  # dominant coarse fragment class
getraster(SLGA{CoarseFragments}; depth=["0-5cm", "5-15cm"])  # returns Vector{NamedTuple}

Future{<:RasterDataSet,<:CMIPphase,<:ClimateModel,<:ClimateScenario}

Future climate datasets specified with a dataset, phase, model, and scenario.

Type Parameters

RasterDataSet

Currently BioClim and Climate are implemented for CHELSA and WorldClim. Future WorldClim is only available for CMIP6.

CMIPphase

Can be either CMIP5 or CMIP6.

ClimateModel

Climate models can be chosen from:

RasterDataSources.ACCESS1_0, RasterDataSources.BNU_ESM, RasterDataSources.CCSM4, RasterDataSources.CESM1_BGC, RasterDataSources.CESM1_CAM5, RasterDataSources.CMCC_CMS, RasterDataSources.CMCC_CM, RasterDataSources.CNRM_CM5, RasterDataSources.CSIRO_Mk3, RasterDataSources.CanESM2, RasterDataSources.FGOALS_g2, RasterDataSources.FIO_ESM, RasterDataSources.GFDL_CM3, RasterDataSources.GFDL_ESM2G, RasterDataSources.GFDL_ESM2M, RasterDataSources.GISS_E2_H_CC, RasterDataSources.GISS_E2_H, RasterDataSources.GISS_E2_R_CC, RasterDataSources.GISS_E2_R, RasterDataSources.HadGEM2_AO, RasterDataSources.HadGEM2_CC, RasterDataSources.IPSL_CM5A_LR, RasterDataSources.IPSL_CM5A_MR, RasterDataSources.MIROC_ESM_CHEM, RasterDataSources.MIROC_ESM, RasterDataSources.MIROC5, RasterDataSources.MPI_ESM_LR, RasterDataSources.MPI_ESM_MR, RasterDataSources.MRI_CGCM3, RasterDataSources.MRI_ESM1, RasterDataSources.NorESM1_M, RasterDataSources.bcc_csm_1, RasterDataSources.inmcm4 for CMIP5;

RasterDataSources.ACCESS_CM2, RasterDataSources.BCC_CSM2_MR, RasterDataSources.CMCC_ESM2, RasterDataSources.EC_Earth3_Veg, RasterDataSources.FIO_ESM_2_0, RasterDataSources.GFDL_ESM4, RasterDataSources.GISS_E2_1_G, RasterDataSources.HadGEM3_GC31_LL, RasterDataSources.INM_CM5_0, RasterDataSources.IPSL_CM6A_LR, RasterDataSources.MIROC6, RasterDataSources.MPI_ESM1_2_HR, RasterDataSources.MRI_ESM2_0, RasterDataSources.UKESM1_0_LL for CMIP6;"

ClimateScenario

CMIP5 Climate scenarios are all RepresentativeConcentrationPathway and can be chosen from: RCP26, RCP45, RCP60, RCP85

CMIP6 Climate scenarios are all SharedSocioeconomicPathway and can be chosen from: SSP126, SSP245, SSP370, SSP585

However, note that not all climate scenarios are available for all models.

Example

using RasterDataSources
datasource = CHELSA{Future{BioClim, CMIP5, GFDL_ESM4, SSP370}}

ClimateModel

Abstract supertype for climate models use in Future datasets.

CMIPphase

Abstract supertype for phases of the CMIP, the Coupled Model Intercomparison Project.

Subtypes are CMIP5 and CMIP6.

CMIP5 <: CMIPphase

The Coupled Model Intercomparison Project, Phase 5.

CMIP6 <: CMIPphase

The Coupled Model Intercomparison Project, Phase 6.

ClimateScenario

Abstract supertype for scenarios used in CMIPphase models.

RepresentativeConcentrationPathway

Abstract supertype for Representative Concentration Pathways (RCPs) for CMIP5.

Subtypes are: RCP26, RCP45, RCP60, RCP85

SharedSocioeconomicPathway

Abstract supertype for Shared Socio-economic Pathways (SSPs) for CMIP6.

Subtypes are: SSP126, SSP245, SSP370, SSP585

Values <: DataMode

Get the dataset as regular measured values.

Deciles <: DataMode

Get the dataset in relative deciles.

layerkeys(T::Type{<:ModisProduct}) => Tuple

Tuple of Symbols corresponding to the available layers for a given product. May issue a request to MODIS server to get the layers list, or might just read this information if the correctly named file is available.

rastername(source::Type, [layer]; kw...)

Returns the name of the file, without downloading it.

Arguments are the same as for getraster

Returns a String or multiple Strings.

rasterpath(source::Type, [layer]; kw...)

Returns the name of the file, without downloading it.

Arguments are the same as for getraster

Returns a String or multiple Strings.

rasterurl(source::Type, [layer]; kw...)

If the file has a single url, returns it without downloading.

Arguments are the same as for getraster.

Returns a URIs.jl URI or mulitiple URIs.

zipname(source::Type, [layer]; kw...)

If the url is a zipped file, returns its name.

Arguments are as the same for getraster where possible.

Returns a String or multiple Strings.

zippath(source::Type, [layer]; kw...)

If the url is a zipped file, returns its path when downloaded. (This may not exist after extraction with getraster)

Arguments are the same as for getraster where possible.

Returns a String or multiple Strings.

zipurl(source::Type, [layer]; kw...)

If the url is a zipped file, returns its zip path without downloading.

Arguments are the same as for getraster where possible.

Returns a URIs.jl URI or mulitiple URIs.

modis_request(T::Type{<:ModisProduct}, args...)

Lowest level function for requests to modis server. All arguments are assumed correct.

Arguments

• layer: String matching the "exact" layer name (i.e. as it is written in the MODIS dataset itself) for the given product. e.g. "250m_16_days_EVI".

• lat, lon, km_ab, km_lr in correct types

• from, to: Strings of astronomical dates for start and end dates of downloaded data, e.g. "A2002033" for "2002-02-02"

Returns a NamedTuple of information relevant to build a raster header, and a Vector of Dicts containing raster data, directly downloaded from MODIS. Those will almost always directly be passed to RasterDataSources.process_subset

process_subset(T::Type{<:ModisProduct}, subset::Vector{Any}, pars::NamedTuple)

Process a raw subset and argument parameters and create several raster files. Any already existing file is not overwritten.

For each band, a separate folder is created, containing a file for each of the required dates. This is inspired by the way WorldClim{Climate} treats the problem of possibly having to download several dates AND bands.

Can theoretically be used for MODIS data that does not directly come from RasterDataSources.modis_request, but caution is advised.

Returns the filepath/s of the created or pre-existing files.

product(T::Type{<:ModisProduct})

Extracts ModisProduct product name as a String

sinusoidal_to_latlon(x::Real, y::Real)

Convert x and y in sinusoidal projection to lat and lon in dec. degrees

This function uses Proj for coordinate conversions. Use import Proj to make it available.