EODataAccessGateway

EODataAccessGateway#

The EODataAccessGateway class is the core component of the EODAG library. It provides a unified interface for interacting with various Earth Observation data providers. This class allows users to configure providers, search for products, download data, and perform other operations related to Earth Observation data management. Below is an overview of its main functionalities, categorized for ease of use.

Constructor#

EODataAccessGateway([user_conf_file_path, ...])

An API for downloading a wide variety of geospatial products originating from different types of providers.

Configuration#

EODataAccessGateway.add_provider(name[, ...])

Adds a new provider.

EODataAccessGateway.set_preferred_provider(...)

Set max priority for the given provider.

EODataAccessGateway.get_preferred_provider()

Get the provider currently set as the preferred one for searching products, along with its priority.

EODataAccessGateway.update_providers_config([...])

Update providers configuration with given input.

EODataAccessGateway.update_product_types_list(...)

Update eodag product types list

Catalog#

EODataAccessGateway.available_providers([...])

Gives the sorted list of the available providers or groups

EODataAccessGateway.list_product_types([...])

Lists supported product types.

EODataAccessGateway.guess_product_type([...])

Find EODAG product type IDs that best match a set of search parameters.

EODataAccessGateway.fetch_product_types_list([...])

Fetch product types list and update if needed.

EODataAccessGateway.discover_product_types([...])

Fetch providers for product types

Crunch#

EODataAccessGateway.crunch(results, **kwargs)

Apply the filters given through the keyword arguments to the results

EODataAccessGateway.get_cruncher(name, **options)

Build a crunch plugin from a configuration

Download#

EODataAccessGateway.download(product[, ...])

Download a single product.

EODataAccessGateway.download_all(search_result)

Download all products resulting from a search.

Serialize/Deserialize#

EODataAccessGateway.serialize(search_result)

Registers results of a search into a geojson file.

EODataAccessGateway.deserialize(filename)

Loads results of a search from a geojson file.

EODataAccessGateway.deserialize_and_register(...)

Loads results of a search from a geojson file and register products with the information needed to download itself

Misc#

EODataAccessGateway.group_by_extent(searches)

Combines multiple SearchResults and return a list of SearchResults grouped by extent (i.e. bounding box).

EODataAccessGateway.guess_product_type([...])

Find EODAG product type IDs that best match a set of search parameters.

EODataAccessGateway.list_queryables([...])

Fetch the queryable properties for a given product type and/or provider.

EODataAccessGateway.available_sortables()

For each provider, gives its available sortable parameter(s) and its maximum number of them if it supports the sorting feature, otherwise gives None.

EODataAccessGateway.import_stac_items(items_urls)

Import STAC items from a list of URLs and convert them to SearchResult.
class eodag.api.core.EODataAccessGateway(user_conf_file_path=None, locations_conf_path=None)[source]#

An API for downloading a wide variety of geospatial products originating from different types of providers.

Parameters:

  • user_conf_file_path (str | None, default: None) – (optional) Path to the user configuration file

  • locations_conf_path (str | None, default: None) – (optional) Path to the locations configuration file

add_provider(name, url=None, priority=None, search={'type': 'StacSearch'}, products={'GENERIC_PRODUCT_TYPE': {'productType': '{productType}'}}, download={'auth_error_code': 401, 'type': 'HTTPDownload'}, **kwargs)[source]#

Adds a new provider.

search, products & download already have default values that will be updated (not replaced), with user provided ones:

  • search : {"type": "StacSearch"}

  • products : {"GENERIC_PRODUCT_TYPE": {"productType": "{productType}"}}

  • download : {"type": "HTTPDownload", "auth_error_code": 401}

Parameters:

  • name (str) – Name of provider

  • url (str | None, default: None) – Provider url, also used as search["api_endpoint"] if not defined

  • priority (int | None, default: None) – Provider priority. If None, provider will be set as preferred (highest priority)

  • search (dict[str, Any], default: {'type': 'StacSearch'}) – Search PluginConfig mapping

  • products (dict[str, Any], default: {'GENERIC_PRODUCT_TYPE': {'productType': '{productType}'}}) – Provider product types mapping

  • download (dict[str, Any], default: {'type': 'HTTPDownload', 'auth_error_code': 401}) – Download PluginConfig mapping

  • kwargs (dict[str, Any]) – Additional ProviderConfig mapping

available_providers(product_type=None, by_group=False)[source]#

Gives the sorted list of the available providers or groups

The providers or groups are sorted first by their priority level in descending order, and then alphabetically in ascending order for providers or groups with the same priority level.

Parameters:

  • product_type (str | None, default: None) – (optional) Only list providers configured for this product_type

  • by_group (bool, default: False) – (optional) If set to True, list groups when available instead of providers, mixed with other providers

Return type:

list[str]

Returns:

the sorted list of the available providers or groups

available_sortables()[source]#

For each provider, gives its available sortable parameter(s) and its maximum number of them if it supports the sorting feature, otherwise gives None.

Return type:

dict[str, ProviderSortables | None]

Returns:

A dictionary with providers as keys and dictionary of sortable parameter(s) and its (their) maximum number as value(s).

Raises:

UnsupportedProvider

crunch(results, **kwargs)[source]#

Apply the filters given through the keyword arguments to the results

Parameters:

  • results (SearchResult) – The results of a eodag search request

  • kwargs (Any)

Return type:

SearchResult

Returns:

The result of successively applying all the filters to the results

static deserialize(filename)[source]#

Loads results of a search from a geojson file.

Parameters:

filename (str) – A filename containing a search result encoded as a geojson

Return type:

SearchResult

Returns:

The search results encoded in filename

deserialize_and_register(filename)[source]#

Loads results of a search from a geojson file and register products with the information needed to download itself

Parameters:

filename (str) – A filename containing a search result encoded as a geojson

Return type:

SearchResult

Returns:

The search results encoded in filename

discover_product_types(provider=None)[source]#

Fetch providers for product types

Parameters:

provider (str | None, default: None) – The name of a provider or provider-group to fetch. Defaults to all providers (None value).

Return type:

dict[str, Any] | None

Returns:

external product types configuration

download(product, progress_callback=None, wait=0.2, timeout=10, **kwargs)[source]#

Download a single product.

This is an alias to the method of the same name on EOProduct, but it performs some additional checks like verifying that a downloader and authenticator are registered for the product before trying to download it.

If the metadata mapping for downloadLink is set to something that can be interpreted as a link on a local filesystem, the download is skipped (by now, only a link starting with file:/ is supported). Therefore, any user that knows how to extract product location from product metadata on a provider can override the downloadLink metadata mapping in the right way. For example, using the environment variable: EODAG__CREODIAS__SEARCH__METADATA_MAPPING__DOWNLOADLINK="file:///{id}" will lead to all EOProduct’s originating from the provider creodias to have their downloadLink metadata point to something like: file:///12345-678, making this method immediately return the later string without trying to download the product.

Parameters:

  • product (EOProduct) – The EO product to download

  • progress_callback (ProgressCallback | None, default: None) – (optional) A method or a callable object which takes a current size and a maximum size as inputs and handle progress bar creation and update to give the user a feedback on the download progress

  • wait (float, default: 0.2) – (optional) If download fails, wait time in minutes between two download tries

  • timeout (float, default: 10) – (optional) If download fails, maximum time in minutes before stop retrying to download

  • kwargs (Unpack[DownloadConf]) –

    Additional keyword arguments from the download plugin configuration class that can be provided to override any other values defined in a configuration file or with environment variables:

    • output_dir - where to store downloaded products, as an absolute file path (Default: local temporary directory)

    • output_extension - downloaded file extension

    • extract - whether to extract the downloaded products, only applies to archived products

    • dl_url_params - additional parameters to pass over to the download url as an url parameter

    • delete_archive - whether to delete the downloaded archives

    • asset - regex filter to identify assets to download

Return type:

str

Returns:

The absolute path to the downloaded product in the local filesystem

Raises:

PluginImplementationError

Raises:

RuntimeError

download_all(search_result, downloaded_callback=None, progress_callback=None, wait=0.2, timeout=10, **kwargs)[source]#

Download all products resulting from a search.

Parameters:

  • search_result (SearchResult) – A collection of EO products resulting from a search

  • downloaded_callback (DownloadedCallback | None, default: None) – (optional) A method or a callable object which takes as parameter the product. You can use the base class DownloadedCallback and override its __call__ method. Will be called each time a product finishes downloading

  • progress_callback (ProgressCallback | None, default: None) – (optional) A method or a callable object which takes a current size and a maximum size as inputs and handle progress bar creation and update to give the user a feedback on the download progress

  • wait (float, default: 0.2) – (optional) If download fails, wait time in minutes between two download tries of the same product

  • timeout (float, default: 10) – (optional) If download fails, maximum time in minutes before stop retrying to download

  • kwargs (Unpack[DownloadConf]) –

    Additional keyword arguments from the download plugin configuration class that can be provided to override any other values defined in a configuration file or with environment variables:

    • output_dir - where to store downloaded products, as an absolute file path (Default: local temporary directory)

    • output_extension - downloaded file extension

    • extract - whether to extract the downloaded products, only applies to archived products

    • dl_url_params - additional parameters to pass over to the download url as an url parameter

    • delete_archive - whether to delete the downloaded archives

    • asset - regex filter to identify assets to download

Return type:

list[str]

Returns:

A collection of the absolute paths to the downloaded products

fetch_product_types_list(provider=None)[source]#

Fetch product types list and update if needed.

If strict mode is enabled (by setting the EODAG_STRICT_PRODUCT_TYPES environment variable to a truthy value), this method will not fetch or update product types and will return immediately.

Parameters:

provider (str | None, default: None) – The name of a provider or provider-group for which product types list should be updated. Defaults to all providers (None value).

Return type:

None

get_cruncher(name, **options)[source]#

Build a crunch plugin from a configuration

Parameters:

  • name (str) – The name of the cruncher to build

  • options (Any) – The configuration options of the cruncher

Return type:

Crunch

Returns:

The cruncher named name

get_preferred_provider()[source]#

Get the provider currently set as the preferred one for searching products, along with its priority.

Return type:

tuple[str, int]

Returns:

The provider with the maximum priority and its priority

static group_by_extent(searches)[source]#

Combines multiple SearchResults and return a list of SearchResults grouped by extent (i.e. bounding box).

Parameters:

searches (list[SearchResult]) – List of eodag SearchResult

Return type:

list[SearchResult]

Returns:

list of SearchResult

guess_product_type(free_text=None, intersect=False, instrument=None, platform=None, platformSerialIdentifier=None, processingLevel=None, sensorType=None, keywords=None, abstract=None, title=None, missionStartDate=None, missionEndDate=None, **kwargs)[source]#

Find EODAG product type IDs that best match a set of search parameters.

When using several filters, product types that match most of them will be returned at first.

Parameters:

  • free_text (str | None, default: None) – Free text search filter used to search accross all the following parameters. Handles logical operators with parenthesis (AND/OR/NOT), quoted phrases ("exact phrase"), * and ? wildcards.

  • intersect (bool, default: False) – Join results for each parameter using INTERSECT instead of UNION.

  • instrument (str | None, default: None) – Instrument parameter.

  • platform (str | None, default: None) – Platform parameter.

  • platformSerialIdentifier (str | None, default: None) – Platform serial identifier parameter.

  • processingLevel (str | None, default: None) – Processing level parameter.

  • sensorType (str | None, default: None) – Sensor type parameter.

  • keywords (str | None, default: None) – Keywords parameter.

  • abstract (str | None, default: None) – Abstract parameter.

  • title (str | None, default: None) – Title parameter.

  • missionStartDate (str | None, default: None) – start date for datetime filtering. Not used by free_text

  • missionEndDate (str | None, default: None) – end date for datetime filtering. Not used by free_text

  • kwargs (Any)

Return type:

list[str]

Returns:

The best match for the given parameters.

Raises:

NoMatchingProductType

import_stac_items(items_urls)[source]#

Import STAC items from a list of URLs and convert them to SearchResult.

  • Origin provider and download links will be set if item comes from an EODAG server.

  • If item comes from a known EODAG provider, result will be registered to it, ready to download and its metadata normalized.

  • If item comes from an unknown provider, a generic STAC provider will be used.

Parameters:

items_urls (list[str]) – A list of STAC items URLs to import

Return type:

SearchResult

Returns:

A SearchResult containing the imported STAC items

list_product_types(provider=None, fetch_providers=True)[source]#

Lists supported product types.

Parameters:

  • provider (str | None, default: None) – (optional) The name of a provider that must support the product types we are about to list

  • fetch_providers (bool, default: True) – (optional) Whether to fetch providers for new product types or not

Return type:

list[dict[str, Any]]

Returns:

The list of the product types that can be accessed using eodag.

Raises:

UnsupportedProvider

list_queryables(provider=None, fetch_providers=True, **kwargs)[source]#

Fetch the queryable properties for a given product type and/or provider.

Parameters:

  • provider (str | None, default: None) – (optional) The provider.

  • fetch_providers (bool, default: True) – If new product types should be fetched from the providers; default: True

  • kwargs (Any) – additional filters for queryables (productType or other search arguments)

Raises:

UnsupportedProductType – If the specified product type is not available for the provider.

Return type:

QueryablesDict

Returns:

A QuerybalesDict containing the EODAG queryable properties, associating parameters to their annotated type, and a additional_properties attribute

search(page=1, items_per_page=20, raise_errors=False, start=None, end=None, geom=None, locations=None, provider=None, count=False, validate=True, **kwargs)[source]#

Look for products matching criteria on known providers.

The default behaviour is to look for products on the provider with the highest priority supporting the requested product type. These priorities are configurable through user configuration file or individual environment variable. If the request to the provider with the highest priority fails or is empty, the data will be request from the provider with the next highest priority. Only if the request fails for all available providers, an error will be thrown.

Parameters:

  • page (int, default: 1) – (optional) The page number to return

  • items_per_page (int, default: 20) – (optional) The number of results that must appear in one single page

  • raise_errors (bool, default: False) – (optional) When an error occurs when searching, if this is set to True, the error is raised

  • start (str | None, default: None) – (optional) Start sensing time in ISO 8601 format (e.g. “1990-11-26”, “1990-11-26T14:30:10.153Z”, “1990-11-26T14:30:10+02:00”, …). If no time offset is given, the time is assumed to be given in UTC.

  • end (str | None, default: None) – (optional) End sensing time in ISO 8601 format (e.g. “1990-11-26”, “1990-11-26T14:30:10.153Z”, “1990-11-26T14:30:10+02:00”, …). If no time offset is given, the time is assumed to be given in UTC.

  • geom (str | dict[str, float] | BaseGeometry | None, default: None) –

    (optional) Search area that can be defined in different ways:

    • with a Shapely geometry object: shapely.geometry.base.BaseGeometry

    • with a bounding box (dict with keys: “lonmin”, “latmin”, “lonmax”, “latmax”): dict.fromkeys(["lonmin", "latmin", "lonmax", "latmax"])

    • with a bounding box as list of float: [lonmin, latmin, lonmax, latmax]

    • with a WKT str

  • locations (dict[str, str] | None, default: None) – (optional) Location filtering by name using locations configuration {"<location_name>"="<attr_regex>"}. For example, {"country"="PA."} will use the geometry of the features having the property ISO3 starting with ‘PA’ such as Panama and Pakistan in the shapefile configured with name=country and attr=ISO3

  • provider (str | None, default: None) – (optional) the provider to be used. If set, search fallback will be disabled. If not set, the configured preferred provider will be used at first before trying others until finding results.

  • count (bool, default: False) – (optional) Whether to run a query with a count request or not

  • validate (bool | None, default: True) – (optional) Set to True to validate search parameters before sending the query to the provider

  • kwargs (Any) – Some other criteria that will be used to do the search, using paramaters compatibles with the provider

Return type:

SearchResult

Returns:

A collection of EO products matching the criteria

Changed in version v3.0.0b1: search() method now returns only a single SearchResult instead of a 2 values tuple.

Note

The search interfaces, which are implemented as plugins, are required to return a list as a result of their processing. This requirement is enforced here.

search_all(items_per_page=None, start=None, end=None, geom=None, locations=None, **kwargs)[source]#

Search and return all the products matching the search criteria.

It iterates over the pages of a search query and collects all the returned products into a single SearchResult instance.

Requests are attempted to all providers of the product ordered by descending piority.

Parameters:

  • items_per_page (int | None, default: None) – (optional) The number of results requested internally per page. The maximum number of items than can be requested at once to a provider has been configured in EODAG for some of them. If items_per_page is None and this number is available for the searched provider, it is used to limit the number of requests made. This should also reduce the time required to collect all the products matching the search criteria. If this number is not available, a default value of 50 is used instead. items_per_page can also be set to any arbitrary value.

  • start (str | None, default: None) – (optional) Start sensing time in ISO 8601 format (e.g. “1990-11-26”, “1990-11-26T14:30:10.153Z”, “1990-11-26T14:30:10+02:00”, …). If no time offset is given, the time is assumed to be given in UTC.

  • end (str | None, default: None) – (optional) End sensing time in ISO 8601 format (e.g. “1990-11-26”, “1990-11-26T14:30:10.153Z”, “1990-11-26T14:30:10+02:00”, …). If no time offset is given, the time is assumed to be given in UTC.

  • geom (str | dict[str, float] | BaseGeometry | None, default: None) –

    (optional) Search area that can be defined in different ways:

    • with a Shapely geometry object: shapely.geometry.base.BaseGeometry

    • with a bounding box (dict with keys: “lonmin”, “latmin”, “lonmax”, “latmax”): dict.fromkeys(["lonmin", "latmin", "lonmax", "latmax"])

    • with a bounding box as list of float: [lonmin, latmin, lonmax, latmax]

    • with a WKT str

  • locations (dict[str, str] | None, default: None) – (optional) Location filtering by name using locations configuration {"<location_name>"="<attr_regex>"}. For example, {"country"="PA."} will use the geometry of the features having the property ISO3 starting with ‘PA’ such as Panama and Pakistan in the shapefile configured with name=country and attr=ISO3

  • kwargs (Any) – Some other criteria that will be used to do the search, using parameters compatible with the provider

Return type:

SearchResult

Returns:

An iterator that yields page per page a collection of EO products matching the criteria

search_iter_page(items_per_page=20, start=None, end=None, geom=None, locations=None, **kwargs)[source]#

Iterate over the pages of a products search.

Parameters:

  • items_per_page (int, default: 20) – (optional) The number of results requested per page

  • start (str | None, default: None) – (optional) Start sensing time in ISO 8601 format (e.g. “1990-11-26”, “1990-11-26T14:30:10.153Z”, “1990-11-26T14:30:10+02:00”, …). If no time offset is given, the time is assumed to be given in UTC.

  • end (str | None, default: None) – (optional) End sensing time in ISO 8601 format (e.g. “1990-11-26”, “1990-11-26T14:30:10.153Z”, “1990-11-26T14:30:10+02:00”, …). If no time offset is given, the time is assumed to be given in UTC.

  • geom (str | dict[str, float] | BaseGeometry | None, default: None) –

    (optional) Search area that can be defined in different ways:

    • with a Shapely geometry object: shapely.geometry.base.BaseGeometry

    • with a bounding box (dict with keys: “lonmin”, “latmin”, “lonmax”, “latmax”): dict.fromkeys(["lonmin", "latmin", "lonmax", "latmax"])

    • with a bounding box as list of float: [lonmin, latmin, lonmax, latmax]

    • with a WKT str

  • locations (dict[str, str] | None, default: None) – (optional) Location filtering by name using locations configuration {"<location_name>"="<attr_regex>"}. For example, {"country"="PA."} will use the geometry of the features having the property ISO3 starting with ‘PA’ such as Panama and Pakistan in the shapefile configured with name=country and attr=ISO3

  • kwargs (Any) – Some other criteria that will be used to do the search, using paramaters compatibles with the provider

Return type:

Iterator[SearchResult]

Returns:

An iterator that yields page per page a collection of EO products matching the criteria

static serialize(search_result, filename='search_results.geojson')[source]#

Registers results of a search into a geojson file.

Parameters:

  • search_result (SearchResult) – A collection of EO products resulting from a search

  • filename (str, default: 'search_results.geojson') – (optional) The name of the file to generate

Return type:

str

Returns:

The name of the created file

set_preferred_provider(provider)[source]#

Set max priority for the given provider.

Parameters:

provider (str) – The name of the provider that should be considered as the preferred provider to be used for this instance

Return type:

None

update_product_types_list(ext_product_types_conf)[source]#

Update eodag product types list

Parameters:

ext_product_types_conf (dict[str, dict[str, dict[str, Any]] | None]) – external product types configuration

Return type:

None

update_providers_config(yaml_conf=None, dict_conf=None)[source]#

Update providers configuration with given input. Can be used to add a provider to existing configuration or update an existing one.

Parameters:

  • yaml_conf (str | None, default: None) – YAML formated provider configuration

  • dict_conf (dict[str, Any] | None, default: None) – provider configuration as dictionary in place of yaml_conf

Return type:

None

Contents