General project configuration¶

The following features can be accessed via a general project configuration file:

Import codelists and mappings from public GitHub repositories
Add all countries to the region codelist (details: A common list of countries)

Configuration file format¶

The configuration file must be located on the same level as the mappings and definitions directories.

The file must be in yaml format and named nomenclature.yaml.

Importing from an external repository¶

In order to import from an external repository, the configuration file must define the repository and the repositories key.

The repository has a name (in the example below common-definitions) and a url. Multiple repositories can be used in a single configuration file:

repositories:
  common-definitions:
    url: https://github.com/IAMconsortium/common-definitions.git/
  legacy-definitions:
    url: https://github.com/IAMconsortium/legacy-definitions.git/

In order for the import to work the url must be given in the above format, i.e. with the leading https:// and the trailing .git/.

Information from an external repository can either be used for codelists (“definitions”) or model mappings, or both. For each definition dimension, i.e. region or variable multiple external repositories can be used as the example below illustrates for variable:

repositories:
  common-definitions:
    url: https://github.com/IAMconsortium/common-definitions.git/
  legacy-definitions:
    url: https://github.com/IAMconsortium/legacy-definitions.git/
definitions:
  region:
    repository: common-definitions
  variable:
    repositories:
      - common-definitions
      - legacy-definitions
mappings:
  repository: common-definitions

The value in definitions.region.repository can be a list or a single value.

For model mappings the process is analogous using mappings.repository.

Filter code lists imported from external repositories¶

Since importing the entirety of, for example, common-definitions is too much for most projects, the list can be filtered using include and exclude keywords. Under these keywords, lists of filters can be given that will be applied to the code list from the given repository.

The filtering can be done by any attribute:

repositories:
  common-definitions:
    url: https://github.com/IAMconsortium/common-definitions.git/
definitions:
  variable:
    repository:
      name: common-definitions
      include:
        - name: [Primary Energy*, Final Energy*]
        - name: "Population*"
          tier: 1
      exclude:
        - name: "Final Energy|Industry*"
          depth: 2

If a filter is being used for repositories, the name attribute must be used for the repository.

In the example above we are including: 1. All variables starting with Primary Energy or Final Energy 2. All variables starting with Population and with the tier attribute equal to 1

From this list we are then excluding all variables that match “Final Energy|Industry*” and have a depth of 2 (meaning that they contain two pipe “|” characters).

Adding countries to the region codelist¶

By setting definitions.region.country as true in the configuration file:

definitions:
  region:
    country: true

the nomenclature package will add all countries to the region codelist.

More details on the list of countries can be found here: A common list of countries.

Adding NUTS to the region codelist¶

By setting definitions.region.nuts (optional) in the configuration file:

definitions:
  region:
    nuts:
      nuts-1: [ AT, BE, CZ ]
      nuts-2: [ AT ]
      nuts-3: true

the nomenclature package will add the selected NUTS regions to the region codelist.

In the example above, the package will add: NUTS 1 regions for Austria, Belgium and Czechia, NUTS 2 regions for Austria, NUTS 3 regions for all EU countries.

More details on the list of NUTS regions can be found here: NUTS classification.

Specify dimensions to be read¶

The configuration file offers the possibility to set the dimensions which will be read by DataStructureDefinition, overriding the dimensions from the definitions sub-directories. If no sub-directories exist (e.g.: when importing dimensions from external repositories), setting dimensions in the configuration file is mandatory.

In the below case we specify region, variable and scenario to be read and used for validation:

dimensions:
  - region
  - variable
  - scenario

Filter model mappings from external repositories¶

We often only want to use a subset of models in a particular project (and not import all mappings), so there is an option to filter for specific model mappings. This works very similarly to the filtering for definitions.

repositories:
common-definitions:
  url: https://github.com/IAMconsortium/common-definitions.git/
mappings:
  repository:
    name: common-definitions
    include:
      - MESSAGEix-GLOBIOM 2.1-M-R12

The above example retrieves only the model mapping for MESSAGEix-GLOBIOM 2.1-M-R12 from the common-definitions repository.

Configuring processors¶

The processors section of nomenclature.yaml allows processors to be declared directly in the configuration file, so they are applied automatically when calling process() without passing an explicit processor argument.

Region processor¶

Setting processors.region-processor as true will automatically create a RegionProcessor from the project’s default mappings/ directory:

processors:
  region-processor: true

This is equivalent to calling:

import pyam
from nomenclature.processor import RegionProcessor

df = pyam.IamDataFrame(data="path/to/file.csv")
dsd = DataStructureDefinition("definitions")
processor = RegionProcessor.from_directory("mappings", dsd)
aggregated_data = process(df, dsd)

NUTS processor¶

Setting processors.nuts to a list of model names will automatically create a NutsProcessor and apply NUTS hierarchical aggregation (NUTS3 → NUTS2 → NUTS1 → Country → EU27) for those models:

processors:
  nuts-processor: [ Model A, Model B ]

More details on NUTS aggregation can be found here: NUTS classification.