DocsWorkflow ComponentsVariables

Variables​Variables

Variables are key-value pairs that help reuse some values across tasks.

You can also store variables on a namespace level so that they can be reused across multiple flows in a given namespace.

How to configure variables

Here is how you can configure variables in your flow:

yaml
id: hello_world
namespace: company.team

variables:
  myvar: hello
  numeric_variable: 42

tasks:
  - id: log
    type: io.kestra.plugin.core.debug.Return
    format: "{{ vars.myvar }} world {{ vars.numeric_variable }}"

You can see the syntax for using variables is {{ vars.variable_name }}.

How are variables rendered

You can use variables in any task property that is documented as dynamic.

Dynamic variables will be rendered thanks to the Pebble templating engine. Pebble templating engine allows you to process various expressions with filters and functions. More information on variable processing can be found under Expressions.

FAQ

How do I escape a block in Pebble syntax to ensure that it won't be parsed?

To ensure that a block of code won't be parsed by Pebble, you can use the {% raw %} and {% endraw %} Pebble tags. For example, the following Pebble expression will return the string {{ myvar }} instead of the value of the myvar variable:

yaml
{% raw %}{{ myvar }}{% endraw %}

Which order are inputs and variables resolved?

Inputs are resolved first, even before the execution starts. In fact, if you try to create a flow with an invalid input value, the execution will not be created.

Therefore, you can use inputs within variables, but you can't use variables or Pebble expressions within inputs.

Expressions are rendered recursively, meaning that if a variable contains another variable, the inner variable will be resolved first.

When it comes to triggers, they are handled similarly to inputs as they are known before the execution starts (they trigger the execution). This means that you can't use inputs (unless they have defaults attached) or variables within triggers, but you can use trigger variables within variables.

To make it clearer, let's look at some examples.

Examples

This flow uses inputs, trigger and execution variables which are resolved before variables:

yaml
id: upload_to_s3
namespace: company.team

inputs:
  - id: bucket
    type: STRING
    defaults: declarative-data-orchestration

tasks:
  - id: get_zip_file
    type: io.kestra.plugin.core.http.Download
    uri: https://wri-dataportal-prod.s3.amazonaws.com/manual/global_power_plant_database_v_1_3.zip

  - id: unzip
    type: io.kestra.plugin.compress.ArchiveDecompress
    algorithm: ZIP
    from: "{{outputs.get_zip_file.uri}}"

  - id: csv_upload
    type: io.kestra.plugin.aws.s3.Upload
    from: "{{ outputs.unzip.files['global_power_plant_database.csv'] }}"
    bucket: "{{ inputs.bucket }}"
    key: "powerplant/{{ trigger.date ?? execution.startDate | date('yyyy_MM_dd__HH_mm_ss') }}.csv"

triggers:
  - id: hourly
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "@hourly"

This flow will start a task conditionally based on whether the input is provided or not:

yaml
id: conditional_branching
namespace: company.team

inputs:
  - id: parameter
    type: STRING
    required: false

tasks:
  - id: if
    type: io.kestra.plugin.core.flow.If
    condition: "{{inputs.customInput ?? false }}"
    then:
      - id: if_not_null
        type: io.kestra.plugin.core.log.Log
        message: Received input {{inputs.parameter}}
    else:
      - id: if_null
        type: io.kestra.plugin.core.log.Log
        message: No input provided

Here is an example that uses a trigger variable within a trigger itself (that's allowed!):

yaml
id: backfill_past_mondays
namespace: company.team

tasks:
  - id: log_trigger_or_execution_date
    type: io.kestra.plugin.core.log.Log
    message: "{{ trigger.date ?? execution.startDate }}"

triggers:
  - id: first_monday_of_the_month
    type: io.kestra.plugin.core.trigger.Schedule
    timezone: Europe/Berlin
    backfill:
      start: 2023-11-11T00:00:00Z
    cron: "0 11 * * MON" # at 11 on every Monday
    conditions: # only first Monday of the month
      - type: io.kestra.plugin.core.condition.DayWeekInMonthCondition
        date: "{{ trigger.date }}"
        dayOfWeek: "MONDAY"
        dayInMonth: "FIRST"

Can I transform variables with Pebble expressions?

Yes. Kestra uses Pebble Templates along with the execution context to render dynamic properties. This means that you can use Pebble expressions (such as filters, functions, and operators to transform inputs and variables.

The example below illustrates how to use variables and Pebble expressions to transform string values in dynamic task properties:

yaml
id: variables_demo
namespace: company.team

variables:
  DATE_FORMAT: "yyyy-MM-dd"

tasks:
  - id: seconds_of_day
    type: io.kestra.plugin.core.debug.Return
    format: '{{60 * 60 * 24}}'

  - id: start_date
    type: io.kestra.plugin.core.debug.Return
    format: "{{ execution.startDate | date(vars.DATE_FORMAT) }}"

  - id: curr_date_unix
    type: io.kestra.plugin.core.debug.Return
    format: "{{ now() | date(vars.DATE_FORMAT) | timestamp() }}"

  - id: next_date
    type: io.kestra.plugin.core.debug.Return
    format: "{{ now() | dateAdd(1, 'DAYS') | date(vars.DATE_FORMAT) }}"

  - id: next_date_unix
    type: io.kestra.plugin.core.debug.Return
    format: "{{ now() | dateAdd(1, 'DAYS') | date(vars.DATE_FORMAT) | timestamp() }}"

  - id: pass_downstream
    type: io.kestra.plugin.scripts.shell.Commands
    taskRunner:
      type: io.kestra.plugin.core.runner.Process
    commands:
      - echo {{outputs.next_date_unix.value}}

Can I use nested variables?

Yes! However, keep in mind that depending on the task, you may need to wrap the root variable in a json() function in order to access specific keys. Here is an example using a list of maps as a variable:

yaml
id: vars
namespace: company.myteam

variables:
  servers:
    - fqn: server01.mydomain.io
      user: root
    - fqn: server02.mydomain.io
      user: guest
    - fqn: server03.mydomain.io
      user: rick

tasks:
  - id: parallel
    type: io.kestra.plugin.core.flow.EachParallel
    value: "{{ vars.servers }}"
    tasks:
      - id: log
        type: io.kestra.plugin.core.log.Log
        message:
           - "{{ taskrun.value }}" # for each element in the servers list, this will print the full JSON object e.g. {“fqn”:“server01.mydomain.io”,“user”:“root”}
           - "{{ json(taskrun.value).fqn }}" # prints the value for that key e.g. server01.mydomain.io
           - "{{ json(taskrun.value).user }}" # prints the value for that key e.g. root

Was this page helpful?