In the context of Data Engineering, workflow orchestration refers to the process of scheduling and arranging tasks that form your [[Data Pipeline|data pipeline]]. A workflow orchestration tool allows you to schedule, run, and observe the entire process.
## Popular Workflow Orchestration Tools
[[Apache Airflow]]
[[Dagster]]
[[Prefect]]
## Workflow Orchestration Advantages
- Create complex custom workflows
- Makes it easier to create [[Idempotence|idempotent]] workflows
- Alert you if something fails
- Allows you to gracefully retry and recover from failures
## Workflow Orchestration Disadvantages
- Adds complexity in scheduling
- Requires additional infrastructure and maintenance costs
%% wiki footer: Please don't edit anything below this line %%
## This note in GitHub
<span class="git-footer">[Edit In GitHub](https://github.dev/data-engineering-community/data-engineering-wiki/blob/main/Concepts/Workflow%20Orchestration.md "git-hub-edit-note") | [Copy this note](https://raw.githubusercontent.com/data-engineering-community/data-engineering-wiki/main/Concepts/Workflow%20Orchestration.md "git-hub-copy-note")</span>
<span class="git-footer">Was this page helpful?
[👍](https://tally.so/r/mOaxjk?rating=Yes&url=https://dataengineering.wiki/Concepts/Workflow%20Orchestration) or [👎](https://tally.so/r/mOaxjk?rating=No&url=https://dataengineering.wiki/Concepts/Workflow%20Orchestration)</span>