Sending Slack Alerts on Airflow Task’s Failure

Once you have an airflow dag up and running, the next step would be to keep track of failures and take action on time. Sending alerts on a Slack channel with failure details can come in handy to monitor dag runs.

Requirements:

1. Python > 3.0
2. Airflow > 2.0
3. Slack Webhook

Python Packages:

apache-airflow-providers-http

Following are the steps for setting up slack alerts.

Setting up Slack

1. Create a Slack Webhook.
2. Create a slack_webhook connection in Airflow > Admin > Connections

• Insert the first part of the webhook URL as host.
• Insert later part formatted as “T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX” as password.

Setting up python function in Airflow dag

• Create a function using SlackWebhookOperator to generate the slack message.

def slack_notification(context):
    slack_msg = """
            :red_circle: Task Failed. 
            *Task*: {task}  
            *Dag*: {dag} 
            *Execution Time*: {exec_date}
            *Error Msg*: {error_message}
            *Log Url*: {log_url} 
            """.format(
            task=context.get('task_instance').task_id,
            dag=context.get('task_instance').dag_id,
            ti=context.get('task_instance'),
            exec_date=context.get('execution_date'),
            error_message=context.get('exception') or context.get('reason'),
            log_url=context.get('task_instance').log_url,
        )
    failed_alert = SlackWebhookOperator(
        task_id='slack_notification',
        http_conn_id='slack_webhook',
        channel='#slack-channel',
        message=slack_msg)
    return failed_alert.execute(context=context)

• Pass the above function as ‘on_failure_callback’ function on default_args.

default_args = {
        'owner': 'Airflow',
        'depends_on_past': False,
        'start_date': days_ago(2),
        'email_on_failure': False,
        'email_on_retry': False,
        'retries': 1,
        'retry_delay': timedelta(seconds=60),
        'on_failure_callback': slack_notification
        }

A fully operational test dag is given below. You will then receive alerts on task failure with failure details.