I had some data bricks notebooks which takes more than one hour for execution . Earlier i was running them on Mircosoft hosted agents. But since the timeouts are as follow

Forever on self-hosted agents
For 360 minutes (6 hours) on Microsoft-hosted agents with a public project and public repository
For 60 minutes on Microsoft-hosted agents with a private project or private repository (unless additional capacity is paid for) https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops&tabs=yaml

So i switched to Self hosted agents to avoid timeout . But still the issue did not resolve.

To verify , if the issue is with agent or with the task , i created a task with bash script for infinite loop , like below

!/bin/bash

TimeStart=$(date +"%Y-%m-%d %H:%M:%S") TimeEnd=$(date -d "$TimeStart 2 hours" +"%Y-%m-%d %H:%M:%S") echo "Start Time: $TimeStart" echo "End Time: $TimeEnd"

while true; do TimeNow=$(date +"%Y-%m-%d %H:%M:%S")

if [ "$TimeNow" \> "$TimeEnd" ]; then
    echo "It's time to finish."
    break
else
    echo "Not done yet, it's only $TimeNow"
fi
sleep 10

done

This task ran for more 3 hours before i canceled it . which mean , the task for databricks Wait for Notebook execution does not respect the timeout policy . Can you please help me set a higher timeout for this task so that i can continue to use this task and ADO for my execution

microsoft / azdo-databricks

Waiting Task getting timedout even on self hosted agents #70

!/bin/bash