search

citrusframework / citrus

Framework for automated integration tests with focus on messaging integration
https://citrusframework.org
Apache License 2.0
456 stars 134 forks source link

wait for s3 file to appear #1178

Open mlybarger opened 3 months ago

mlybarger commented 3 months ago

hi, i have a longer running process that we're using citrus to test. the process is kicked off from an airflow job that we invoke from citrus. the job will launch some processing ec2's, then it will launch an EMR to convert the data to parquet, and load to our data lake.

when the emr is complete, it puts a done file in a known spot in an s3 bucket. current, we just have a <sleep seconds="1800" to give enough time for the process to finish.

what's a better way to say "wait for the done file to show up, or timeout after 30 minutes."

we're using xml dsl on an older version of citrus 2.7.9, , but are looking to update citrus soon.

bbortt commented 3 months ago

there are retry actions available, did you take a look at these?

https://citrusframework.org/citrus/reference/4.2.1/html/index.html#containers-repeat

christophd commented 3 months ago

there is also a repeat-on-error container that performs a nested action with retry mechanism. you can define the maximum number of retries and a delay between attempts.

https://citrusframework.org/citrus/reference/4.2.1/html/index.html#containers-repeat-on-error

christophd commented 3 months ago

BTW, looking forward to seeing you update to later Citrus versions. Please, let us know how it goes 🤞