Closed lovelysandlonelys closed 3 months ago
lead_times = [12, 24, 48]
for lead_time in lead_times:
FOLDER_STRUCTURE = f'Magnolia/{lead_time}'
min = 20
file_path = f'Magnolia/{lead_time}/magnolia_{lead_time}'
model = f'{min} * * * * mkdir -p ./logs/$(date "+\%Y")/$(date "+\%m")/{FOLDER_STRUCTURE} && docker exec semaphore-core python3 src/semaphoreRunner.py -d ./{file_path}.json >> ./logs/$(date "+\%Y")/$(date "+\%m")/{file_path}.log'
min = 25
file_path = f'Magnolia/{lead_time}/magnolia_transform_{lead_time}'
transform = f'{min} * * * * mkdir -p ./logs/$(date "+\%Y")/$(date "+\%m")/{FOLDER_STRUCTURE} && docker exec semaphore-core python3 src/semaphoreRunner.py -d ./{file_path}.json >> ./logs/$(date "+\%Y")/$(date "+\%m")/{file_path}.log'
print(model)
print(transform)
Meeting Notes:
If the interval is minutely:
print(format_timing(300, 3600)) # Hourly run with offset of 10 min
print(format_timing(0, 360)) # No offset run every six min
print(format_timing(1200, 10800)) # 20 min offset run every 3 hours
print(format_timing(0, 3600)) # No offset run hourly
print(format_timing(60, 360)) # No offset run every six min
5 /1 0-59/6 20 /3 0 /1 1-59/6
Two distinct tasks
NOTE: the CRON logs are just logging only the standard error not the cron log errors so they are kinda useless
NOTE: the offset value for each inundation model should be offset by some seconds so that all the models are not all running at the same time and utilizing 80% of the CPU