Hibernations: Instances which are running code are 'hibernated' when they shoudn't be

ministryofjustice / analytics-platform

Parent repository for the MOJ Analytics Platform

MIT License

14 stars 1 forks source link

Hibernations: Instances which are running code are 'hibernated' when they shoudn't be #28

Closed RobinL closed 6 years ago

RobinL commented 6 years ago

We had a user running a long-running task yesterday. It looks like it got interrupted at 22:00:

It's quite peculiar that it got interrupted, but the code seemed to 'restart'. (RAM went down to 0, as did CPU, but then it came back up soon after 22:00)

Expected behaviour

The CPU was running at 100% immediately prior to 22:00, so we would have expected her session not be interrupted.

This user's session shouldn't have been interrupted as a result of their instance hibernating

RobinL commented 6 years ago

I got Anthony to run an infinite while loop last night. Here are the results:

So I think the 22:00 issue was a red herring. it looks like everything gets shut off at 23:00 summertime = UTC+1

RobinL commented 6 years ago

Aldo's been working on this here: https://github.com/ministryofjustice/analytics-platform-idler/pull/8

RobinL commented 6 years ago

Seemed to work last night: