Do event updates on a local copy and mv back

[taldcroft]

@jeanconn - I did some testing locally on the implications of the events.db3 file going missing or changing during an active session.

Testing:

• Copy $ska/data/kadi/events.db3 to a local version
• Truncate the local version in time to generate diffs that are obvious in query results
• Open a window with KADI=$PWD to point to local events.db3
• Open ipython and do queries to confirm truncated events
• In another window, cp $ska/data/events.db3 back to the local events.db3
• Confirm that in the original ipython, subsequent event queries still produce expected truncated results, including queries for models that I didn't do before.
• Quit ipython and restart, and confirm that the full (non-truncated) events.db3 is now being used.

As far as I can tell the file system does the right thing, which is to say that the active process holds on to the file handle and so queries continue working, using a phantom database that only exists in the internal file system but not visible to user.

cc: @linamaria11

[jeanconn]

I don't think it matters, but was your "local" version on local disk or still NFS?

[taldcroft]

NFS, but not /proj/sot/ska. Instead /data/baffin served by numa.

[jeanconn]

Sounds good. I can't imagine that there would be significantly different behaviors for different mounts.

[taldcroft]

It turns out that local testing was not entirely complete as I accidentally had a copy of ltt_bads.dat in the directory, and in a task schedule config one needs to specify /bin/mv, /bin/mkdir etc.

The new version is now installed to /proj/sot/ska/data/kadi and has been shown to work (after fixes). Plan to put in a bug-fix version update (including #121 also) and tag a release, but since affected files are not in the install tarball I think no immediate action is required now except to copy taskschedule* files to GRETA also.

[jeanconn]

Not sure what happened with this, but any reason not to merge and tag?

[taldcroft]

Yes, please merge and tag.