Rewrite 4/10: Table model updates

[ketiltrout]

This PR updates the peewee table models for the data index (StorageGroup, StorageNode, ArchiveAcq, ArchiveFile, ArchiveFileCopy, ArchiveFileCopyRequest). It does not deal with AcqType or FileType, which will be handled in a later PR.

This PR updates the field lists for these tables (adding new fields and removing unused fields). All field updates are backwards compatible with alpenhorn-1. It also add a few convenience methods for various common database queries related to the tables. The methods are mostly database queries that were performed explicitly within the I/O code. Moving these queries to methods here is to reduce the complexity of the I/O code itself.

StorageGroup and StorageNode will have a subsequent update to add the new I/O framework, but the other table models (ArchiveAcq, ArchiveFile, ArchiveFileCopy, ArchiveFileCopyRequest) have no further changes pending.

StorageGroup

Fairly light changes:

• Marks the name field as unique
• Adds a copy_state method which takes an ArchiveFile and returns the value of has_file (Y/M/X/N) for copy/copies of the ArchiveFile in this group.

StorageNode

A more extensive update.

StorageNode fields

• name is marked unique
• suspect is dropped. It was never used or implemented properly.
• No change to storage_type, but I have updated its description in the docstring to better indicate what 'F' means (i.e. it's anything that's not an archive and not transiting storage). This closes #49 .
• max_total_gb allows null/None (for nodes which can't determine their maximum size). The unusual default of -1 is dropped. The code retains support for using -1 to indicate an unknown max_total_gb, but will never set the value to -1 (it will use null/None instead).
• min_avail_gb default explicitly set to zero.
• min_delete_age_days is dropped. It has never been used.

  StorageNode methods and properties

• local: a boolean property which is True when this StorageNode is local to the running alpenhorn (whether or not it is active).
• archive: a boolean property which is True when the StorageNode is an archive node (when storage_type=='A')
• under_min: Returns False if avail_gb is None, otherwise returns min_avail_gb > avail_gb.
• over_max: Returns False if max_total_gb is None or non-positive, or else a boolean: sum of all ArchiveFile.size_b on this node (see total_gb below) is greater than max_total_gb.
• named_copy_present given an acq name and a file name, returns True if a copy of the file in the acq exists on this node.
• total_gb sums up ArchiveFile.size_b for all files on this node and returns the result in GiB.
• all_files returns a list of paths (relative to root) for all files existing on this node.

ArchiveAcq

Only change:

• name is marked unique.

ArchiveFile

Field changes:

• adds a constraint that the tuple (name, acq) is unique. This constraint is already present in the CHIMEDB production database. We may have added it by hand there.

Properties and methods:

• path the path to this file, i.e. the path concatenation of acq.name and name.
• archive_count returns the number of archive nodes containing a copy of this file.

ArchiveFileCopy

Fields:

• Add ready, a boolean which might indicate that a file is ready for I/O. This field is used to communicate between hosts which files are available for pulls, but, as the docstring indicates, shouldn't ever be consulted directly, because different I/O types can have different interpretations of the meaning of this field. (In James's work, we called this "prepared").
• Allows size_b to be null/None (for instances where the size-on-node can't be determined).
• Adds last_update a timestamp that is automatically updated whenever the ArchiveFileCopy record is changed.

Properties and Methods:

• To implement the automatic last_updated, the save and update methods are re-implemented to add the new last_udpated value before handing things off to peewee. This could also be done at the DB level: MySQL allows an ON UPDATE CURRENT_TIMESTAMP clause in the table schema, but that isn't supported by peewee.
• path a property returning an absolute pathlib.Path to the file copy.

ArchiveFileCopyRequest

Field changes:

• nice is dropped. Has never been used.
• completed has a default value of False added, which I think is the default of a non-null boolean field if none is given, but it's better to be explicit.
• timestamp is given the current time as a creation-time default

Other changes

Because it's used in the StorageNode.local property, this is a convenient place to update the hostname-finding logic: The function alpenhorn.util.get_short_hostname is renamed to alpenhorn.util.get_hostname and now supports specifying the hostname explicitly in the alpenhorn YAML config, giving us the flexibility to put logical values in StorageNode.host (like, say, "scinet"), instead of being forced to use whatever value the actual name of the host we're running on has (like, say, "nia-dm1").

The unit tests for (at least the updated parts of) alpenhorn.acqusition, alpenhorn.archive, and alpenhorn.storage have been restored/updated after being disabled in #144. (They were also renamed to drop _model from the name of the test files, because I like testing alpenhorn/<name>.py in tests/test_<name>.py).

A large set of DB-data-producing fixtures has been added to conftest.py. These will be used extensively by the I/O framework unit tests.

[ketiltrout]

I made ArchiveFile.archive_count and StorageNode.under_min into properties. I've added verbs to the other parameter-free methods, but I'm willing to revert their names and make them properties, too, if that seems better.

Also, I'm now realising the contents of this PR would probably have been better introduced when they were used in the I/O rewrite, but unlikely to be worth changing now.

[ketiltrout]

I pushed the timestamp update into the schema. Downside is it only works in MySQL. Upside is don't have to re-implement ArchiveFileCopy.update, I guess.