debug Prodigy session issue

[rlskoeser]

@laurejt notes towards configuring verbose logging (PRODIGY_LOGGING=verbose):

Here's the line in cdh-ansible where we configure the environment variables for prodigy running in supervisor:

https://github.com/Princeton-CDH/cdh-ansible/blob/main/inventory/group_vars/prodigy/vars.yml#L53

I suggest we add a new ansible variable for logging level and pass that in to the supervisor config.

[rlskoeser]

testing on staging, reloading my session in the browser and I get some alternation (not necessarily every time) between the next page to annotate changes and the total number of annotations saved on my session goes from 34 to 36 (which makes me suspect something is going on between the two servers (whether a session thing or a prodigy hashing issue)

[laurejt]

This appears to be an issue of inconsistent state across the two servers.

Although we should define the set_hashes function for our custom recipes, this does not appear to be causing the issue (i.e. the function defaults should work for our input and task).

[rlskoeser]

We've updated the config on the PUL load balancer to only use one VM for each staging/production instance, to avoid the possibility of switching between different task queues. This should be resolved now, but I'm not sure how or if we can confirm.