search

peterwebster / henson

Master data store for the Hensley Henson Journals project, and issue tracker. The application code is kept elsewhere.
1 stars 1 forks source link

Dashboard refresh fails #259

Closed peterwebster closed 2 years ago

peterwebster commented 2 years ago

@alisonrclarke @dpl0js I rather doubt that this is a very high priority: the issue is that on the Dashboard page (in the Note column) the numbers of published/unpublished entries don't update (although they should). They should do so automatically, but they don't, and the 'Refresh dashboard table' function below also fails. Since we're soon going to be finished with new batch ingests, I wouldn't spend any time on this. It's useful but far from essential at this stage, @alisonrclarke @dpl0js

(This is not connected to #256 or #263)

alisonrclarke commented 2 years ago

Error in the logs is [Mysql2::Error::ConnectionError: MySQL server has gone away: BEGIN]

We might be able to fix this by increasing the wait_timeout on the db connection.

alisonrclarke commented 2 years ago

Looks like wait_timeout is already high. Looking more closely at the dashboard refresh method it seems to be the query for published items that is likely to time out as it's a more complex query. I'm going to try a couple of things to help: first, split out the calls to each object type so they can run separately; second, try to catch the InvalidStatement exception caused by the timeout on published items count, and in that case just replace with the total count (which at this stage should be around the same number I think) - does that sound reasonable?

peterwebster commented 2 years ago

Hi @alisonrclarke : yes, the total count should match the published count, as I don't think there are any entries that have been ingested and then subsequently found to require hiding from public view. In principle they might not match, but it's unlikely, except in the short period of time after items have been ingested but not yet approved to publish. And I'd be unlikely to refresh the dashboard at that point anyway. So, yes, this is fine.

peterwebster commented 2 years ago

@alisonrclarke on test, the count of journal entries (c2100) at https://henson-test.durham.ac.uk/journals matches the dashboard (2102 published, 1841 unpublished) which matches the reality (see the search results, which lists items published to no.2091 and unpublished thereafter https://henson-test.durham.ac.uk/?end_day=19&end_month=1&end_year=1935&page=21&per_page=100&q=&search_field=only_journals&start_day=1&start_month=1&start_year=1900 )

So, this all seems to be in synch. However, I can't test the actual dashboard refresh in action because (at the moment) the Publish All (Journals) on the dashboard fails (latest attempt begun at 1001), which would change the totals to give the dashboard something to refresh to. When I try to set an individual entry to publish from the /edit screen, this also fails. These have not been an issue on the production server, but I don't want to risk pushing the changes in case this error is a result of your recent changes (rather than some independent and pre-existing issue.) So, could you see what's going on? If the issue is something else, then perhaps it can remain unsolved on test while we push your change to production.

peterwebster commented 2 years ago

OK, part of this is great, and the other super-strange.

I've manually set a few journal entries to Publish, and the dashboard refresh reflects these. So, that's great.

However, I did the same with some persons and the change was reflected in the dashboard, but the Published count went down instead of up. However, on production there are very few unpublished ones, and no more are expected, so this should not cause us any problems. (What the public sees at /people is accurate enough as it stands.) So, unless this is a thirty-second fix, I think we leave it. @alisonrclarke )

dpl0js commented 2 years ago

@peterwebster @alisonrclarke Many thanks to you both. Just to say that I would like to keep adding new Persons to the repository. However, they won't be in industrial quantities any more so it won't really matter if they are not reflected in the total number, should a thirty-second fix not prove possible.

alisonrclarke commented 2 years ago

I realised today that I hadn't properly farmed out the jobs for each data type to resque, so my attempted fix probably wouldn't have completed as intended. I've just pushed a change and have just set off another dashboard refresh.

peterwebster commented 2 years ago

@alisonrclarke had another look this morning. All is still good with Journals. Something odd still seems to happen with persons - ie. when one sets a person to publish, the count on the dashboard immediately goes down, and then goes up to where it should be once you run the refresh. This we can live with, so let's push this to production.

peterwebster commented 2 years ago

OK @alisonrclarke : I've now tried to publish/depublish items on production, and run the dashboard refresh, and it now reflects those changes (although you have to be patient!). So, I think this is now done, and I close the ticket.