changes in aspace getting stuck

[ccleeton]

The Aspace Operations group have noticed that there have a been a few instances where a change was made to a finding aid and it is saved and the changes do not appear. It seems to get "stuck" requiring staff to resave. Is there a way to run a report or investigate this in an other way.

Example: https://findingaids.princeton.edu/catalog/MC247_c2144?onlineToggle=false this note - Conditions Governing Access Materials in this series will open for research in December, 2024. This was changed to add the month, previously it only had the year. When it was not displaying in the finding aid we went back to Aspace and found it was indeed there and saved but we had to save it again for it to appear.

Sudden Priority Justification

With inconsistent indexing, folks are unable to be sure their work is getting in, and so they're repeatedly having to go back to check and it's creating some concern that some of their work is missing.

Success Criteria

Schedule a full hard reindex every night. We'll probably have to install "Whenever", Figgy has examples on what that looks like.

[regineheberlein]

Another example: C1372_c26464-10543 and C1372_c28672-49784, two components that were added 2/19 and have not yet made it to pulfalight (as of 2/22).

[ccleeton]

@tpendragon Wondering if you might know if this is something that could be looked at as maintenance since it is impeding work. There is no way to know what is publishing correctly and what is not.

[tpendragon]

@tpendragon Wondering if you might know if this is something that could be looked at as maintenance since it is impeding work. There is no way to know what is publishing correctly and what is not.

Seems reasonable. Can you give it a sudden priority justification in the issue description that roughly says that? Presumably this can wait a few weeks for us to spec out? If so we'll take a look at it then. Thanks for the ping!

[ccleeton]

This impedes our work because random notes or other changes fail to publish, but we would have to check every single one to know that it failed or was successful.

[tpendragon]

@ccleeton @faithc I'm guessing we're going to have a hard time figuring this out. Worst case scenario, how do you feel about just reindexing everything on some regular basis as a fix to this?

[faithc]

I think this is a question for Will Clements/ the ASpace Operations Group. Can you check in with him, @ccleeton ?

[ccleeton]

@tpendragon What would a reindex mean. It happens hourly right now correct? But if things aren't getting fixed by that how would a reindex help?

[tpendragon]

@ccleeton Every hour we reindex things that ArchivesSpace tells us has "changed" - if it's missing some, as it seems it is, then a full reindex would catch everything whether or not ASpace told us it changed.

[ccleeton]

Thank you for the clarification. How often would you possibly do a full one? Are they scheduled now for any sort of regularity?

[tpendragon]

We only do them right now when something's very broken. We could do it regularly, and whatever interval "regularly" is would mean these things that slip through the cracks are fixed then. Like - if you had to wait a week, is that too long? Three days? A day? (Daily full reindexing is probably too much)

[hackartisan]

I just did some messing around running some bits of the code in my terminal and didn't find anything amiss with the way the api request is written. the only thing I noticed that is weird is that this line never seems to actually show up in the logs, although the indexjob is definitely getting enqueued and run, and the Event is getting updated. https://github.com/pulibrary/pulfalight/blob/6b5341d3aa82a4cb0d2379f0582e747bf69995b0/app/services/aspace/indexer.rb#L41

My one thought is that it might be worth trying an event per repository, rather than just one for all indexing. then we could update the appropriate event at the end of the job instead of after enqueuing.

[hackartisan]

Well I figured out why the log messages aren't writing; the rake task sets the logger to STDOUT before it runs. I guess this is so when you run the task manually you see the output directly.

https://github.com/pulibrary/pulfalight/blob/6b5341d3aa82a4cb0d2379f0582e747bf69995b0/lib/tasks/reindexing.rake#L6

[ccleeton]

reopening this as we are still seeing issues.

Example - Will Clements made a change to https://findingaids.princeton.edu/catalog/MC279#access to add a sub-note about Existence and Location of Copies and it is not appearing.

Edit was made yesterday and we are still not seeing it this morning. https://aspace.princeton.edu/staff/resources/1590#tree::resource_1590

[faithc]

I don't think we ever requested that this note display on the site so I'm not sure this is solely an indexing issue? The issue Charles noted in the pulfalight channel does seem to be primarily related to indexing.

[ccleeton]

Thanks @faithc but it does appear here - https://findingaids.princeton.edu/catalog/MC001#find-more

[ccleeton]

Charles also added this in pulfalight channel which might be related - Hi all, The boxes for the Dance Subject Files (TC107) are not able to be requested. This is in rare books repository. The location data was updated on Friday.

[faithc]

Thanks @faithc but it does appear here - https://findingaids.princeton.edu/catalog/MC001#find-more

Ah, never mind then. Ignore my comment about that! Ha

[tpendragon]

@ccleeton Can you make sure these problems are fixed tomorrow and if so close this ticket again? Thank you!