Review Changes with Stakeholders before 2.1 deploy

[pgwillia]

Master

Removed

• [x] Completely disable logging of warnings around the "excel spreadsheet" issue PR#2049

Changed

• [x] Added DOI reset feature for admins #1739
• [x] migration to fix concatenated subjects (part 2) #1449
• [x] Catch and log embargo expiry job save errors #1989
• [x] Added Collection and Community to reindex rake task #2444

Fixed

• [x] oaisys: change etdms date source to graduation date as per LAC spec #2298
• [x] Give proper response when solr 400s The backstory here is that Bing has a bad set of URLs for searches from somewhere or other that it keeps hitting over and over again, I assume because it just thinks "hmm 500 better try again later", and it's causing a bunch of noise in Rollbar for no benefit – I'm hoping that seeing the 400 will convince it to drop them from the index finally #2086
• [x] Search with sort without default sort direction no longer errors #2077
• [x] pre-loading main FA fonts, increased link spacing on mobile, and ordered the header tags in a more accessible way. Various fixes from lighthouse suggestions PR#2254
• [x] Fix issue where we improperly 404'd when a deleted Collection is being displayed in the edit history #2504
• [x] Fix communication with pushmi_pullyu by changing the format for entries in redis queue #2527

Chores

• [x] reviewed
• Initialize disabled ReadOnlyMode #2100
• Updated Architecture diagrams PR#2135
• Remove entirely unnecessary config file. PR#2044
• Fixture names have been modified to ensure their uniqueness PR#2302
• bump rubocop and fix cop violations PR#2072
• bump rubocop and fix more cop violations PR#2132
• Danger token in Github Actions #2282
• bump rdf-n3 and fix isomorphic_with? regression PR#2070
• bump omniauth-rails_csrf_protection gem for omniauth compatibility PR#2096

Integration

• [x] Refactored Controlled Vocabulary support to allow for new, raw vocabs without i18n translations. The motivation here is that we have a bunch of URIs we want to machine-map to human readable values, and it doesn't make sense to introduce intermediate symbols we'd have to cobble together somehow, plus that would involve polluting the i18n file with hundreds of new entries.

API Examples:

      ControlledVocabulary.value_from_uri(namespace: :digitization, vocab: :subject, uri: "http://id.loc.gov/authorities/names/n79007225")
       => ["Edmonton (Alta.)", false]
      ControlledVocabulary.uri_from_value(namespace: :digitization, vocab: :subject, value: "Edmonton (Alta.)")
       => "http://id.loc.gov/authorities/names/n79007225"
      uri = "http://id.loc.gov/authorities/names/n79007225"
      ControlledVocabulary.digitization.subject.from_uri(uri)
       => "Edmonton (Alta.)"
      ControlledVocabulary.digitization.subject.from_value("Edmonton (Alta.)")
       => "http://id.loc.gov/authorities/names/n79007225"
      ControlledVocabulary.era.language.english
       => "http://id.loc.gov/vocabulary/iso639-2/eng"

Further discussion of the context can be found at #2119

• [x] Many "description" or "abstract" fields (at the Item level as well as Communities and Collections) contain HTML tags. Because these are text fields, HTML is not rendered in the UI and text looks garbled and it's way less readable than ideal. Markdown should work really well for this since that's already used in many of the tools staff working in repositories are familiar with. Added redcarpet gem which renders markdown in our decorators and strips markdown in our Solr exporters #1322

• [x] Jupiter II work is underway to incorporate Digitized materials into Jupiter in the digitalcollections namespace. We've begun by modelling, developing the user interface, and tasks for ingest of FolkFest Programs.

  • Peel redirects #1769
  • Make Digitization::Book Depositable
  • Make Digitization::Newspaper Depositable
  • Make Digitization::Image Depositable
  • Make Digitization::Map Depositable
  • Volume and Issue label attribute to Digitization::Book
  • Corrected missing pluralization in Digitization::Book attributes
  • Make Digitization::Book more like other items and other small fixes
  • book metadata for folk fest digitization #2010
  • Add Digitization::Book ingest artifacts to model #2011
  • Add task that will kick off job for batch ingestion of digitization metadata from a csv containing triples #2011

• [x] Added feature flags to Jupiter. The motivation for this change is so that we can continuously deploy and turn on or off features as needed. Admins can enable features through the admin panel #1897

• [x] Improve batch ingest workflow by using Google Drive for staging and a user interface for creating and reviewing batches. Requires new secrets to be configured and batch_ingest feature flag. Further context can be found #1986

  • Add new models (BatchIngest and BatchIngestFile) for improved batch ingest work #1986
  • Add new google drive client service to be able to retrieve files/spreadsheets from Google Drive #1986
  • Add batch ingest controller and views for CRUDing batch ingests #1986
  • Add batch ingest form with google file picker and spreadsheet validation #1986
  • Add batch ingestion job for batch ingesting items into ERA #1986
  • Add various fixes and improvements to batch ingestion work

• [x] Subdomains being used for front doors to the application. era and digitalcollections are the ones in use so far. This requires that the host subdomain match these exactly, and new secrets to be configured. Further context can be found #1707

  • Add 'digitalcollections' subdomain for future front door
  • Refactored item download/view behaviour in routes and views to be reusable in digitization namespace and application wide

Added

• [x] Added highlighting of terms within search results descriptions (flipper: fulltext_search, term: 'ipsum') #1800
• [x] Labels for active facet badges (flipper: facet_badge_category_name) #1261

Removed

• [x] Remove logo_id foreign key on item/thesis which was causing issues with deletions PR#2335

Changed

• [x] Default behaviour within a facet to 'OR' (flipper: or_facets) #1990

Chores

• [x] reviewed
• Namespaces for the Controlled Vocabularies #2118
• Brakeman linting to Github Actions workflow
• Make autocomplete explicit PR#2449
• Removed rufus-scheduler -- it's an unused dependency PR#2434
• Moved visibility vocabulary into a jupiter_core namespace
• Move doi_url to Doiable class
• Bump flipper-ui, flipper-active_record and flipper and remove redundant configuration
• Bump rubocop to 1.15.0 and Style/TrivialAccessors default changed PR#2343
• Bump rubocop to 1.18.2 and fix cop violations PR#2415
• Bump rubocop-minitest to 0.14.0 and note really smelly tests PR#2416
• Fixed a flaky test where the page hasn't finished loading #2129
• Refactored one of our smelliest tests to use fixtures and reduce number of assertions per test #2419

[pgwillia]

• [x] add config.hosts << for uat and staging environments

[pgwillia]

• [x] @pgwillia is going to investigate if there is a queue for pmpy ready to go or how the back log will be resolved.
• @pgwillia will tag two releases:
  • [x] first for the main branch so that we can do quality assurance on the pmpy fix
  • [x] second for the new features including batch ingest. Will include release notes for @henryzhang87