search

AtlasOfLivingAustralia / species-lists

Tool for managing species lists in the ALA
1 stars 0 forks source link

capture migration issues #65

Open adam-collins opened 4 months ago

adam-collins commented 4 months ago

Log of issues found when migrating apps from specieslist-webapp to species-list.

  1. specieslist-webapp does not expose isBIE or isSDS on /ws/speciesList service. Untested fix on branch https://github.com/AtlasOfLivingAustralia/specieslist-webapp/tree/migration-fix

  2. cognito so I can be seen as ADMIN when logged in. Commit https://github.com/AtlasOfLivingAustralia/species-lists/commit/a0f8a2781f42cd9ba51dc5d5aa27da5d0326471b

  3. add isBIE search parameter for ala-bie-hub. Commit https://github.com/AtlasOfLivingAustralia/species-lists/commit/aa7aeec049307ddcc31ae2c894c2aedef3e41c9a

  4. cognito so I can send machine to machine JWT. Commit https://github.com/AtlasOfLivingAustralia/species-lists/commit/fd7edd0ca57deebf9dd51ce9b9811226627dea18. Makes it work but I am uncertain it is the proper approach. Needs validation and maybe integration into ala-security-project.

  5. fix error when using addSpeciesListItem. Commit https://github.com/AtlasOfLivingAustralia/species-lists/commit/3e3c97d0040c77e2a4f87db3acbaf44e9e2bed69.

  6. addSpeciesListItem appears not to trigger namematching, but for the purpose of bie-index that is OK.

adam-collins commented 4 months ago
  1. Unsure of the connection between species-lists and collectory. Does not appear to be a push by species-lists. I'm inferring a need for a script that can remove collectory database entries for species lists. A backup and restore function would be helpful for these entries would also be helpful.
djtfmartin commented 4 months ago

Ive deliberately omitted the connection between species-lists and collectory - but this should be more widely discussed.

The entries in the collectory that result from a species list being created serve very little purpose, and just result in a rather blank collectory page. I'd favour the BIE indexing lists (authoritative only) instead, and then search results can link straight to the list instead of to a poor in content collectory page. This is an enhancement that come can later.

They also have the affect of skewing stats of data resources. At the time of writing, the homepage for ALA reports:

Screenshot 2024-02-21 at 12 59 57

but of the 4843 data resource, 3562 are species lists, and only 60 of these species lists are authoritative.

cc @peggynewman

adam-collins commented 4 months ago
  1. Add isSDS for REST queries and harvest during migration. Commit https://github.com/AtlasOfLivingAustralia/species-lists/commit/fbbb0c8fb04ed5b82a4461036a6cd133770cc603. Requires specieslist-webapp 5.1.3-SNAPSHOT for migrate to have access to these fields.
adam-collins commented 4 months ago
  1. specieslist-webapp /downloadList generates duplicate column headers, e.g. family, when the default headers also appear in kvpValues. This results in those columns being discarded when migrated into lists. However you cannot upload a list with a column 'family' because it is discarded/ignored. This is not ideal. The primary issue is that sds builds a sensitive-species-data.xml with this uploaded family, not the matched family. I will change sds to use the matched family. This should impact nothing new as it is already an issue if the sds item is not matched in lists.
adam-collins commented 4 months ago

spatial-hub issues

  1. species-lists work required for uploading lists only with taxonID. See https://github.com/AtlasOfLivingAustralia/species-lists/issues/83
  2. species-lists work required for ingesting, a service with the valid values for mandatory fields. See https://github.com/AtlasOfLivingAustralia/species-lists/issues/82
adam-collins commented 4 months ago
  1. (update) working to add AuthMachineJwt to the security lib, pull request https://github.com/AtlasOfLivingAustralia/ala-security-project/pull/44. When merged to develop it will need to be included in the branch used by species-lists as well https://github.com/AtlasOfLivingAustralia/ala-security-project/tree/epic/spring_boot3_support/develop. When that is done, AuthMachineJwt.java and AthMachineJwtConfiguration.java can be removed from species-lists.