Please prepare for Release 1.6.0 of Aries VCR

[swcurran]

Please prepare for a 1.6.0 release of Aries VCR:

1. Review the PRs/commits since 1.5.0 tag and produce a human consumable set of release notes about what changed. Should likely by important PR by PR and expected impact on the users of the service.
2. Review the changes from the view of a service using the API. What other APIs are likely to be impacted by the changes -- especially the names change to add BNs to the names array.

Add the release notes to the issue and then we'll take the refined list, share it with stakeholders and use the list for the tag created for 1.6.0 and related release.

[amanji]

Summary of major changes:

API

Topic Search Changes and OrgBook BC BN Searching

This change introduces major updates to search, adding a v4 API. The early versions of the API (v2 and v3) are unchanged from a signature perspective, but the data returned may change in some cases (see below).

Topic searches in previous versions were based on Credential attributes that were linked back to their owner Topics. This resulted in duplicate Topic results when query terms (such as names) match multiple Credentials. The v4 API introduces Topic search endpoints that are based on Topic attributes. Topic attributes are an aggregation of their owned Credential attributes. This change requires rebuilding SOLR indexes.

In the OrgBook BC deployment, this update was accompanied by a change to the categorization data, adding the BN (Business Number) to the "names" vs. a credential attribute to enable searching by attribute. As a result, some existing APIs were initially altered to include the BNs. Before releasing to production, a change (see below) filtered the BN results out for the existing v2 API searches.

We recommend that OrgBook API clients should migrate to the new version of the search API to enable support for BN searching.

Change log:

Server

• Introduces new search API endpoints for Topic (/v4/search/topic) and Credential (/v4/search/credential). Search API endpoints for Credential are derived from v3
• New index fields introduced for Topic that allow searching on Name, Address and Source ID
• Fixes issue with filtering Topic/Credential by category facets, where result set was not narrowed down by selection
• General code cleanup

Exclude names other than entity_name and entity_name_assumed in formatted Topics

As noted above, in order to retain the same data semantics as in the previous v2 search, additional names (notably the BN) have been filtered out of the the following endpoints:

• v2/topic/{id}/formatted
• v2/topic/ident/{type}/{source_id}/formatted

Fix typo in api/v3/views/rest.py

Fixes a typo that prevents the credential verification logic in the UI from working.

OrgBook BC Front End Client

Updates for API Additions

Updates to use new and evolved API calls in the OrgBook BC front end:

• Upgrades basic and advanced search to v4 from v3
• Adds new facets to advanced search for Credential types
• Upgrades autocomplete to v3 from v2, which expands autocomplete beyond Names to include Address and Source ID
• Improves advanced search handling of URL query parameters to allow for bookmarking of queries. URL query parameters are also carried over from basic search to advanced search
• Aligns historical facet filtering for basic and advanced search
• General code cleanup

Fixes a name display bug with External Schema/Services.

When 'name.' + is not found in i18n translations, entry displays as unformatted key. This fix strips the ‘name.’ prefix so just will display.

Angular upgrade

This PR upgrades the client Angular application to 11.02

Change log:

• Upgrades Angular to 11.02 (Note: this upgrade removes support for IE <= 10)
• Replaces localization-router node package and associated http loader
• Updates style imports

Adds collapse list of alternate names for credential search results

This PR adds topic display names and a collapsable element of alternate topic names/identifiers to OrgBook search results.

Change log:

• Fixes an issue where topic search results showed a blank name if topic credentials did not use prescribed name types.
• By default, the topic source_id is displayed for topic search results otherwise the display name is derived from name types in the following order:
  • display_name, entity_name_assumed, entity_name
• If topic names other than the display name are available, remaining names/identifiers are displayed in a collapsable element
• Removes local and remote names on credentials unless an entity_name and entity_name_assumed is mapped to credential
• Adds localization to Credential Types dropdown in Advanced Search

4. Add remote debugger configuration and docs for vcr-server.

The remote debugger for vcr-server is now available when setting DJANGO_DEBUG=True. The default value for DJANGO_DEBUG has also been set to False, to avoid running the debugger unintentionally.

General Changes

Adds documentation for updating BN in db Name table

Documentation for enabling autocomplete search on BN in existing Aries-VCR databases

Add project lifecycle badge, license badge

Added project lifecycle badge. For lifecycle badge information, see: https://github.com/bcgov/repomountie/blob/master/doc/lifecycle-badges.md

Audits and updates production npm package vulnerabilities

Dependency updates.

[amanji]

Actual List of changes:

1. Excludes names in formatted Topics
2. External Schema/Services credential type rendering bug
3. Adds documentation for updating BN in db Name table
4. Resolves errors with Swagger pages not loading in development
5. Fix typo in api/v3/views/rest.py
6. Topic search refactor
7. Default DEBUG to false
8. Add project lifecycle badge, license badge
9. Adds return statement for function generating remote names
10. Angular upgrade
11. Adds collapse list of alternate names for credential search results
12. Add remote debugger configuration and docs for vcr-server
13. Audit and update production npm package vulnerabilities

[amanji]

Raw output of PRs:

1. Excludes names in formatted Topics

Change log:

Names returned in the results will exclude any names that are not of the type entity_name or entity_name_asssumed.

Affects the following endpoints:

• v2/topic/{id}/formatted
• v2/topic/ident/{type}/{source_id}/formatted

2. External Schema/Services credential type rendering bug.

When 'name.' + not found in translation resources, displays as name., just display .

3. Adds documentation for updating BN in db Name table

4. Resolves errors with Swagger pages not loading in development

Issue introduced by #591 and #602

The DEBUG variable was previously enabled by default in development modes. This flag automatically handles static files in Django for displaying Swagger pages. The variable was also linked to the DJANGO_DEBUG variable. The latter was repurposed to allow for remote debugging of Django in development and disabled by default resulting in issues with static files and displaying of Swagger pages.

Change log:

• DJANGO_DEBUG variable has been unlinked from the DEBUG variable.
  • Django debugging can now be enabled separately by setting DJANGO_DEBUG when staring the docker containers
• DEBUG is re-enabled by default in development mode again

5. Fix typo in api/v3/views/rest.py

Fixes typo that would prevent the code in the UI verify logic (see here ) to work.

6. Topic search refactor

This PR introduces major updates to search (v4).

Topic searches in previous versions were based on Credential attributes that were linked back to their owner Topics. This resulted in duplicate Topic results when query terms match multiple Credentials. v4 introduces Topic search endpoints that are based on Topic attributes. Topic attributes are an aggregation of their owned Credential attributes. This change will require rebuilding SOLR indexes.

Change log:

Server

• Introduces new search API endpoints for Topic (/v4/search/topic) and Credential (/v4/search/credential). Search API endpoints for Credential are derived from v3
• New index fields introduced for Topic that allow searching on Name, Address and Source ID
• Fixes issue with filtering Topic/Credential by category facets, where result set was not narrowed down by selection
• General code cleanup

Client

• Upgrades basic and advanced search to v4 from v3
• Adds new facets to advanced search for Credential types
• Upgrades autocomplete to v3 from v2, which expands autocomplete beyond Names to include Address and Source ID
• Improves advanced search handling of URL query parameters to allow for bookmarking of queries. URL query parameters are also carried over from basic search to advanced search
• Aligns historical facet filtering for basic and advanced search
• General code cleanup

7. Default DEBUG to false

Some issuer functions don't work when DEBUG=true

8. Add project lifecycle badge, license badge

Resolves #599

Added project lifecycle badge. For lifecycle badge information, see: https://github.com/bcgov/repomountie/blob/master/doc/lifecycle-badges.md

9. Adds return statement for function generating remote names

10. Angular upgrade

This PR upgrades the client Angular application to 11.02

Change log:

• Upgrades Angular to 11.02 (Note: this upgrade removes support for IE <= 10)
• Replaces localization-router node package and associated http loader
• Updates style imports

11. Adds collapse list of alternate names for credential search results

This PR adds topic display names and a collapsable element of alternate topic names/identifiers to OrgBook search results.

Change log:

• Fixes an issue where topic search results showed a blank name if topic credentials did not use prescribed name types.
• By default, the topic source_id is displayed for topic search results otherwise the display name is derived from name types in the following order:
  • display_name, entity_name_assumed, entity_name
• If topic names other than the display name are available, remaining names/identifiers are displayed in a collapsable element
• Removes local and remote names on credentials unless an entity_name and entity_name_assumed is mapped to credential
• Adds localization to Credential Types dropdown in Advanced Search

12. Add remote debugger configuration and docs for vcr-server.

The remote debugger for vcr-server is now available when setting DJANGO_DEBUG=True. The default value for DJANGO_DEBUG has also been set to False, to avoid running the debugger unintentionally.

13. Audit and update production npm package vulnerabilities

[swcurran]

@amanji -- are there any other v2 and any v3 API calls other than the "formatted" one that might be affected by the recategorization of "BN" as a name? I'm just worried about other OrgBook API clients that are out there.

Nice work on the collection of changes. I reformatted them a bit to make them easier to consume by API users.

[amanji]

Will compile a list of endpoints affected by the BN update

[amanji]

These are the endpoints that will be affected by the name data additions. In majority of the cases it is in the Topic that is referenced in a Credential of interest that has multiple Name entries whereas the base data remains unchanged.

• v2/credential/{id}/formatted

• v2/topic/{id}/credential

• v2/topic/{id}/credential/active

• v2/topic/{id}/credential/historical

• v2/topic/{id}/credentialset

• v2/search/credential

• v2/search/credential/{id}

• v2/search/credential/facets

• v2/search/credential/topic

• v2/search/credential/topic/{id}

• v2/search/credential/topic/facets

• v3/search/credential

• v3/search/credential/facets

• v3/search/topic

• v3/search/topic/facets

• v3/search/topic/attribute/{attribute_query}

[swcurran]

Let's talk about the potential impacts. Does the orgbook-api repo "just work" still? E.g. for anyone following that pattern, will the user interface keep working? If so, what APIs does it use from this list?

We can take the /v4 ones off the list.

Do you think there are any risks that other groups are using this code?

[amanji]

Tested against the latest dev deployment, didn't discover any issues with client functionality. Also tested latest API changes against the orgbook-api demo and the demo works as previously.