search

bcgov / MFIN-Data-Catalogue

The Finance Data Catalogue enables users to discover data holdings at the BC Ministry of Finance and offers information and functionality that benefits consumers of data for business purposes. The product is built using Drupal and adheres to the Government of BC's Core Metadata Standard.
Other
6 stars 0 forks source link

Consistent pattern for describing database data set location #404

Open ChristaBull opened 3 months ago

ChristaBull commented 3 months ago

User story

As a user I want a way to consistently identify where a dataset located in a database can be found

Describe the idea

The location for a metadata record describing data that resides in a database isn't a single path (e.g. http://some-location.com) like other data set types. In order for someone to understand where a database data set resides they often need to know several pieces of information such as the database server name, database name, schema name, and entity (e.g. table) name.

Instead of capturing these details in the single existing text box could we have an text input for each specific detail describing a database data sets location so that the information can be consistently communicated to the public or internal users appropriately.

NicoledeGreef commented 3 months ago

Note: Location of a resource only displays to logged in users (no public access).

We should identify all the different structures that are expected. MS SQL Server is just one case and it is covered by database server name, database name, schema name, and object name.

postgres and HANA are likely different.

NicoledeGreef commented 2 months ago

We have discussed different approaches to addressing this but I am going to icebox this for future reference. This will allow for us to collect some metadata within the system for 6 months or so which we can then analyze to inform future enhancement. By leaving the Location text area as it is, it is flexible for all assets types and it will be interesting to see what is entered.

For SQL Server, we can train to a standard such as: [iapetus\findata].[FIN_SHARED_DATA_PROD].[staging].[object_name] and for other databases, similar.

Without changing the Location text area field at this time, we could implement validation so that when a user selects SQL Server as the database type for a Data asset, the validation routine checks to see that what has been entered adheres to the desired pattern.

The validation doesn't have to look for completeness but rather that there is at least one value within square brackets and separated by periods. That is to say that it should not prevent a user from saving if they have not entered all of the values. Or it could be implemented as a visual warning to the users as opposed to prevention of a Save.

@ChristaBull - what do you think of this validation suggestion?