Metadata attribute configurable by Admin as mandatory

[ozarkblue]

I'm trying to come up with some governance process for artifact creation and maintenance in Registry. Now this attribute metadata is very good option to capture additional metadata details. That will help for search and adoption. Now as this feature is free for all developer role to update hence we've to keep a review option before any upload to enforce and ensure certain properties (like Dept name, product id ,channel detail etc) are added by teams.This way all artifacts will be consistant.

It would be much better if we have options so that Admin can create and add few properties (making some as mandatory ) for artifact creation and apply that for certain artifact groups/ all groups / all artifacts etc.That way we can take out this manual process and ensure some amount of metadata governance is driven by tool feature.

Starting this discussion to bring out thoughts on this. Above is just one use case.

@EricWittmann thoughts ?

[apicurio-bot[bot]]

Thank you for reporting an issue!

Pinging @jsenko to respond or triage.

[EricWittmann]

Interesting proposal. When you mention metadata I assume you're referring to the configurable/editable Tags and Properties that users can edit for each artifact, right? You're not referring to something in the actual content of your schemas? I think so but I just wanted to confirm.

I think this might be an interesting application of our Custom Rules requirement (not yet implemented). This would allow users (admin users only perhaps) to define custom rules that could be applied in addition to the existing Compatibility and Validity rules. If the custom rule feature were sufficiently customizable it would be possible for the registry to reject content if insufficient metadata were provided.

That might not be what you're looking for though - could you perhaps describe the data or user flow you had in mind in more detail?

[EricWittmann]

One other thing to note is that we never imagined that Registry would have strong governance features. Instead we assumed that any governance needed would be done externally to the registry, with for example a single Service Account user as the only account allowed to make changes. So essentially some kind of CI/CD or governance process that would result in an API call to put the schema into the registry (for use cases where strong governance was required).

WDYT @Apicurio/developers ?

[ozarkblue]

Thanks @EricWittmann for you comments above. Sorry for late response. Yes implementing this with rule could be one option. and Yes this is for additional metadata 'property' This is what I envisioned:

1. Admin user should be able to open a property management screen (or section of setting screen)
2. Option to add /update/delete property
3. Option to apply each property in multiple levels like - group level (then specify group names with comma separate,) or global level
4. mark each property mandatory or optional by default
5. once a property is added only that will be applicable for new assets or new version that is uploaded. Can skip old assets as that way you can avoid conflict with existing artifacts
6. based on this configuration once new artifacts added/version uploaded if valued are not passed then error message should Cleary specify proper name and applicability and mandatory - to inform user to update valued and reload.
7. once added the mandatory fields should mark with * next to key field - this will further inform and educate users to identify these and take care going fwd.

Regarding Governance feature - Even though CI flow is being used with a service ID but for large Org there are multiple teams that are managing their artifacts in Git repo and using CI flow to upload. Having this feature will help to bring some inbuilt governance feature.

On a personal effort- At present I've created a CI flow though(that's one of the reason to be late responding this thread in last couple of days) to upload artifacts in registry from Git repo and planning to take care of these with additional validation as part of CI flow - which I plan to take out later once this is part of tool in future. Also any update with UI will be covered too this being part of UI feature. As some consistency is necessary and will help uniformity of schema metadata when search-and-find is easy though UI already (this is one of good feature I like in Apicurio).This way some mandatory metadata can be enforced like - business unit name, product team name, point of contact email, event channel name etc, across artifacts from multiple teams.

[EricWittmann]

Thank you for providing this extra context, @ozarkblue - it's very interesting.

I do still think the right approach is probably based on Rules. We need enhancements in that area to support this, but since rules are already applied at the appropriate levels (global, artifact and in the future group), I think it makes sense. What would be needed is rule configuration and enforcement at the group scope plus a smarter UI when configuring rules, since this configuration is much more complex than existing rule config.

WDYT @Apicurio/developers ?