Guardian analytics: insights and top down data way points

[anvabr]

Problem description

Guardian policy flows can generate significant amount of interlinked and interdependent data. The overall scale of industry activity, and the complexity of real-life methodologies reflected in the volume of generated data, can, in practice, severely restrict the usefulness of the data. Too much data, where the sought data cannot be found, or where the volume or structure prevents anyone from 'making sense' of the data, for all intents and purposes is the same as no data - because the data is inaccessible and incomprehensible.

Requirements

The overall goal is to create a tool which provides non-technical high-level users (banks, auditors, analysts, accountants, etc) with insights out the box, and can serve as a way point for further data analytics, discovery and diagnostics.

• Introduce an AI data analytics backend which automatically and continuously processes and analyses Guardian data across the entire ecosystem with the purpose of identifying patterns, dependencies, trends, important factors and variables, groups (cohorts) of data, etc.
• Develop an analytics UI and visualisation tool which is useable by non-technical users. This UI should allow users to continuously 'zoom in' or 'shift horizontally' on the data or the analytics results to examine them on a more detailed level or for a different parameter set: a different location, industry, sector, time range, specific values etc.
• Implement functionality to visualise and search the structure (or organization) of Guardian data, which allows users to determine for example which schemas refer to particular attributes, in which policies they are used, and then seamlessly navigate to which data has been generated that's connected to these.
• Provide an intelligent search interface which allows users at any point, while looking at a selection of data or an analytics result, to search for a 'similar' data or analytics result, while parameterising the search with additional criteria. E.g. 'find similar set of MRVs but from projects in West Africa and those produced between 2021 and 2023'.
• Allow users to save the configuration of their searches, UI settings/configurations and other parameters generating analytics results, etc so users can repeat and further enhance their previous analysis later - with newly available data.

Definition of done

• Requirements above are satisfied with a Guardian module which can be optionally run.
• Hardware and storage requirements of this optional analytics module are understood and published in a sizing guidelines.
• User-level documentation is created.

Acceptance criteria

The goal of this initiative is achieved when non-technical users:

• find the out of the box analytics insights useful
• are able navigate the analytics UI to discover and make sense of the data relevant to them

[anvabr]

Example use-cases/views that have been requested by the community:

• ability to query accounts tied to policies/tokens and data, such as:
  • accounts/DIDs involved by project (tied to a token ID)
  • drill into the information of where those accounts/DIDs made attestations and the questions they're answering in a schema tree view tied or role tied to a workflow
• list of all formulas/calculations in an item of groups of items (such as Tool[s], Policy[ies], Schema[s], etc)
• token stack (bar chart) by project developer/proponent or by methodology or location
• pull a corresponding schema tree from a methodology or pull up a policy comparison
• bookmark for comparison at a methodology [data, document, or module level]
• color-code the schemas, or borders as indicators of VC/document type and/or differentiate in the policy who issued it
• apply tagging directly from the UI
• view the data superimposed on the geographical location on the map (like OpenForest)

And a suggested list of nice-to-have features (mostly applicable to the 'tree' or 'network' view of the document relationships):

• show somehow the type of the relationship between items (i.e., with different line format and a legend, or with text)
• filter by relationship type (child, father, sibling, etc.)
• reorder by timeline (VC creation date)
• ability to customize the label (in the image "VC-Document", "VP-Document", "Role-Document") to show a specific attribute of the VC
• increase the size of the circle based on an attribute of the VC I can specify
• highlight VC/VP with the same value for an attribute (i.e., same owner, same location, etc.)
• let me customize the tooltip content with specific attributes of the item
• visualize if the item is part of the same policy
• actions
  • single click to freeze the highlighted visualization, so I can move away the mouse without losing the relationships, and I can go clicking on a related item
  • single click on the blank background to unlock the frozen visualization
  • double-click the item to open the small popup window with raw data, not replacing the current page so I can retain the current selected visualization

[dubgeis]

@anvabr This ticket is for the top down to define the way points [or even go as far as telling the market what's missing]. ie I am a national government, or an inter-governmental organization like the World Bank or IMF and they need data. They should be able to articulate this and they would need the use cases and views listed above. They would also need the ability to say I need more and define what that looks like.

[anvabr]

Useful link: https://icvcm.org/core-carbon-principles/