blacksmithgu / datacore

Work-in-progress successor to Dataview with a focus on UX and speed.
MIT License
1.48k stars 15 forks source link

[Request] Possibility for plugins to provide custom metadata for querying #18

Open Fevol opened 1 year ago

Fevol commented 1 year ago

I'll start off by saying that I have no idea whether this is even feasible, I haven't the slightest idea if the code can easily be extended to support this feature, or if it requires an entire rewrite. Feel free to close this issue if you feel that it would not be possible.

Description

Problem statement: plugins may extract their own metadata from notes, which are not necessarily contained within the frontmatter

Example 1 (arbitrarily chosen simple plugin) For every note in the vault, the plugin keeps track of whether a note was changed during the current session (since app start-up).

In-memory, this plugin contains a mapping of changed: Record<TFile, boolean>, with a 'true' result indicating that the file has indeed been changed.

The user of the plugin would like to be able to use this metadata to get an overview of all the files that they changed located within a certain folder, using the datacore plugin they already have installed.
Example 2 (my plugin) The plugin processes custom syntax and renders it within notes. All the occurences of the syntax are also being stored within a database (something akin to cache: Record <TFile, Syntax[]>), where the syntax itself can contain a list of custom metadata.

Similarily to the previous example, the user wants to list all the occurences of the syntax within a certain folder, but also filter it on the custom metadata within the syntax itself.

Idea

Each of the plugins described above is able to provide custom metadata given a file, which could in turn be used when querying data. The existence of these datatypes could be registered as such:

export class MyPlugin extends Plugin {
    database: Record<TFile, boolean>;
    async onload() {
        this.app.workspace.onLayoutReady(() => {
            if (datacore)
                this.registerAPI(
                    datacore.registerMetadataProvider('myplugin', (file: TFile) => database[file])
                );
        });
    }

    onDatabaseUpdate() {
        datacore?.triggerQueryUpdate();
    }

    // And some code for unregistering metadata provider on plugin unload
}
(Example code) 

Once registered, users would now be able to also use the metadata provided by the plugins within their query requests:

```datacore
LIST
FROM #unfinished
USING myplugin
WHERE myplugin = true
``​`
(Example based on current Dataview syntax, names/keywords used are just for demonstrative purposes)
xDovos commented 1 year ago

i think it won't be possible like this but you could just write an simple api that gives access to the data so that you can load your data inside the js. for example currently you can already do something like this in dataview by loading the api with app.plugins.plugins.myplugin.api or something like this (this is how you can load the dvjs api inside other plugins.)

blacksmithgu commented 1 year ago

There are two aspects to this:

  1. Having plugins add new objects to the datacore storage (which can then be queried / loaded / cached) is fairly easy - all you need to do is use the datastore store APIs and provide some types to store.
  2. Having plugins add new fields to existing objects is a bit harder - essentially, you would have to hook into some post-processing whenever datacore reindexes a page so you can add new fields to them. You would also need to be able to ask datacore to re-index an object because some of your local metadata has changed.

I think (1) is very easy to support and is essentially already supported. For (2) I would need to provide hooks for allowing objects to be decorated with new fields, and to re-index objects outside of the regular page reload. Maybe some basic APIs for that aren't so bad.