search

blacksmithgu / datacore

Work-in-progress successor to Dataview with a focus on UX and speed.
MIT License
1.5k stars 16 forks source link

lookup columns, checkboxes, formulae? #11

Closed bcmatteo closed 4 months ago

bcmatteo commented 1 year ago

Thanks for creating this, and all you did with dataview!

I'm not a notion user, but have extensively used coda. Three of the best features there are lookup columns which enable the creation of on the fly relational databases (and dramatically simplify maintaining lists of field values), checkboxes for interacting with boolean fields, and simple formulae, which allow for things like logging changes in boolean fields on the fly... I know this can sorta be done now in obsidian by stitching together code and plugins, but it gets awkward and laggy fast...

I looked at your (impressive) roadmap and could't tell if you were thinking along these lines. +1 if you are... Would love to ditch coda and consolidate on obsidian for tasks and the more complex relational stuff that I do...

blacksmithgu commented 1 year ago

Checkboxes and formula will be present (as custom columns). This also implicitly means lookup columns will be available (since you can run subqueries inside of a query). I'm not super confident in the performance of lots of lookup columns in big tables but I've generally found rendering takes 90%+ of total query time so perhaps it may just work great!

bcmatteo commented 1 year ago

awesome.

on the performance side, any chance you will be implementing a manual update option, or some sort of cache for formula results? e.g. lets say all my notes have a type field that has in total maybe 20 values and I want to be able to select a value and filter a table on it. Ideally I don't burn performance by constantly updating the select list when the search requires the whole vault, but the values update infrequently. I've run into versions of this problem with dataview which can bring obsidian to its knees- e.g. 200 notes sorted by group in dataview js can bog down when viewing and get completely stuck if updating a value in the table. I imagine that if there was some way to limit the refresh these problems would go away. but maybe this is a limitation in my understanding of javascript?

thanks for building these plugins. they are great.

On Fri, Sep 15, 2023 at 1:18 AM blacksmithgu @.***> wrote:

Checkboxes and formula will be present (as custom columns). This also implicitly means lookup columns will be available (since you can run subqueries inside of a query). I'm not super confident in the performance of lots of lookup columns in big tables but I've generally found rendering takes 90%+ of total query time so perhaps it may just work great!

— Reply to this email directly, view it on GitHub https://github.com/blacksmithgu/datacore/issues/11#issuecomment-1720870147, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACPOSM52X6PLCGQJ66ETVKLX2QFMRANCNFSM6AAAAAAZJPO2MA . You are receiving this because you authored the thread.Message ID: @.***>

blacksmithgu commented 1 year ago

Datacore does paging by default which limits the number of visible results, which is a big performance bump. Currently refreshing is controlled via a delay (the higher delay means the longer the plugin will wait before refreshing; negative delay means do not refresh). Views only update if a relevant page in the search has actually changed (using some diffing logic that was not present in dataview).

My hope is to avoid needing "manual refresh" by making the default refresh functionality sufficiently fast.