Open laurastephsmith opened 2 years ago
I've tested in a clean vault, with 100 blank notes. Adding the 80,000 files does not cause the problem to appear. But adding copies of my 800 notes from my production vault causes the problem to appear.
So I suppose either:
Problem occurs in test vault with 800 blank notes (when the 80,000 other files are present).
The additional 180Gb of files you are adding are not markdown files, right? I wouldn't expect Dataview to crash and burn on this since it ignores non-markdown files. Do you get any errors in the developer console?
Correct - photos, videos... 30 years of junk! No errors in the console. And to reiterate, the problem doesn't exist in 0.4.26.
I've decided against putting all my files inside my vault directory now anyway, so this bug won't affect me. The quick switcher just... wasn't very quick any more :laughing: - but I'm very happy to test for you if you need me to!
I made a few small fixes to improve this a little more, though I'm still not sure what changed between 0.4.x and 0.5.x to break this; I'll need to add some more metrics tracking / logging to narrow this down.
Okay - if you want me to test it when you've done that, let me know.
What happened?
Dataview Version Affected
Issue exists in 0.5.16 but does not exist in 0.4.26.
The Problem
When adding about 80,000 files (180 GB) into a vault, Dataview basically stops working. Queries can only return a small number of notes in the vault. Any query that's expected to return any other notes returns nothing.
This is not a stress-test by the way! I've moved my Obsidian vault to be the root of my Dropbox. I want to be able to access any of my Dropbox files from within Obsidian.
To Reproduce
System Info
The bug was still present with all plugins disabled except for BRAT and Dataview. But this output here is with all my plugins enabled.
SYSTEM INFO: Obsidian version: v0.14.6 Installer version: v0.14.6 Operating system: #1 SMP Debian 5.15.15-1 (2022-01-18) 5.15.0-3-amd64 Login status: not logged in Insider build toggle: off Live preview: on Legacy editor: off Base theme: light Community theme: Typomagical Snippets enabled: 12 Safe mode: off Plugins installed: 48 Plugins enabled: 48 1: Divide & Conquer 2: Advanced Mobile Toolbar 3: Obsidian Big Calendar 4: Breadcrumbs 5: Calendar 6: Convert url to preview (iframe) 7: Customizable Sidebar 8: Cycle through Panes 9: Editor Commands Remap 10: Format Hotkeys 11: Highlightr 12: Macros 13: MetaEdit 14: Note Refactor 15: Apply Patterns 16: Auto Link Title 17: Collapse All 18: Code Editor Shortcuts 19: Excalidraw 20: Excel to Markdown Table 21: Filename Heading Sync 22: Footnote Shortcut 23: Frontmatter Tag Suggest 24: Global Hotkeys 25: Hider 26: Hotkeys for specific files 27: Kanban 28: Link Archive 29: Linter 30: Local File Interface 31: Pandoc Plugin 32: Pocket 33: Shell commands 34: Smart Typography 35: Smarter Markdown Hotkeys 36: Sort & Permute lines 37: Style Settings 38: Task Collector (TC) 39: Tasks 40: Text Format 41: Tidy Footnotes 42: Version History Diff 43: Zotero Desktop Connector 44: Obsidian42 - BRAT 45: Omnisearch 46: Snippet Commands 47: Tag Wrangler 48: Dataview
[^1]: Test query in its entirety is
TABLE o
, where o is a field I'm using in many notes.DQL
No response
JS
No response
Dataview Version
0.5.16
Obsidian Version
0.14.6
OS
Linux