SpyglassMC / Spyglass

Development tools for vanilla Minecraft: Java Edition data pack developers.
https://spyglassmc.com
MIT License
302 stars 31 forks source link

⚡️ provide `exclude` patterns to file watcher args #1607

Open TheAfroOfDoom opened 18 hours ago

TheAfroOfDoom commented 18 hours ago

right now we filter files for Spyglass to care about after we've already told chokidar to watch all our root directories.

chokidar ends up watching every file in the directory and is very slow, even if we later ignore certain files based on config.env.exclude.

not only that, but i couldn't get exclude patterns to work that you'd expect to work. for instance:


datapacks/omega-flowey/data/*/test wouldn't match the corresponding test directory relative to any of the project roots (mostly fixed ✅)

so i'd get lots of spyglass errors due to packtest's non-vanilla commands just on project open

before after
image image

Spyglass still shows errors if you open the file, but this is progress.

we could fix ^ by hooking the exclude config into connection.onHover (etc.) somewhere, but that's out of scope here


explicitly ignored datapack directories would still get parsed and show up in references/definitions (also mostly fixed ✅)

i have a build directory that is .gitignored, and this would still show up when i open a function's references.

on project load, these files are no longer bound to the function references, but they unfortunately still are bound again if you open the file. still an improvement though, and we can probably remedy further with similar methods to the packtest related stuff above

before, project load -> parses the explicitly .gitignore'd directory
image
after, project load -> doesnt parse (still parses if opened though)
image

resourcepack/pack.mcmeta (from project root(s)) wouldn't match a RP pack.mcmeta; i'd still get pack version errors ⚠️

note that while the watcher is no longer watching this file if i add it to my exclude list, Spyglass still finds it because findPackMcmeta doesn't use the exclude config yet.

maybe less relevant though bc i know Misode's been working on improving that specific bug anyway


major performance gain

regardless of the above issues being partially remedied, the biggest gain we get is from chokidar not recursively adding every file under every project root to the watched files list. with a properly formatted ignored list, chokidar stops early at a directory that matches an ignore pattern.

profile comparisons:

#ready time

before, gitignore specified but broken parsing after
image image

watch file count

before, gitignore specified but broken parsing after
image image
TheAfroOfDoom commented 7 hours ago

TODO should maybe also bump cache version