Open transitive-bullshit opened 4 months ago
We definitely want to release a Node SDK at some point, though I'm undecided on which of a few approaches to take:
The other thing to decide would be which API to surface.
For now, I ended up going with option 1.
You can see my PoC here with more info on the PR and motivations here. Note that this project is not OSS yet, but it will be hopefully by end of week, so I'm just documenting my exploration as I go for anyone else who's interested in this type of integration.
So far, using grit
as a subprocess with @getgrit/launcher
as an optional dependency seems like a good solution to maximize compatibility, with the linting engine still working even if grit
fails to install on some platforms. I'm currently only using grit apply --dry-run --jsonl
with a readonly pattern, and this is only enabled for any markdown rules which contain an optional grit
or gritql
code block like this one: https://github.com/gptlint/gptlint/blob/feature/gritql/.gptlint/prefer-array-at-negative-indexing.md.
I've done an initial analysis on using gritql
to filter the source file context sent to our LLM-based linting engine, and it reduces the context significantly for several of the built-in rules, which is amazing. I still need to do a full suite of evals w/ gritql ablations to see how this impacts rule accuracy / precision / recall / cost / speed at scale, but I'm really happy with the results so far. A lot of files which we were previously naively linting can be quickly ruled out without doing any LLM calls if there are no gritql
matches.
btw if anyone else is exploring similar integrations and wants to chat, feel free to DM me. Cheers 👋
Hey hey 👋
Had a great chat w/ Morgante yesterday & am considering integrating
gritql
into our LLM-based linting pipeline.I think the main blocker will be exposing
gritql
as an NPM package so we can use it programmatically from JS/TS instead of invokinggritql
as a subprocess.I'd be happy to work on this after we launch. Just wanted to open an issue to discuss and get feedback on 1) whether your team thinks this is a good idea and 2) any potential blockers we're likely to run into.
Really really love
gritql
&& excited for its potential!Thanks 🙏