search

obi1kenobi / trustfall

A query engine for any combination of data sources. Query your files and APIs as if they were databases!
Apache License 2.0
2.3k stars 66 forks source link

Documentation for query language? #547

Open VorpalBlade opened 5 months ago

VorpalBlade commented 5 months ago

I have been looking around (readme, rust doc, crates.io) but been unable to find where the trustfall query language is documented. This should probably be linked from one or more prominent locations (readme, top level rustdoc etc).

If such documentation doesn't exist it should be added. I would recommend both some basic tutorials but also reference docs (what operators are available etc)

Without me understanding how to write a query for this it is very hard to evaluate if trustfall would solve my problem.

obi1kenobi commented 5 months ago

Agreed — working on it!

In the meantime, would love to hear about the problems you're looking to solve. Happy to answer any questions and onboard you personally if you choose to go for it. Open to a quick chat this week?

obi1kenobi commented 5 months ago

Right now the best public resource we have are the playgrounds, where you can see a variety of query functionality over data sources like the HackerNews API, or Rust libraries' data (as rustdoc JSON).

Each of the example queries has comments explaining what each bit does. We're working on a revamp of the playground as part of shipping full docs on the language and engine.

VorpalBlade commented 5 months ago

I found that trustfall_core apparently depends on async-graphql, that sounded familiar, and looking at it the syntax appears similar to graphql.

Is trustfall using (a variant of?) graphql? If so it should be documented in the README so that users who are largely unfamiliar with web tech (like me, I have a background in embedded/hard real-time) could even know that it is something we need to read up on.

My interest is in things related to some of the things you mentioned in the presentation of trustfall (implementing various types of code linting and refactoring, for a large C++ code base, yes I would love to rewrite in Rust, but legacy code doesn't dissappear over night).

I don't know how suitable trustfall would be, I assume it would in part depend on having a quick to query datasource, similar to the rustdoc json format? Does anything like that even exist for c++?

obi1kenobi commented 5 months ago

Trustfall reuses GraphQL syntax, but heavily extended and with very different query semantics in order to improve performance and add functionality. GraphQL doesn't natively allow things like aggregations, recursion, arbitrary filtering, etc. so reading GraphQL docs is generally going to give you an incorrect impression of what to expect when using Trustfall. For example, GraphQL queries return fully-nested data like { field: { other_field: { ... } } } whereas Trustfall returns tabular responses like what you'd get if you ran a SQL query.

The syntax is GraphQL-based so that we can reuse things like syntax highlighting, autocomplete, schema visualizers, etc. instead of having to write everything from scratch.

I'm not familiar with the C++ tooling landscape, but even if a rustdoc JSON equivalent doesn't exist, it's still possible to use Trustfall. We already have a Python integration where we just query Python ASTs directly and do our own "rustdoc JSON" equivalent to build tools on top of. I don't see why something similar couldn't exist on top of C++.

If you're thinking about building your own C++ integration and lints on top of it, I'd be happy to help you get started. Or if your organization would prefer to buy rather than build, I'd be happy to build it — if so, let's chat. You've probably seen how cargo-semver-checks defines lints as queries — a C++ linting and refactoring approach would look much the same.