Allow passing additional modules to `compile`

[max-sixty]

What's up?

From https://github.com/GlareDB/glaredb/pull/1895

As discussed above, the module system in PRQL is quite nascent. But it's also quite close, and so to the extent it would be helpful for you, that would be a useful impetus for us to polish it and document it. What specifically would be most helpful for GlareDB there?

When I was implementing the integration, I thought that it'd be preferred if external modules could be registered. Either through the Options, or passed directly into compile. I'm happy to open up an issue in prql with more details as what would be preferred from our side; we have a lot of custom table & scalar functions that probably would be best maintained in our own repo than within prql's internal modules.

It's a bit different from our existing module system, but I think it should be viable; I can see "a library of functions that a query can call into" being quite a good pattern.

Any thoughts?

[aljazerzen]

The compile function is meant to be the "simple" interface, for compiling a string to a string.

We do already have _tree() function variants that work on SourceTree.

SourceTree is basically a map from filenames into their contents, which is exactly what we need here, I think.

[max-sixty]

Yeah. Possibly we could provide a helper method so that if folks have a utils.prql string and want to call functions from that from foo.prql, that's easy?

Maybe we make it possible to create a SourceTree in memory, from text?

[aljazerzen]

I mean, this is as easy as it gets:

pub fn compile_tree(prql: &str, util: &str, options: &Options) -> Result<String, ErrorMessages> {
    let sources = SourceTree::new([
        (PathBuf::from("Project.prql"), prql.to_string()),
        (PathBuf::from("util.prql"), util.to_string()),
    ]);

    prql_to_pl_tree(&sources)
        .and_then(|ast| pl_to_rq_tree(ast, &[]))
        .and_then(|rq| rq_to_sql(rq, options))
        .map_err(|e| e.composed(&sources))
}

[max-sixty]

Ah nice, that is good.

I mean, this is as easy as it gets:

You may be underestimating the benefit of nice abstractions :)

But I think that's would be a reasonable place to start for folks like GlareDB, and then we can make a nicer abstraction if that'd be helpful. WDYT @universalmind303 ?

[aljazerzen]

Yeah, I might be biased toward exposing IRs of the compiler and believing that using that is easy :D