Allow user context in relation transformers

[Kixiron]

I've got a situation where I'd like to construct one of two different dataflows within a transformer based on the external context of the program. When in the program using ddlog is in CLI mode I want to construct one dataflow optimized for that and when it's in LSP mode I'd like to use another optimized implementation. Being able to pass context to application nodes would allow this, maybe something along these lines:

use type_map::concurrent::TypeMap;
use std::any::Any;

// DDlog api
impl HDDlog {
    pub fn insert_transformer_data<T: Any + Send + Sync + 'static>(&self, value: T) -> Option<T> {
        self.transformer_data.insert(value)
    }
}

// Relation transformers, could be made opt-in by using an attribute on the transformer
pub fn MyTransformer(transformer_data: &TypeMap, ...) -> ... {
    let program_mode = transformer_data.get::<ProgramMode>().unwrap();

    match program_mode {
        ProgramMode::Lsp => { .. }
        ProgramMode::Cli => { .. }
    }
}

// Api usage
let (hddlog, _) = HDDlog::new(1, false).unwrap();
hddlog.insert_transformer_data(ProgramMode::Lsp);

[ryzhyk]

Why not introduce an extra input relation with a single record that represents program mode or any other metadata the transformer needs and pass this relation to the transformer. This way you can support this use case without making changes to the compiler.

[mihaibudiu]

How can you find out in DDlog whether you are in a CLI mode? Perhaps we can have a "system" relation that exposes such attributes. And an ENV relation that exposes (read-only) environment variables. etc.

[ryzhyk]

If you really need to know the mode while constructing the dataflow, why not just use a global variable that you can set from Rust and read in the transformer code?

[Kixiron]

Why not introduce an extra input relation with a single record that represents program mode or any other metadata the transformer needs and pass this relation to the transformer. This way you can support this use case without making changes to the compiler.

The values from the input relations won't exist until runtime (as opposed to dataflow construction time)

If you really need to know the mode while constructing the dataflow, why not just use a global variable that you can set from Rust and read in the transformer code?

Yes, this is an option, but global variables aren't very good and I'd prefer an alternative

How can you find out in DDlog whether you are in a CLI mode? Perhaps we can have a "system" relation that exposes such attributes. And an ENV relation that exposes (read-only) environment variables. etc.

For this in particular the idea is that the rest of the datalog code doesn't actually need to know anything, this is just an optimized implementation of the relation itself

[ryzhyk]

Yes, this is an option, but global variables aren't very good and I'd prefer an alternative

They might be the lesser evil in this case. Adding an ad hoc feature to the API seems even less pleasant to me. But if we go down this path we might need to merge the API refactoring PR first. It moves some of the definitions from the api module to differential_datalog, which may help.

[ryzhyk]

@Kixiron , any objections is I close this as wontfix? Or do you still think we need this feature?

[Kixiron]

Well, I opened it because I need the feature. Multiple of my projects involving ddlog are language-based ones that already have things like interners and arenas that are used throughout the rest of the program and make up much of the input data. Without being able to pass in interners and all that jazz I currently have to deserialize the data at the ddlog boundary and serialize it within ddlog (and vice versa for outputs), making it not ideal in a lot of ways

[ryzhyk]

Isn't that #804?

[Kixiron]

It's both, that one is for doing it in transformers and this one is for doing it in vanilla relations