search

vmware / differential-datalog

DDlog is a programming language for incremental computation. It is well suited for writing programs that continuously update their output in response to input changes. A DDlog programmer does not write incremental algorithms; instead they specify the desired input-output mapping in a declarative manner.
MIT License
1.37k stars 118 forks source link

Feature gate unused Rust files? #411

Open d-e-s-o opened 5 years ago

d-e-s-o commented 5 years ago

Reviewing https://github.com/vmware/differential-datalog/pull/410 @mbudiu-vmw brought up the point that some (tangentially more and more) of the Rust files we copy into a Rust DDlog project may not be used, suggesting we may want to gate them somehow.

I suspect it may be perceived as unnecessarily limiting if not everything is there. People would probably expect a Cargo feature flag (which could be an implementation variant) instead of not having files present to begin with. We also need to be careful with dependencies among files/crates.

But perhaps we should start by clarifying the problems we see with the current approach. Is it only that files are lying on disk?

d-e-s-o commented 5 years ago

@mbudiu-vmw as you brought up the issue, what problems do you see with the existing approach?

mihaibudiu commented 5 years ago

I find it very weird to compile a fibonacci program and then to get a file called ovsdb in the generated directory

d-e-s-o commented 5 years ago

I see. But the files are at least not compiled unless the crate is used, are they? That is, the file(s) being present is the only drawback. It seems that unless we are talking a huge amount of files, that is something that is not too big a concern.

mihaibudiu commented 5 years ago

Maybe this is not an urgent issue, but it is an issue which we'll have to resolve. Clearly, we cannot keep adding unrelated code there.

d-e-s-o commented 5 years ago

But the files are at least not compiled unless the crate is used, are they?

I think they are actually compiled.

Well, let's dissect the files we are (I am) concerned with (in rust/template):

These files are all compiled (except for flatbuf) by virtue of being modules.

Then we have directories (crates):

So, when I just compile a generated Rust project I see observe, tcp_channel, differential_datalog being compiled as well.

I guess we need to make a call where we want to have feature gates. We already have a Cargo feature for flatbuf and so I would propose to add another one for the distributed parts. I am thinking about how to reorganize things a bit such that everything related to d3log is in a single directory in the template, not spread in a flat manner as is the case currently. Of course, that still doesn't solve the files-on-disk problem. But again I'd use caution having a two level gating system (although I believe we already have that in a way with the -j flag to ddlog itself), as that quickly gets complex and can be a source of confusion and non-obvious errors.