[Design] Task-oriented module structure

[adamreichold]

I have tried to reduce clutter and simplify technicalities so far, but I think with the current status (no more library-binary split, support only for Markdown), we could reach a more code organisation by splitting our modules along the tasks, e.g.

• parse: Parse Markdown in documents a.k.a. list of blocks. Contains both the data structure as well as the parser producing it similar to parser::code now.
• print: Prints documents based on the documents created above.
• compile: Compiles source files based on the documents created above.
• reverse_compile: Merge source files into code blocks.

I am mainly hoping to be able to move a lot of code from the main module into these task-specific sub modules. But I also hope to reduce the "code scatter", e.g. unrelated functionality being in a single module like document as well as related functionality being scattered over multiple modules like compiling which mostly happens in the main module but also within document::code.

@mlange-42 What do you think?

[mlange-42]

This sounds like a very useful measure! I also think the structure you propose sounds reasonable.

In line with the restructuring, we may already prepare for what I proposed yesterday in #53, in case you would support that idea. However, when returning to the binary/library split for that sake, we probably don't need to expose much more than Document and its content structs.

[adamreichold]

I think #53 does not imply a library/binary split as those external processors would not need to be Rust programs at all. The stable interface for the external processors would be a JSON schema, not a Rust library.

We could factor out a Rust library that contains really only the serializable structures so that processor writers do not need to do the parsing themselves. But if our data model is so complicated as to need this, we should probably provide the same courtesy to all implementors and provide a proper schema, e.g. a Protobuf specification, that can be used from any language and provides more facilities for schema evolution.

[mlange-42]

Ok, that sounds reasonable. No real experience in this field so far on my side...

[mlange-42]

Done in #89, #98, #99, #102, #103, #104.