Rework build system / release management. Consider publishing on a central

[ComFreek]

tl; dr: the development workflow for anyone working on a project on top of MMT with the requirement to change MMT every now and then is lacking.

Current issues

• There's no easy way to create projects depending on MMT

  • MMT is not published (as a package) anywhere, instead only JARs are provided. Unmanaged dependencies always complicate life more.
    • ⇨ Polish build system before publishing, especially fix #437, #438.
      • ⇨ Publish on https://search.maven.org/
      • ⇨ Alternatively, set up a self-hosted instance

• There's no easy/documented way to reference MMT in a multi-project SBT Setup, cf. #441 Fixed by #449, documentation outstanding in #441

  • This is crucial for quickly fixing issues or adding features to MMT while testing them in one's own project.
    Otherwise, one had to republish the new version and update the dependency in one's own project. This is time-consuming.

Motivation

Currently, I am publishing MMT locally on my PC. Recompiling and republishing takes ~12 minutes, which is not due to my PC, but rather that SBT cannot perform iterative compilation when packaging somehow.

[Jazzpirate]

We will need to discuss this in more detail. The documentation used to contain a specific workflow for MMT-based projects. Apparently it's now been rewritten such that it advertises a "wrong" workflow. Might be that I misunderstand a couple of things though. Generally, creating a new sbt project to work on MMT should be strongly discouraged, depending on the specifics.

[ComFreek]

Generally, creating a new sbt project to work on MMT should be strongly discouraged, depending on the specifics.

What else do you suggest? Many resources recommend SBT for Scala-based projects, but I am not opposed to other commonly used build systems if there are any. Unless you are developing MMT itself I don't count working inside the MMT repository or on any branch of it as a good/feasible practice at all. Proper dependency management needs to be introduced anyway at some point in the future when the monolithic repository will be split up (which I hope it will be).

Any proper solution, which then builds on top of dependency management, will need #447 by the way.

Apparently it's now been rewritten such that it advertises a "wrong" workflow.

I am sorry, that's me who introduced the apparently wrong workflow. (IIRC I only added the "getting started as an MMT API user" tutorial, but didn't change any preexisting workflow recommendations.)

[Jazzpirate]

Ah, okay, I was assuming we are talking about "developing MMT itself", or projects that ultimately will end up being a part of MMT. Which yours for example will be ;)

but didn't change any preexisting workflow recommendations.

The relevant part for you would be https://uniformal.github.io/doc/setup/devel.html but clearly that also needs more information wrt test classes and running them from within IntelliJ ;)

Proper dependency management needs to be introduced anyway at some point in the future when the monolithic repository will be split up (which I hope it will be).

That's very likely not going to happen. The individual modules are way to interdependent. Analogously to this, but even worse because of constant student fluctuation and a lack of dedicated funds and full time software engineers: https://cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/fulltext

With respect to people outside of our group who want to build projects on top of MMT that are not part of MMT: They should simply add the mmt.jar as an unmanaged dependency. It's the simplest way to go about things and the jar is entirely self-contained. This has to the best of my knowledge never caused problems, any I'm confused why the unmanaged lib-jars should be a problem, since they're packaed with the mmt.jar...?

[rappatoni]

In light of this I have taken the liberty to add a disclaimer to the "getting started as an MMT API user" tutorial redirecting budding MMT developers (like myself) to the "correct" workflow for developers. I will hopefully be able to add some documentation for the testclasses, especially Magictest soon.

[ComFreek]

@rappatoni I am using my "incorrect" workflow #449 for more than two weeks now and it is working very fine! Finally, I can have files in my project which are not pushed to MMT (e.g. local prototypes, test files) and for which I do not have to manage a .gitignore or "manual" ignore strategy while git adding. Is there any disadvantage of it or advantage of the "correct" method I currently do not see?

[rappatoni]

@ComFreek I have not been able to add the MMT sources as sources for the external library but maybe you can help me with that problem next week.