Open review for ATOL solution

[agarciadom]

I was able to build and run the solution using the provided code, after adding atol to the list of tools in the config/config.json file. I did have to make sure I defined the GITHUB_ACTOR and GITUB_TOKEN environment variables to those of a personal token with the read:packages permission before running the script, but this doesn't seem to have been documented anywhere. Could the solution authors add a README.md to their solution that clarifies this?

Once I got the results, I copied over the src/main/viz folder in my Epsilon solution and quickly touched up the diagrams.R script to produce figures on memory/runtime. These pretty much reproduce the results on the paper, drawing up the same comparisons (ATOL being faster but significantly more hungry on memory).

I'll continue adding comments on the quality of the artifact and its paper below.

Memory, initialisation

Memory, load

Memory, run

Runtime, initialization

Runtime, load

Runtime, run

[agarciadom]

I notice that the .generated.atl file seems to have its rules in a different order, so I couldn't easily do a side-by-side diff to compare the two. One thing that I've liked to see in the paper is whether there is any type of tool support to let an ATL user know whether their .atl file has anything that wouldn't work in ATOL (e.g. some kind of IDE warning/error).

I found the frequent -- @type X comments a bit distracting. Are these needed, or is it just for helping debug the higher-order transformation?

There is also a change adding .value132 to the AdministrativeGender rule, which I didn't quite understand. Is .value132 part of the original metamodel, or is it something new produced by ATOL?

The RESOLVE helper seems rather hard to understand. Is your end goal to have users write regular ATL scripts and have the HOT "compile down" to the ATOL-compatible subset of ATL, or would you expect users to write directly ATOL-compatible scripts, including such resolvers?

Overall, this seems very impressive technically, but it'd be nice to know what is your end goal in ATOL regarding the intended user experience.

[agarciadom]

Regarding the paper, I found it generally to be easy to follow for someone already familiar with ATL. There are some minor typos and tweaks to be done for the post-proceedings version, some of which I will list below.

• How did you compare the results of your solution against those of the reference solution? I did not find unit tests for your transformation. You may want to adapt the unit tests of my Epsilon solution, which uses the java-diff-utils library to do a line-based diff between the obtained and expected model (while ignoring lines likely to have changed due to UUIDs).
• Can you provide some quick mentions of the sort of strategies that an ATOL user would employ to reduce the memory footprint? Is this something that can be done by tweaking the ATL rules, or is it more about optimising the data structures used inside ATOL/AOF?
• As mentioned above, I think it'd be nicer to use R-based figures like the ones I produced from the Epsilon diagrams.R script (which you should be able to run as just ./diagrams.R if you have Rscript installed).
• In Section 5.1.2, why does ATOL return a target tuple and not just the first element? As a reader, it would be good to know why that it was decided to diverge in that way in ATOL.
• As a reader of the paper, I think it would be good to have links to the current publicly available versions of ATOL/AOF, in case I wanted to try out the tools myself.
• ATOL is actually faster than ATL even in this batch scenario: do you have any ideas why? Is it being incremental for parts of this batch scenario, or is it because it is directly compiled into Java code rather than going through the EMFTVM?

Minor tweaks:

• "adding typing hints where ATOL fails" (missing "h")
• Page 4: "When this happens, the old rule"
• Page 7: "replaced by unique lazy rules without trouble" (troubles -> trouble). I'd say "without problems" or "without trouble". Google Ngrams seems to agree: https://books.google.com/ngrams/graph?content=without+trouble%2C+without+troubles&year_start=1800&year_end=2019&corpus=en-2019&smoothing=3
• Page 8: there is a line that exceeds the right margin by a bit ("with ATOL, it is mandatory ...").
• Page 8: What is a "custom join operation"?
• To make the paper more self-contained, it may be good to introduce readers who may not be familiar with ATL to lazy rules and how they differ from unique lazy rules.