Add sample files (NSD + code import) in source tree

[GitMensch]

The main reason for this FR is to be able to check changed code against these files, but it would be to ship these in the "binary distribution", too (not sure how to do this best), to allow users to start directly with a sample.

@codemanyak wrote

What exactly do you mean with "include them in-source"?

@GitMensch wrote

I mean: create a directory "samples\NSD" and place the files in there. You may want to add "samples\Pascal", too (for adding files that are known to be importable without errors).

Having them in the git repository and in the release source tarball, maybe even in the jar allows people a simple first test, provides kind of a documentation and gives all the developers something to test their changes against.

@codemanyak wrote

I see. This would even allow to run automatic integration tests - when we'll eventually get there to set up a CI environment...

Moved from #357, NSD examples in https://github.com/fesch/Structorizer.Desktop/issues/357#issuecomment-284074857 mainly need to have the new attributes for author/license in (or otherwise be put into an AUTHORS file) and could be added.

The same would be nice for Pascal / C code that is known to create a usable NSD with the import feature.

[codemanyak]

I'm not quite convinced of the shipping of sample files with the "binary" distribution (tends to blow up the download with more or less redundant stuff). I think for the end users the sample files should be available on the home page for download, ideally linked from the User Guide. EDIT: At least the user should be able to decide whether (s)he wants to get the samples with the package. But to offer different download packages would only be a more complicated version of the opportunity to fetch the samples from the home page. A clear advantage of delivering the samples with the binary, however, would of course be that chances are better to ensure the consistency with the Structorizer version. It would obviously not make sense to pack the samples into the jar file (this should even explicitly be prevented). Instead they would have to appear in a "samples" sub-directory tree.

[GitMensch]

@codemanyak I suggest providing the user with the samples via the web-page which can just link to the repository (directly into wherever the sample files are) with a nice README that provides something a "common" user can read+understand.

Adding the samples and expected results is in any case a useful thing (actually should seen as pre-requisite) for automated testing brought up in the past more than once, last time in #357.

Should we rename this issue to cover the "testing options" or do you want to handle this separated?

[codemanyak]

@GitMensch

Adding the samples and expected results is in any case a useful thing (actually should seen as pre-requisite) for automated testing brought up in the past more than once, last time in #357.

Quite correct, no doubt about it. And indeed I started several approaches (locally first, since I didn't want to pollute the public repository before I have developed some consistent concept). But then discarded them each time, because none of them looked satisfactiry to me after some weeks of intermission. The trouble is that there are some conflicts of interests in the game. On the one hand, the sample diagrams should make sense. They should serve as good examples how to work with Structorizer, how to design a neet algorithm, how to decompose a complex problem. Hence, arrangements should be among them. On the other hand, they shouldn't be the solutions of popular teaching tasks. Then they should support all important testing cases in a condensed way. Test cases may not only reflect export and import issues (though these are most easily tested as the GUI can be avoided). To keep them as orthogonal as possible some very reduced, twisted-looking nonsense diagrams or source code files may be way more sensible than complex meaningful ones. Next challenge is how to categorize non-trivial samples? What folder structure will be helpful? Where to place related ini file samples etc.? This is usually the point where I'm running out of time and patience.

[GitMensch]

I see. I highly suggest to for now add least add the sample[s] that are "referenced" in the manual (by images, if I remember correctly), the exported result for different languages along with their re-imported version. Possible folder structure: samples\$SFILE.nsd tests\samples\$SFILE\export\$LANG\$SFILE.extension tests\samples\$SFILE\re-import\$LANG.nsd

Even if there's only one $FILE (sample) this would already cover much more than we currently have (especially if the parts under "tests" are already generated with a unit test allowing you to check via git if/where changes are when you change the import/export code.

Adding more tests / samples later is a second step but possible with the suggested approach.

[GitMensch]

I'm still looking to be able to run any automated tests and also to have an option to open samples directly after "install". @codemanyak any update on this or a possible plan to do so? If those are up-and running I'd like to finish a CI/CD option to have Structorizer build and test (until JDK8 support is dropped (any plans for this btw?) with both JDK8 + 11) on every commit.

[codemanyak]

As you may have realized, I only happened to add some arrz and related code export samples. In this context I cared for deterministic subroutine order on code generation. I have not started to write test cases so far, though the existing samples already helped a lot in systematic manual testing after relevant code changes.

[GitMensch]

So https://github.com/fesch/Structorizer.Desktop/tree/master/samples may not include all the samples you'd consider useful but a start and the export of all samples.

The jUnit tests would now just call the export function and then have the files compared, correct? If you provide a code sample (a main) how to do the export programatically (and also get the list of exporters) I may spend the time to create a "real" jUnit-Test (parameterized with all exporters) to create the files. Deal?

[codemanyak]

@GitMensch Deal, thank you very much. The code export test would indeed basically mean to start the export with defined preferences in batch mode and textually to compare the results. As some generic identifiers will be created for some kinds of elements, a pattern matching will be necessary to avoid false fails, though. I might provide some preference files for the export (at least for the BASIC export, two different preference sets are needed).

[codemanyak]

@GitMensch With the new version 3.30-06 there are two export setting files now in folder samples/export. They differ just in the "line number" setting for BASIC export: export_basic0.txt produces vintage BASIC, export_basic1.ini produces more contemporary BASIC code. The preferences for the other export languages are identical. The batch export result could then be compared with the respective export sample in the subfolders of samples/export. One of the difficulties for Unit testing is the occurrence of generic identifiers with hash codes in the name as these will always differ between export jobs (unless the diagram stays loaded in a Structorizer instance, which is not feasible with batch export). Another difficulty easier to handle is the occurrence of a "= = = = 8< = = = = ..." line between the subroutines and the main program on batch export, which isn't there in the export samples since these used to be produced from an interactive session. So what a Java main could or should basically do is just executing the batch export, i.e. a command line like this (Windows variant, appropriate paths provided):

Structorizer.bat -x C -o D:\tests\ELIZA_2.3.c -s D:\workspace\codemanayk\Structorizer\samples\export\export_basic0.ini D:\workspace\codemanyak\Structorizer\samples\arrz\ELIZA_2.3.arrz

Or on Linux, respectively:

Structorizer.sh -x C -o ~/tests/ELIZA_2.3.c -s ~/Structorizer/samples/export/export_basic0.ini ~/Structorizer/samples/arrz/ELIZA_2.3.arrz

How to execute a shell command in Java is quite straightforward and explained here: https://alvinalexander.com/java/edu/pj/pj010016