Create list of MD pages automatically

[snomos]

Presently all Markdown pages generated from doccomments are listed in the Makefile.am/MD_PAGES variable. This is error prone since the gawk scriopt will error out when nothing was found.

I suggest we scan all dirs & potential source files for doccomments, and build the list of files to include dynamically. This is less error prone/more robust, and makes for less maintenance burden.

grep -rl '!![=≈]? ' src/* tools/*

should be enough.

This will also make it easier for us when we reorganise the src/ dir, as we don't need to manually change all the lists of documents to be extracted from.

[flammie]

I wonder if the error in awk was only added because forrest did not like empty files and can be removed too?

But in any case grepping etc. is good... another thing on todo list is that the generated doccomments -> markdown files should go back to gitignore probabl, we don't need commits of them any more.

[snomos]

I think you are right about the error and Forrest.

And I agree wrt gitignore, but I was thinking that we should wait with adding them to gitignore until all languages are converted. Now that. I spell it out, I see that that thought does not hold - it is better to add to gitignore right away. Will do.

[snomos]

This grep command gives a reasonable list:

grep -rl --exclude 'Makefile*' --exclude '*.pmscript' --exclude '*.xfscript' '!! ' ../src/* ../tools/*

At some point we should probably also add support for xfscripts, regexes and pmscripts. But for now this should be a good start.

[flammie]

3f9a84084c6b0fc02663d16ca31bb064f4682e80 should do it but it also has potential to break things

[Trondtr]

I seem to have been trapped in the middle, I get (here for sje, but for all lgs):

Making all in docs make[2]: No rule to make target `adverbs.lexc-stems.md'. Stop. make[2]: No rule to make target conjunctions.lexc-stems.md'. Stop. ... make[2]: *** No rule to make targetadpositions.lexc-stems.md'. Stop. make[2]: `root-morphology.md' is up to date.

So what to do??

[flammie]

the github action works now so I think it might be a problem in the mac sed here:

        x=`basename $$f | sed -e 's/\.\(lexc\|twolc\|cg3\)$$//'` ; \

on linux it works like so:

basename src/fst/stems/adverbs.lexc | sed -e 's/\.\(lexc\|twolc\|cg3\)$//'
adverbs

it seems that fails here.

[Trondtr]

gsed pro sed? And yes, both the generated .md and the generate-markdown... files sould be ignored, already now, as a matter of fact.

[flammie]

Ah yes, we need to include it in the configure checks then, I'll write that in the next template too, can you try if configure works in template-lang-und on Mac before merger?

[flammie]

The gsed is now part of the checks in all langs but also the filename handling is solely based on abusing basename and dirname... the makefile rules will receive similar handling eventually.