Add 'test latest changes script'

[svx]

Add a test script to papyrus [Makefile] which runs spell and link check on latest changes, got committed to the docs.

The idea is to run check not on the 'whole docs' but on the parts got changed or added.

This will:

• speed up testing
• push quality, because now we actually run tests more often, meaning test will fail more often, meaning we will fix stuff.

  To figure out:

• what about robots and screenshots, still run them all the time ?

  Nice:

By doing this over time, the docs will improve a lot !

[svx]

Possible Solution, really dirty WIP

svx@beaker:~/.../test/papyrus (5.0)$ git show HEAD --name-only --pretty="format:"
CONTRIBUTORS.rst
README.rst
docs/CONTRIBUTORS.txt
docs/index.rst

Gives us the list of changed files in the latest commit, now we have to run checks against these files, and done !

[svx]

another option could be:

 git diff HEAD~1 --name-only

[svx]

possible solution:

Use aspell

create a script with the following scenario [on ci or local]:

• check if we have all dependencies [aspell]
• rm old aspell.plone.pws
• convert spelling_wordlist.txt to aspell.plone.pws by doing this we always have the latest, which is great.

sed '1 i personal_ws-1.1 en 0' spelling_wordlist.txt > .aspell.plone.pws

• run spell check: cat askforhelp.rst | aspell list --personal={pwd}.aspell.plone.pws | sort -u

If we get a output stop with error message and report output, otherwise it was a success

[svx]

Another question would be do we still want to do that in papyrus, personally I think we remove that out of papyrus, papyrus is for building the docs and testing them 'as a whole' before deploy.

So I do not want to fill the repo with extra tests, I also do not want to put them into the documentation repo, because it is for docs only.

Since they are tests, I say we should put them into mr.docs, since mr.docs is invented to run tests on our docs locally and/or on CI setups.

[svx]

There we go, a proof of concept, the script needs some more checks and beauty touches, like error and log reporting, but it works so far.

https://youtu.be/dguuwzCSjaM

What is does:

It takes the last commit to a repository and will check all .rst files in /docs for typos, other files like .txt are ignored.

Todo:

• add logs
• add better error reporting, split it into list of file names, if we have more than one file that we check
• ...

[svx]

More progress:

I found a lovely solution how to do that with sphinx, meaning we can skip the 'whole dirty' aspell workaround, plus we have now also link-checks out of the box.

What we have to solve now, is the behaviour of the test, if we commit more than one file. For example we have a commit including changes in:

• index.rst
• about/styleguide

We have to pass every single file name now to the test script.

The test will be only 'green' if the files are without typos and without broken links

[svx]

OK, original idea is not working because of all the different branches, we need a script which runs check against certain branches, we can not do:

git diff --name-only HEAD master

Because we are using different branches for different Plone versions.

I would like to change that for papyrus anyway, because IMHO for papyrus at least for my idea for version 5.1 we do not need that anymore.

Possible workaround till we are there:

• we add in each branch a directory for tests
• add tests according to the branch in there

For me this still feels dirty, but this maybe a temp. solution till this IMHO horrible papyrus mess :) is solved !

Still as I already said I am not certain that papyrus is still the right place for running tests, not sure yet what would be a good solution. Point is we still need papyrus for robots, at least for the moment, but maybe we could add a .coafile to the docs repo and run tests on CI ?

@polyester any suggestions, opinions, comments ?

[svx]

Something like that should work:

changed_files="$(git diff HEAD~1 --name-only | grep ".*\.rst$")"