Conduct extensive tests of the latest release (v0.7)

[proycon]

• [ ] Verify once more that all desired functionality that was implemented in @martinreynaert's prototype at https://github.com/martinreynaert/TICCL/blob/master/TICCLops.PICCL.pl (commit state 62a398f) is present. (This is what my entire Nextflow rewrite was based on after all. Some exceptions and all Nederlab work aside, I didn't implement things not in there that were not explicitly requested)

You might want to repeat the following items for multiple languages, and with/without post-processing options such as tokenisation (ucto) and linguistic enrichment (Frog). You may also want to test both from the interface as well as from the command line:

• [ ] Verify OCR & TICCL pipeline from PDF containing scanned images
• [ ] Verify OCR & TICCL pipeline of individual images with PDF reassembly option enabled
• [ ] Verify OCR pipeline without TICCL
• [ ] Verify TICCL pipeline on plaintext (without OCR)
• [ ] Verify TICCL pipeline on PDF that contains text (without OCR)

In testing all of the above, consider a) whether it runs (i.e. no crashes) and b) whether results are actually sensible/correct.

For most of these scenarios, an integration test for the backend is already available in test.sh, and those tests are only limited to aspect (a). There are no tests that invoke the front-end and wrapper. Please suggest (small) data and improvements for further automated tests where applicable.

Regarding the webservice-end of things:

• [ ] Verify whether the provided parameters and profiles (input templates and outpute templates) cover all you want to offer. (taking into account the fact that we currently merely offer a generic CLAM interface rather than a tailored front-end for end-users).
• [ ] Verify whether the desired intermediate output is published (this goes for any of the scenarios)

Regarding software dependencies:

• [ ] Verify whether the software versions offered by the latest debian/ubuntu/CentOS suffice and whether PICCL is capable with sufficiently handling the inevitable version discrepancies that occur between distributions. Are some versions must-haves and are some versions show-stoppers?

Please always report problems or feature requests in separate new issues (not in this thread), but do confirm things that are working properly here (just tick the boxes) so we have verification. Ideally, wherever they can't be automated yet, these tests should be repeated when a major release update occurs.

[JessedeDoes]

Can we test the latest version in the Nijmegen web service?

[proycon]

It's still one release behind, but I can update it later today/tomorrow.

[JessedeDoes]

Thanks!

[proycon]

The latest release has been deployed in Nijmegen.

[JessedeDoes]

I still get errors on plain text with ticcl disabled:

error.log

Is this related to the fact that uploaded plain text is listed as "PDFFormat" in the uploaded documents list?

[proycon]

Hmm, that is quite wrong indeed (but probably not the cause of the error). I'll look into it and run a test myself.

[proycon]

I finally fixed this persistent problem (meaning I actually tested it now as things kept popping up). It's in the latest master branch (so not released/deployed yet). I'll do some extra tests prior to release to see if I didn't break anything obvious in the process. Still, elaborate testing by @martinreynaert as proposed in this issue remains very relevant!

The wrapper script has become somewhat convoluted as what we ideally need but don't have is automatic workflow discovery, considering the complexity grows. I did some cleanup and renaming to at least make things clearer.

A problem here is also that the wrapper script currently is not covered by the CI tests (as those only tests the nextflow workflows directly rather than the full integration and webservice overhead).

[proycon]

v0.7.6 is released and deployed now.

[proycon]

This is not gonna happen anymore I guess, v0.9.0 has been released now, expiring this issue...