Fix issues 811 812 819 823

[johann-petrak]

This should fix the following issues:

• 811 Implement holdout evaluation in addition to cross validation
• 812 xval example code is incorrect
• 819 Do devset stratification
• 823 Devset split does not work with max_processes=1

The following notes are important:

• dev set stratification is done if the processor has the attribute dev_stratification and it has a non-None value. Otherwise, the behaviour is exactly as before, for backwards compatibility and also to make sure that this does not impact any other situations with different tasks, multihead etc.
• 823 is fixed ONLY if dev_stratification is specified and not None: if it is False, no stratification is done, but the split is still done properly independently of the chunk size
• The devset split in the data silo for xval is now done as configured in the processor, however only for non-nested xval and non-qa task. The other code has not been implement by myself and I am not able to test the qa part and I am not even sure I understand why the nested xval is even there. This should not be a problem as the splits ARE still done, but depending on the xval stratification setting, not the parameter specified in the processor (for qa, the processor does not even have that parameter atm).
• I did test this extensively and added tests to make absolutely sure that no unwanted overlapping of examples occurs and I also timed my test runs to see if there is any performance impact from the new ConcatTensorDataset class I implemented.
• I will try to create a PR for the ConcatTensorDataset to the Pytorch people, if they accept it, the class could get removed from FARM eventually (and they may even improve it, who knows, there is potential for optimization).
• I have renamed the stratified parameter of the DatSiloForCrossVal make factory function to stratification to match the parameter name dev_stratification in TextClassificationProcesser, where that name was chosen because the parameter is not just boolean but can also be None (for the old way of doing things) or a callable for one's own way to perform the split.

I have not tested this with more complex multihead models - yet: i will use this myself with multihead models, so should there be issues with that I will correct them then.

[Timoeller]

Thanks for combining the contributions into one very dense PR.

I think the addition of the ConcatTensorDataset might have broken something for other tasks, see the failing CI. Could you look into it? Maybe separate the additional ConcatTensorDataset from other tasks as a workaround?

Disclaimer: I will only be available for reviewing further adjustments from Monday onwards again.

[johann-petrak]

Sorry this was really caused by a stupid typo I introduced when I tried to avoid a warning message being too long in a single line.

The ConcatTensorDataset has been implemented as a subclass of ConcatDataset and literally works exactly the same unless one requests a slice or list of indices instead of a single index - which can never happen in any code not added by me here, because that would have been impossible so far.

I did run the tests here of which one fails: test_ner_amp.py::test_ner_amp but I suspect due to completely different/unrelated reasons. The tests in github actions seem to pass now: https://github.com/deepset-ai/FARM/runs/3013430217?check_suite_focus=true

[Timoeller]

btw as slight improvement for the next PR please mention "fixes issue num" explicitly in the PR description, then the corresponding issues get closed automatically.