test data: Make data generation streamlined, standard and randomized

[sloorush]

Have a standardized module to make data generation standard across all the tests for the primitives.

Thought dump on this issue:

1. Some tests rely on a very specific way data is generated so that the result can be computed with simple mathematical patterns instead of relying on computation using other primitives.
2. Randomization of data is required to test for a larger, varied and discontinuous group of data but point 1 is again a problem.

   Excerpt from Peter's email: In general, I believe it is a good idea to randomise as much as possible about the input. For example, your test data for 32-bit integers all fall into a relatively small range (compared to the possible range of a 32-bit number). That means, there are a lot of 32-bit integers for we never test for. It sometimes happens that an edge case exists with certain special values (powers of 2 for example). My preference is to generate the data at random and print out the initial ⎕RL value, in case of a failure. That way we will, over time, cover a much larger input domain and become more confident that ∪ works for all 32-bit integers, and not just those you picked here. The same idea applies to the shape of the data in some cases.

3. Boundary value analysis is required for all datatypes.
4. More ways to generate non-simple arrays (such as nested arrays, or namespace references for example).
5. Make a library for data generation to remove the code duplication, inconsistency and over-complicated behaviour of data generation for all of the primitives separately.

[sloorush]

Information about a lot of data is present in: http://wiki.dyalog.bramley/index.php/Performance_Measurement/pqa#Expression_labels

I am planning on following the same pattern (if not the same code), for the tests for the masses of data

[sloorush]

One more excerpt from Peter's email

I really think we need to have more random data. We talked a bit about this at the internal meetings as well (and I think I have mentioned it a couple of times before when reviewing earlier tests). You do provide test cases for what is mostly hand-picked data, which covers all the non-nested data types. But since the data itself is hand-picked, that means there are certain inputs we will never test. Having the data itself be random, but still guaranteed to be of the correct datatype, will help us test many more cases over time. Perhaps after running the tests for a month we happen to run the test with some random input which does not produce the expected result (which also means there should be a way to reproduce the failed run by setting ⎕RL manually) For example, in your tests all the 4-byte integers that are tested are always in the ranges 100000-100100 and ¯100000-¯100100, which are only 200 numbers out of the 4 billion possible values. So, I think some time should be spent working on producing a set of functions to generate this random data, as I feel it is important, and it is something that will be useful in pretty much all the tests you write.

---

Something which isn't so important for divide but will be important when testing other of the scalar primitives such as + or ×: Looping over the combinations of datatypes in the input is not enough. The result type should also be considered for each combination of the input types. For example, adding two vectors of 2-byte integers could produce a Boolean vector, a 1-byte integer vector, a 2-byte integer vector, or a 4-byte integer vector. While I think we should test all these combinations (by choosing the data carefully, but still at random within the ranges that are needed), the 4-byte and 2-byte results are the most interesting. In the 4-byte integer vector result case, the addition "overflows" the input data type, and that is handled differently in the interpreter.

[sloorush]

Information dump: E & J are interesting characters to look for. 11×1, 1×11, 11×11 matrices are interesting