memory bug prevents setup of robust pipelines

[tstoeger]

(without minutes / hours of completely avoidable human waiting time)

While trying to visually check correctness of some simple pipelines on my computer, I repeatedly ran into unexpected memory bottlenecks, which were absent from original CP.

Since the data, that should be passed on in handles, is small (< 100MB), and no excess memory is required on brutus (which does not display figures) and a few months ago similar jobs of CPP did not require suspiciously high memory on Brutus, I suspect that that the excess memory requirement is due to the displayed figures.

Supporting this hypothesis: if I clear the figures after the 2nd cycle, the amount of occupied memory drops by half.

What it means in practice: If one sets up pipelines, one will soon have to wait several minutes due to an avoidable bug in the CPP (which is absent from CP).

Moreover this RAM problem gets worse and worse with the number of sites / cycles, that one wants to test. Thus one will sooner or later reach the point where CPP because incredibly slow during setup of pipelines.

In practice, one might be able to avoid this problem by switching to a high-RAM machine (such as Helium...) , since the point where the memory bottleneck (which would not be there if memory is properly cleaned) is reached, will be later (though this is no solution of the actual problem)

see attached screenshot for documentation.

[tstoeger]

Closing some GitHub issues of mine which have been idle for around two years. Underlying issues might still be present.