Efficiency

[Claudius-Appel]

Superthread for various performance-related issues, mostly as a sketch pad prior to making exclusive issues

See: https://github.com/Claudius-Appel/duflor/labels/efficiency

[Claudius-Appel]

Translate hsv_boundary check to Cpp via RCpp package and evaluate performance on n=40: What/how much do I gain in doing so?

Edit: Conversion was successful, still must be committed. Code is simple to read, but must be documented still. Both internally and for roxygen.

Double-check that the code is actually doing what it is supposed to.

fixed in ecef350329d1ca337970a85066e454643fdffaf0

[Claudius-Appel]

optimise load_image(): it's the biggest bottle-neck.

However, also the one hardest to optimise, because I am sure as hell not going to create my own parser.

So I would need to find a C++-based loader that outputs in the format I expect.

Consider:

• cImg-documentation example

[Claudius-Appel]

Consider:

Benchmark efficiency of computing adjacencies:

• Is it viable to port it over to Cpp?
• Can we actually use it to determine what range of pixels is to be considered valid for a given spectrum? Check the old code to see how python.CV handles the detection of random background hits (I don't think they are actually cropped out of the pixel-count at all, but I have to check).

[Claudius-Appel]

Consider: image-by-image analysis:

Given that RAM is a limited resource I should consider whether it is better / more performance to do the following:

• analyse image by image by which I mean that you go loop through all images and on each loop you do the entire analysis work-flow for this one image and at the end you only retain information, so you only write to a data frame this image returned x pixels for green area, x pixels for yellow, x pixels for pink and maybe you retain the maybe you can get away with retaining the coordinates of them that you have to consider whether or not that would still pile onto too much RAM usage given that you might have to do another images we don't know that I'll have to benchmark but I think that this flow is much better than running first orloading all images to matrices then for each matrix analyzing it and then populating another matrix with it's respective results per run I think that if you do the latter for second method you very quickly run out of RAM and we'll have to start working from finally which is apparently slow because well unless unless you're right working on Linux workstation where you can do 20 gigabytes of RAM you simply can't load all those images 500 megabytes of raw sale size so no not by 200 that's like a thousand one gigaby is 100 gigabytes of RAM you you're not getting 100 gigabytes of RAM on a Windows system

See > Claudius-Appel/duflor#17

[Claudius-Appel]

In the cpp- pixel count function setup, currently:

• for each hsv-set, loop over the entire range of pixels and return all hits.
• this means that all x*y pixels are checked for all N hsv ranges.

Instead:

• would it be faster to iterate over all N pixels and for each pixel check if it fulfills conditions of each range. In the case of a hit, it would be attributed to that ranges xy-records.
• in theory, this would mean iterating over all pixels only 1 time, instead of N times.

Prereqs:

• confirm that the cpp solution does not record xy-coords
  • however if it does, maybe I can record x/y pairs in a dataframe, and name the x/y with a suffix of the range it belongs to.

See > https://github.com/Claudius-Appel/duflor/issues/16

[Claudius-Appel]

Before doing more shit, wait for performance-tests on BexBT's end.

Then, consider carefully how much more efficient you actually want this to be, that is how slow is acceptably slow for us? There are more pressing matters, after all (#15)

[Claudius-Appel]

Determine performance gain of loading via RbioFormats:read.image(), see Claudius-Appel/duflor#19