kinchungwong
commented
6 years ago While tile-based and strip-based execution engine is a good addition to OpenCV, they should be seen as complements (in addition) to row-based execution engine. Based on my experience, I take objection to some of the wording and tone, and also to their underlying premise.
I do agree that FilterEngine needs a redesign.
I do not intend to make claims without showing any code to back it up. However, I need to secure my legal release from my former employer, which may take time.
Here is my claim:
- Certain image algorithms are most efficient with row-based execution engine and corresponding memory management strategy. Specifically, any algorithms whose coordinate transformation matrix is
[ [a, 0], [0, b] ](for any a, b, including but not necessarily 1.0) (in other words, "non-rotating", or "horizontal and vertical axis preserving". My math terminology may be wrong, sorry about that), will benefit from a row-based strategy. - A correctly-implemented row-based engine can work with strip-based data trivially.
- Strip can be considered a special case of Tile, where the number of "horizontal partitions" is one. True tiling refers to both horizontal and vertical partitioning of pixel data.
- Image algorithms which are "rotating", including rotation (where angle is not 0 degree modulo 360, and specifically including 90, 80, 270, and all values in between) and non-affine transformations (perspective, log-polar, etc), will benefit from tile-based execution.
- Generalized
remapbenefit from tile-based execution if and only if theremapis a diffeomorphism. Same idea as cache locality. Otherwise, any non-flat memory layout will harm its performance in these situations. - For types of algorithms which can be implemented with both tile-based and row-based, tile-based typically result in more data copying than row-based.
In simple words, tile-based is beneficial to a few algorithms, but we should continue to use row-based execution for all other algorithms.
Tile-based execution may be necessary to inter-operate with GPU, but GPU usage already implies memory-copying. GPU may also impose maximum texture size.
Here is a teaser: imagine having this class, which is a "pool" of "row buffers (aligned)", and imagine algorithms which can be written to exploit such usefulness:
// for simplicity, here we ignore strict-OOP, visibility, etc.
// We also skip class template, which is trivial to add later.
// Reference counting for each row (either shared_ptr or on top of that) can also be added later.
// "RowBufferU8" and aligned_vector is only for illustration.
// A correct implementation would need unique_ptr for a row, along with aligned allocator/deallocator.
// Use of int, size_t not taken cared of carefully here. A real implementation will try to get those correct.
struct RowBufferPool
{
using RowBufferU8 = std::aligned_vector<uchar, 16uL>;
size_t elementCount;
std::unordered_map<int, RowBufferU8> m_active;
std::vector<RowBufferU8> m_unused;
public:
RowBufferU8& add_or_get(int key);
RowBufferU8* get(int key) const;
void remove(int key);
private:
void ensure_spare(ptrdiff_t count);
void add_spare();
};In my experience, this can greatly simplify row-based image processing as well as greatly reduce unnecessary data copying (memory transfer). SIMD code can also be greatly simplified.
If an algorithm involves change of horizontal number of pixels, multiple instances of row buffer pools can be used; one pool for each pixel width (depending on the algorithm). For example, if an algorithm involves doubling of pixel width, then one pool can have width W, another pool can have width 2*W.
If an algorithm involves mixed numerical types, multiple instances of row buffer pools can be used, one for each numeric type.
FantasqueX
the feature request about evolution proposal OE-17