boris-kz / CogAlg

This project is a Computer Vision implementation of general hierarchical pattern discovery principles introduced in README
http://www.cognitivealgorithm.info
MIT License
90 stars 42 forks source link

Why a 4-channel image in misc.py/draw_blobs is created? Performance suggestions #19

Closed Twenkid closed 3 years ago

Twenkid commented 5 years ago

(That's a copy of: https://github.com/boris-kz/CogAlg/commit/ee56f9e099dd8db3a265de5bdeae369941edeac6#r31985883 because I don't know whether notifications are sent for such comments.)

Why are you creating a 4-channel image in misc.py, would you explain?

frame_img = np.array([[[127] 4] X] * Y)

(I see 127 is just a gray default value).

Then normal 3-channel BGR/RGB values are stored: frame_img[y, x, :3] = [255, 255, 255] if P[0] else [0, 0, 0]

Do you plan to add transparency/another data channel in the future?

A performance question as well:

Do you justify the usage of BMP, instead of a low-level compression PNG for example - the images are with low complexity, IMWRITE_PNG_STRATEGY_RLE = 3 (run-lenght encoding) could be used and it would be very fast.

Since there would be a lot of saves, another possible optimization would be buffering the images and then recording them all at once (or at least in batches), in order to avoid or reduce the random disk-access interruptions during the run. Well, of course unless you want to see immediately the images while they are being exported.

SSDs are fast, I don't know if it'd make a lot of difference, but for this or for other cases it would be something like that:

buffered = {}
ret, enc = cv2.imencode('.png', seg_img)
if ret:
  label = '/blob%dseg%d.png' % (blob_idx, seg_idx)
  buffered[path] = enc
(...)
for path in buffered:
  open(path, "wb").write(buffered[path])

I also suggest using a RAM-disk, e.g. Imdisk: https://sourceforge.net/projects/imdisk-toolkit/

...

boris-kz commented 5 years ago

That's Khanh's function, you can contact him via his repository: https://github.com/khanh93vn/CogAlg But performance here is not important, reconstruction will always be a lot faster than core processing..

On Fri, Jan 18, 2019 at 5:04 AM Todor Arnaudov notifications@github.com wrote:

(That's a copy of: ee56f9e#r31985883 https://github.com/boris-kz/CogAlg/commit/ee56f9e099dd8db3a265de5bdeae369941edeac6#r31985883 because I don't know are notifications sent for such comments.)

Why are you creating a 4-channel image in misc.py, would you explain?

frame_img = np.array([[[127] 4] X] * Y)

(I see 127 is just a gray default value).

Then normal 3-channel BGR/RGB values are stored: frame_img[y, x, :3] = [255, 255, 255] if P[0] else [0, 0, 0]

Do you plan to add transparency/another data channel in the future?

A performance question as well:

Do you justify the usage of BMP, instead of a low-level compression PNG for example - the images are with low complexity, IMWRITE_PNG_STRATEGY_RLE = 3 (run-lenght encoding) could be used and it would be very fast.

Since there would be a lot of saves, another possible optimization would be buffering the images and then recording them all at once (or at least in batches), in order to avoid or reduce the random disk-access interruptions during the run. Well, of course unless you want to see immediately the images while they are being exported.

SSDs are fast, I don't know if it'd make a lot of difference, but for this or for other cases it would be something like that:

buffered = {} ret, enc = cv2.imencode('.png', seg_img) if ret: label = '/blob%dseg%d.png' % (blob_idx, seg_idx) buffered[path] = enc (...) for path in buffered: open(path, "wb").write(buffered[path])

I also suggest using a RAM-disk, e.g. Imdisk: https://sourceforge.net/projects/imdisk-toolkit/

...

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/issues/19, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGUL87wwTxj7CUKBmEv4ETDP-E_xgks5vEZwsgaJpZM4aHl-g .

Twenkid commented 5 years ago

OK, I'll comment there. He gave me a detailed answer in this repository for now.

But performance here is not important, reconstruction will always be a lot faster than core processing..

I understand. Yes, these optimizations are not essential, it won't make a lot of difference, especially for single tests and low resolution, Khan mentioned 64x64 for now.