search

ilia3101 / MLV-App

All in one MLV processing app.
https://mlv.app/
GNU General Public License v3.0
274 stars 30 forks source link

Add element to ssshist #267

Open esoha-nvidia opened 1 year ago

esoha-nvidia commented 1 year ago

sssshist stores a histogram of how often each ssss value is used. There are 16 of them, from 1-16. Though 0 is unused, it's more convenient to index the elements by ssss so, rather than subtract one before indexing, we will make the array larger by one element.

This fixes #266

cgohlke commented 1 year ago

I think all references to sssshist (only three lines) can be safely be removed from the source code.

esoha-nvidia commented 1 year ago

It was to make a histogram, presumably for the purpose of analyzing statistics afterwards. Those statistics could help build a new Huffman tree for re-encoding data. But it's unused, yeah. As an alternative to deleting, it would also be possible to hide it behind a compile-time macro.

This would potentially make the code run faster. On the other hand, this code already looks quite inefficient so it probably doesn't matter too much! 😆

esoha-nvidia commented 1 year ago

https://github.com/ilia3101/MLV-App/blob/ef8a955dc0f742e1890b83a355e8bd3b423cdad0/src/mlv/liblj92/lj92.c#L227 The lookup table is size 218 bytes! This is ridiculous! The cache misses are probably going to wreck the performance!

masc4ii commented 1 year ago

Do you see any chance to optimized that code? I also identified this lj92 compression to be slow, while using just one CPU core... For full video processing from RAW to viewer, we loose many frames per second when using lossless decoding compared to uncompressed footage ... and the image processing (same for both) is very complex.

BTW: great, you guys work with this code! Thank you!

esoha-nvidia commented 1 year ago

@masc4ii It can be optimized but the code is written quite poorly so I'd be wary of even trying it. The variable names are obtuse and everything is in c and not c++. The code is just archaic so I'd be too afraid to fix it. It might be better to somewhat start over.

There is also the problem that my ideas for performance improvement are belonging to my employer so I cannot share them.

Are you able to run a performance test and tell me how many bytes-per-second output you have for lj92 as compared to, say, gunzip? I would expect that lj92 files should decompress about 4 times faster than gzip files. Certainly not slower because both are Huffman entropy encoding but gzip is much more complex than lj92.