Unsolicited IO from library code

xandkar commented 4 months ago

I upgraded to 1.2.1 and it fixed this problem. Now I upgraded to 1.2.4 and the problem is back, albeit in different places, but there're still spurious print statements on errors.

These should either be log statements (ideally with tracing) or not be present at all, since this is a library being called by apps with different expectations about IO.

I'll be happy to do the work of converting them to tracing calls if you're OK with it.

mindeng commented 4 months ago

Thank you very much for your suggestion. Changing the print/eprint to tracing is a great suggestion. If you are willing, your PR is very welcome! Thanks!

mindeng commented 4 months ago

@xandkar Before modifying the code, make sure you have pulled down the latest main branch code to prevent conflicts. Because I made some bug fixes, as well as some small internal improvements, thank you!

mindeng commented 4 months ago

@xandkar Can you post these printouts? I want to see what went wrong. It would be better if you could attach the files where the parsing failed. Thank you!

xandkar commented 4 months ago

I often see stuff like this for videos:

box size of box 'mdat' is too big: 156013308
box size of box 'mdat' is too big: 107592213
box size of box '****' is too big: 440786851
box size of box '****' is too big: 440786851

I'll try to find/make something I don't mind sharing publicly :)

xandkar commented 2 months ago

@xandkar Can you post these printouts? I want to see what went wrong. It would be better if you could attach the files where the parsing failed. Thank you!

Hi @mindeng,

Here're a couple of video files which result in errors (one webm - made by Cheese, and one mov - made by an iPhone SE).

Error texts are in ${video_file_name}.txt

https://www.dropbox.com/scl/fo/jw7795q1jkliocw0b33sz/ADiANmDPAHvzCzm0rh1xyho?rlkey=213fy7z17soxugbhhse4xng6z&dl=0

mindeng commented 2 months ago

@xandkar Can you post these printouts? I want to see what went wrong. It would be better if you could attach the files where the parsing failed. Thank you!

Hi @mindeng,

Here're a couple of video files which result in errors (one webm - made by Cheese, and one mov - made by an iPhone SE).

Error texts are in ${video_file_name}.txt

https://www.dropbox.com/scl/fo/jw7795q1jkliocw0b33sz/ADiANmDPAHvzCzm0rh1xyho?rlkey=213fy7z17soxugbhhse4xng6z&dl=0

Got! I'll find time to look into it, thank you!

mindeng commented 2 months ago

@xandkar Can you post these printouts? I want to see what went wrong. It would be better if you could attach the files where the parsing failed. Thank you!

Hi @mindeng,

Here're a couple of video files which result in errors (one webm - made by Cheese, and one mov - made by an iPhone SE).

Error texts are in ${video_file_name}.txt

https://www.dropbox.com/scl/fo/jw7795q1jkliocw0b33sz/ADiANmDPAHvzCzm0rh1xyho?rlkey=213fy7z17soxugbhhse4xng6z&dl=0

For the mov file, the error has been fixed. The reason for the error is that I added a size limit on the box body when parsing box headers, which is not necessary, because the box body size is not matter when parsing/traveling box headers, so I removed this limitation, and the error is fixed. (v1.5.2 CHANGELOG.md)
- But when parsing the box body, it's indeed necessary to do that limitation, for the robustness of the program. Additionally, I also changed the size limit on the box body to a more reasonable value. (This is not relevant to this issue, just FYI in case you are interested).
You can check it quickly by running cargo run --example rexiftool <FILE>.
For another webm file, since the crate only supports mp4/mov video files at the moment, so the behavior is expected. I will find time to study this file format and strive to support it.

Thanks again for your feedback and suggestions, very valuable!

mindeng commented 2 months ago

@xandkar Can you post these printouts? I want to see what went wrong. It would be better if you could attach the files where the parsing failed. Thank you!

Hi @mindeng,

Here're a couple of video files which result in errors (one webm - made by Cheese, and one mov - made by an iPhone SE).

Error texts are in ${video_file_name}.txt

https://www.dropbox.com/scl/fo/jw7795q1jkliocw0b33sz/ADiANmDPAHvzCzm0rh1xyho?rlkey=213fy7z17soxugbhhse4xng6z&dl=0

Hi @xandkar !

webm is already supported! It also supports mkv, mka and other files.

However, this version has undergone major changes. I have rethought the API design, introduced new, more ergonomic APIs and usage methods, and also optimized performance (especially for batch file processing scenarios). Therefore, the release of this version may be slightly later.

If you have other file types that need support, please feel free to let me know, thank you!

xandkar commented 2 months ago

@xandkar Can you post these printouts? I want to see what went wrong. It would be better if you could attach the files where the parsing failed. Thank you!

Hi @mindeng,

Here're a couple of video files which result in errors (one webm - made by Cheese, and one mov - made by an iPhone SE).

Error texts are in ${video_file_name}.txt

https://www.dropbox.com/scl/fo/jw7795q1jkliocw0b33sz/ADiANmDPAHvzCzm0rh1xyho?rlkey=213fy7z17soxugbhhse4xng6z&dl=0

Hi @xandkar !

webm is already supported! It also supports mkv, mka and other files.

However, this version has undergone major changes. I have rethought the API

design, introduced new, more ergonomic APIs and usage methods, and also

optimized performance (especially for batch file processing scenarios).

Therefore, the release of this version may be slightly later.

If you have other file types that need support, please feel free to let me

know, thank you!

Awesome! Thank you for all your work and attentiveness, @mindeng!

mindeng commented 2 months ago

Hi @xandkar

Version 2.0.0 has been released! Thank you for your support!

mindeng / nom-exif

Unsolicited IO from library code #8