openpreserve / jpylyzer

JP2 (JPEG 2000 Part 1) validator and properties extractor. Jpylyzer was specifically created to check that a JP2 file really conforms to the format's specifications. Additionally jpylyzer is able to extract technical characteristics.
http://jpylyzer.openpreservation.org/
Other
69 stars 28 forks source link

feature request : abbreviate </plt> marker output for large images #185

Closed boxerab closed 2 years ago

boxerab commented 2 years ago

I am using jpylyzer to validate very large satellite images, which always have PLT markers. 95% of the output for these images is simply <plt/>.

Perhaps, if there are N plt length values, then <plt>N</plt> would achieve the same effect, but be much easier to parse for humans. Thanks!

bitsgalore commented 2 years ago

Good call. Actually the current (2.0) version of jpylyzer only reports empty plt elements, but this will change in the upcoming 2.1 release, which adds some basic parsing of plt markers (see also the draft documentation here). But this will only make your problem even worse!

Instead of reporting the number of plt length values (which would also create a conflict with the 2.1 implementation), another solution might be to add a command-line switch that suppresses the reporting of the plt elements altogether. Or perhaps an even more generic solution: a command-line switch that suppresses any element in a (user-defined) comma-separated list of element names (this might make sense for some other codestream-level markers as well). Would that make sense?

Thanks for the suggestion, and I'll add this to the list for jpylyzer 2.1!

boxerab commented 2 years ago

@bitsgalore thanks for the quick reply. The only other markers in the code stream that would have this issue would be SOP and EPH for start and end markers of packet header. PLM would also have this issue, but nobody seems to use it. So, not sure that a generic solution is needed, but maybe generic is easier to manage in the long run.

What I would like to see from PLT validation would be :

  1. are there are or aren't there PLT markers
  2. are they compliant with the standard
  3. are the lengths correct - this would involve parsing the packets as well as the markers, and making sure they're equal

So, basic validation i.e. point 1 would just require a yes/no output for PLT markers.

VirtualTim commented 2 years ago

Hey @boxerab, I added in some ptl marker reporting in this pr: #170. So as bitsgalore says, it's going to get "worse" in 2.1. I do agree that it would be good to have this behind a flag. Perhaps it should be added to --verbose?

boxerab commented 2 years ago

Hey @boxerab, I added in some ptl marker reporting in this pr: #170. So as bitsgalore says, it's going to get "worse" in 2.1. I do agree that it would be good to have this behind a flag. Perhaps it should be added to --verbose?

Nice, I will try it out. I think it's always useful to have PLT output, but perhaps it can be abbreviated somehow, and with --verbose the full output is then shown.

bitsgalore commented 2 years ago

After some more thinking I've come up with the following solution, which involves a new --packetmarkers option:

I created a separate packet-markers branch if you want to give this a try (this also has updated documentation, POD file and XSD schema). Let me know if you have any thoughts/further suggestions. I'd like to include this all in the endlessly-delayed 2.1 release.

VirtualTim commented 2 years ago

That sounds like a good approach to me.

bitsgalore commented 2 years ago

Perfect, changes now merged into 2.1 dev branch!

boxerab commented 2 years ago

Thanks!