Closed nkrabben closed 4 years ago
thanks Nick - this is something that's cropped up before (https://github.com/richardlehane/siegfried/issues/94). My previous fixes have been stop-gaps but I've been tinkering with a more fundamental fix & it is good to have this as a prompt to work on it.
Basically this issue occurs for "noisy" signatures with multiple segments that generate lots of partial matches. fmt/134 is probably the worst offender. I'm currently too exhaustive in following up these matches - I'd like to make this bit of the code "lazier", it has just been hard to do so without breaking everything.
Hi @nkrabben I'm working on this issue now - if you have a sample you can share (either here or via email to keep private) would be great help
this is partly fixed (the verbose basis bit) in v1.7.9. But I have more work to do on the speed side of this issue, so re-opening
second part of this bug (slowdown for MP3 with lots matches) is now fixed on develop branch and will be in next release, see this issue: https://github.com/richardlehane/siegfried/issues/128
Running siegfried across a collection, it's really slowing down on a group of mp3's. https://www.nationalarchives.gov.uk/pronom/fmt/134
The basis field for these matches can be up to 3,000,000 characters long with repetitions of similar data such as [795 105] 9,650 and [807 105] 13,068. I'm not sure what's causing this. If useful, I can probably provide a copy of the files causing this bug in the new year.