itsbrex
commented
7 years ago FYI: I disabled all plugins and then ran beet config when i posted the above.
Closed itsbrex closed 4 years ago
itsbrex
commented
7 years ago FYI: I disabled all plugins and then ran beet config when i posted the above.
itsbrex
commented
7 years ago Beets worked on a number of the files, but then stopped all of a sudden. Running it again produces the above.
nayfield
commented
6 years ago can confirm this sort of behavior. In short, the chroma plugin is not resilient enough to handle failures of the underlying library.
Trying to get better automatic matching of a very large library: 662G Music 142G Music2
(du -sh information - Music is the original dir. I'm moving to Music2 on successful import and the stats above are after an import -q run)
Unfortunately chroma is useless because it crashes very early (before directories starting with "A"!):
chroma: chroma: fingerprinted b'/path/to/Music/100x Vive La France (2013) [V0]/CD4/13 - Plastic Bertrand - \xc3\x87a Plane Pour Moi [1977].mp3' chroma: matched recordings ['12c7ee0d-3dd0-47c7-ae7b-b7027fc979b6', '2c35e720-7261-4176-b375-324d908f0591', '32d3c9d0-51c8-4393-bbd3-1bb0adf53e72', '748ac88a-3bd1-4ae6-9288-46892aa7935b', '8b1d9ba3-e8c3-47f9-8cea-371ea54c3b64', '98c755b6-ad05-4ded-99a4-f6346878a1b8', 'e1299622-8676-405a-9d0a-4ae7f2528e2f', 'f3e76131-fc28-4f37-8427-a3c2f0e1aa12'] on releases ['e4d2c0ea-6504-4ad5-9fd6-9578716b3a13', 'd0a75f36-1814-4e13-91a0-83f30afd4b5e', '00a9d60b-9a07-44c7-8dcd-8ae1b2d73325', '3a81ba14-8d3b-4e2e-b9cc-1733057a8ac2', 'ad7a7522-3c7d-4f2d-914d-5d0f3a39cd6c', '271164a8-41e5-4657-baba-3eace3051236', '48363ec8-209d-441a-830d-397f87cf73c5', '0b3874de-09dc-45a5-97b1-528b50f3e59a', 'f0b647ce-8db2-4682-ac34-50d0cfd390fd', '760aa8e6-b741-460c-8fa3-172ae620491a', '5cd517bb-88a8-3547-a518-e992c250aa67', 'be87cae3-d288-400f-b89a-de33424cedaf', 'b782d865-7ff6-4f5d-be34-8c59a9eeedd9', '60332639-fcf8-43ec-be70-c65440c10a98', '43eed3bb-ce95-45e0-82e3-708bf3103b94', '4d5f833b-2230-3203-a1aa-8ff11955356e', '602ed0ba-cc41-4d4c-b1fb-3b8e913ca4da', '947bedb3-b687-4700-b432-d375abd3dcbf', '8e11bc9c-6075-4c78-b786-c45286750d0a', 'ade62cd7-21ee-4f9e-b58f-c24e5fde1bf7', '703a1d33-07b3-4cba-ac39-24e11bec4400', 'd4289f72-058a-40f7-be7e-8aaa0866d977', 'cd7f3d28-b6f1-4492-8944-3f608fa70f60', '5adb2798-652b-42fd-8ee5-62cb0f34c2f3', '8b4058ca-7d38-47b2-9917-f7d22b995e5b', '38bafba3-8e5a-447c-8bb9-fc604e39a1ef', 'db5db580-ee1b-40ed-bc94-ce924c65f325', '0503f3bd-8c89-47c7-a54e-fe4aa133febd', '1183d07b-7d5a-4118-a565-ec4f1fad70fc', 'f4d5b310-5a44-4e3b-b08d-d2b24b8ca282', '0fa64f9b-3b29-4236-9df4-419828ac2ced', '7018a2f9-5ca6-4be2-950a-12f3b48f1714', '9a12cec2-dbc7-4ac6-85dd-ab708db66cd5', '2a699b6d-e77f-498f-b080-3e272f3f6bef', '8d15314b-c507-4e6e-828a-e0c5bbc489f9', '65d1604a-421b-4e2d-8aa0-68051671bb77', '9ddad889-a5dd-401d-acc5-a6f87a002179', 'ab7b1729-d7af-46c3-953c-ffda16253580', '70fb6607-1bd3-4081-9568-6d1fb43928a2', '58c68197-b331-4f12-899e-1be7f54cb5c5', 'f3c862f7-571a-442a-904d-bfdaeaa4f8c9', 'b9b4bf06-14ba-460f-a76b-ecee60304b6f', 'f9d75637-d3df-3c33-bec9-1310ab70cc0f', '8a0e6a36-c8ac-3bed-9595-97eb0e80a17c', 'f0450e43-5b4f-405d-9d3b-0171ae89d343', '2febad15-0be8-323d-8979-e1e1beb21881', '0e6c4761-2a51-4a41-9460-a9d4eac4b5b4', 'bf653048-fa3a-40b4-92c9-2a9d99e11921', 'cb0d106b-1272-477d-98b2-11fcf63ba079', '666f8f25-c482-44eb-9bbe-e4912f658715', '2626b232-2e3d-4fc1-ac89-f2fb1c9d9078', '05f62c9f-5253-44da-a4e6-7f26a9942c8a', 'f98d1d69-e2b6-4363-ba8d-bdc0316c95f9', '5726b455-1e9d-41a1-bea1-e320114618c4', 'c4dc38cc-2d65-3749-abb4-e24bc595d78d', '9c682ad7-59ac-4fcf-a34a-70fb1369f9ea', 'cf253d0a-0e40-480d-8691-5fc6f0164248', '939e9681-3a71-4445-8b1d-bc232c8aec4e', '89b14dc9-5665-4257-8ad8-f9707c28f7ac', 'd918398b-beca-4947-88f5-a211035038e5', 'b6216058-f4a8-4b22-bc65-590062bd9852', 'a0306195-7b7b-44e9-9aa0-af453cbd60bc', '6ad2c54f-2a2d-4ad7-9939-71d67885ab31', 'b48d5c1d-1209-4c24-a915-7c756441a0e0', '29ff0a38-1cd8-4e68-816b-d9e4e873f1bf', '35ee6782-e6ce-4f49-965c-d5f040bd02ce', '19e7e92e-0bb3-4c1a-acff-25e4da01c736', '403c3858-80d2-4a6e-a43e-dd18c756af7f', 'c0b44d17-22f7-4a40-8f2e-7e3a54fb83d3', '61df8213-a2c6-45d7-9805-fbb143d83c8e', 'ec88a1d5-7368-4749-ac6b-794d7a59723f', 'd1333131-c8f1-472b-b9b8-33bac9c39072', '4df14f19-4c89-4103-8448-97dc1b3424d1', '8338b2d7-ef24-4b62-8acf-62891a7515bd', 'b0de0be5-f2ae-3025-89f2-fad3cb1f4542', '68aca243-e956-4684-b37d-227544ceacf6', 'edaf2153-1042-453b-a193-9f56601c2f57', '5deb2639-4203-47ad-82e2-a5a12d28ca70', 'bedf0a8f-5cac-4cb1-b2d5-29ed6182877e', 'dd9a94a9-3c37-4f16-a361-62b3cab84a5d', '9765126b-f93d-4f03-8f39-46a37379d311', '68b0c225-77a3-400d-bb36-0d4e8254ca26', 'ad7a7522-3c7d-4f2d-914d-5d0f3a39cd6c', 'dd9a94a9-3c37-4f16-a361-62b3cab84a5d', '742d283a-e9c6-4365-a417-15d046aee26c', '1303db29-9b3b-45d8-a043-60a979e5855c', 'c89ee069-26a7-4070-8bb4-bbeb94a494de'] Killed
jackwilsdon
commented
6 years ago Could you try it with Python >=3.4 and report back if you have the same issue? It sounds like Python <3.4 doesn't support 24 bit audio.
nayfield
commented
6 years ago Using python 3.6:
$ rpm -q python3 python3-3.6.5-1.fc28.x86_64
I did try to bisect the problem using the acoustid UI on that directory. It was able to get a signature without crashing but from getting that part to work I have a suspicion that it's using a different codepath (IIRC I was getting fingerprints from beets but needed to install another helper before the acoustid UI program would produce anything...)
In my attempts to bisect the problem it appears that it's likely a memory leak and not specific with the file. I copied the file to a single directory and the previous (successfully fingerprinted) one to another -both directories can be imported (skipped) without crash.
My hunch is that it's an underlying leak in the library you dlopen which is exacerbated by the combination of a very large library and beets being efficient and re-using a DSO for a lot more uses than the typical use case (scanning a single album and uploading results).
I'd be happy to repeat with a simple debugging patch if you'd like, or just grab something like /proc/
and FYI system is relatively stock Fedora 28 with: $ rpm -q libchromaprint python3-acoustid beets libchromaprint-1.4.2-2.fc28.x86_64 python3-acoustid-1.1.5-4.fc28.noarch beets-1.4.6-4.fc28.noarch
nayfield
commented
6 years ago And if my hunch is right you can sidestep the issue in libchromaprint by either giving your import threads a TTL (exit after 200 imports) or at least destroying the Fingerprinter object periodically (if I read the python binding correctly).
The former pattern might be good defensive practice in general - beet import putting a TB of music through a given DSO might exercise edge cases that are never seen in 'regular' use cases.
jackwilsdon
commented
6 years ago I think you're seeing a different issue @nayfield, it might be worth opening a new ticket.
sampsyo
commented
6 years ago It does sound like there's a separate issue here (a memory leak) deserving of a separate issue. In particular, running a Python memory profiler to check whether the leak is in our code or in the Chromaprint library would be most useful.
nayfield
commented
6 years ago it's a leak and in your code since it's been happening with chroma off. will open a new issue once i get more detail
nayfield
commented
6 years ago I stand corrected due to cowboy practices. The leak is in gstreamer
stale[bot]
commented
4 years ago Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Problem
Running
beet fingerprint ''on main directory fails.Running this command in verbose (
-vv) mode:Led to this problem:
Setup
My configuration (output of
beet config) is: