search

dpwe / audfprint

Landmark-based audio fingerprinting
MIT License
536 stars 121 forks source link

Numpy Memory Error #66

Closed Johndirr closed 4 years ago

Johndirr commented 5 years ago

When trying to build a database from an audio file thats about an hour long I get the following error:

File "C:\Temp\Python\audfprint-master\stft.py", line 93, in stft
    windowed_frames = frames * window
numpy.core._exceptions.MemoryError: Unable to allocate array with shape (148776, 512) and data type float64

I tried this with an mp3 file and a wav (both have the same duration). It works fine for much shorter audio. Is there anything I can do about this besides splitting the file?

PS. I'm on windows 10.

Hellowlol commented 5 years ago

Add more memory? 😂

dpwe commented 5 years ago

Splitting the audio is a good approach because matching is more efficient when applied to shorter units (e.g. a couple of minutes). You can use overlapping segments to avoid missing a match that spans a split.

DAn.

On Fri, Aug 16, 2019 at 18:27 RetroHelix notifications@github.com wrote:

When trying to build a database from an audio file thats about an hour long I get the following error:

File "C:\Temp\Python\audfprint-master\stft.py", line 93, in stft windowed_frames = frames * window numpy.core._exceptions.MemoryError: Unable to allocate array with shape (148776, 512) and data type float64

I tried this with an mp3 file and a wav (both have the same duration). It works fine for much shorter audio. Is there anything I can do about this besides splitting the file?

PS. I'm on windows 10.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/dpwe/audfprint/issues/66?email_source=notifications&email_token=AAEGZULKXXTWCZGHOQIMT2DQE3POBA5CNFSM4IMLKCEKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HFWJOIQ, or mute the thread https://github.com/notifications/unsubscribe-auth/AAEGZUPA6UKQAP6TJXLSH33QE3POBANCNFSM4IMLKCEA .

Johndirr commented 4 years ago

Add more memory? 😂

Tried it with 24 GB of free memory. Same error as before. Is this process really so demanding?

Hellowlol commented 4 years ago

How long is the audioclip?

Johndirr commented 4 years ago

About an hour.

dpwe commented 4 years ago

The original error message reports problems allocating a 148776 * 512 array of float64, which sounds right for 1 hour of 23 ms frames (the default frame duration). That corresponds to ~600MB by my calculation, which is a lot of memory, but should be feasible on a machine with many GB of RAM. Are you sure Python has access to the memory? Perhaps psutil.virtual_memory() can give useful information?

DAn.

On Sun, Oct 27, 2019 at 9:24 AM RetroHelix notifications@github.com wrote:

Add more memory? 😂

Tried it with 24 GB of free memory. Same error as before. Is this process really so demanding?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/dpwe/audfprint/issues/66?email_source=notifications&email_token=AAEGZUKXSNQRK5BCTU5NWJLQQWCBBA5CNFSM4IMLKCEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECK6KAA#issuecomment-546694400, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEGZUIVVRXIXSL7QXAXELLQQWCBBANCNFSM4IMLKCEA .

Johndirr commented 4 years ago

I searched for memory errors in genereal after you wrote about psutil. In the end the problem was a 32bit installation of Python. It works with a 64bit installation of Python.

Thank you for your time.