struct.error: 'L' format requires 0 <= number <= 4294967295

[danielw97]

Hi, This may be related to #87, although I've been doing some testing on wsl now and would really love to get to the bottom of this and possibly find a solution as I'm sure this will affect other people going forward. When converting longer books that would create a .wav file greater than 4 GB, the following error is produced during the final concatenation: Traceback (most recent call last):
File "/home/daniel/miniconda3/envs/epub2tts/bin/epub2tts", line 8, in
sys.exit(main())
^^^^^^
File "/home/daniel/miniconda3/envs/epub2tts/lib/python3.11/site-packages/epub2tts.py", line 735, in main
mybook.read_book(
File "/home/daniel/miniconda3/envs/epub2tts/lib/python3.11/site-packages/epub2tts.py", line 561, in read_book
concatenated.export(outputm4a, format="ipod", bitrate=bitrate)
File "/home/daniel/miniconda3/envs/epub2tts/lib/python3.11/site-packages/pydub/audio_segment.py", line 895, in export wave_data.writeframesraw(pcm_for_wav)
File "/home/daniel/miniconda3/envs/epub2tts/lib/python3.11/wave.py", line 547, in writeframesraw
self._ensure_header_written(len(data))
File "/home/daniel/miniconda3/envs/epub2tts/lib/python3.11/wave.py", line 588, in _ensure_header_written
self._write_header(datasize)
File "/home/daniel/miniconda3/envs/epub2tts/lib/python3.11/wave.py", line 600, in _write_header
self._file.write(struct.pack('<L4s4sLHHLLHH4s',
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
struct.error: 'L' format requires 0 <= number <= 4294967295
Exception ignored in: <function Wave_write.del at 0x7fdf61d86d40>
Traceback (most recent call last):
File "/home/daniel/miniconda3/envs/epub2tts/lib/python3.11/wave.py", line 447, in del
self.close()
File "/home/daniel/miniconda3/envs/epub2tts/lib/python3.11/wave.py", line 565, in close
self._ensure_header_written(0)
File "/home/daniel/miniconda3/envs/epub2tts/lib/python3.11/wave.py", line 588, in _ensure_header_written
self._write_header(datasize)
File "/home/daniel/miniconda3/envs/epub2tts/lib/python3.11/wave.py", line 600, in _write_header
self._file.write(struct.pack('<L4s4sLHHLLHH4s',
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
struct.error: 'L' format requires 0 <= number <= 4294967295

To me, this suggests that pydub converts everything back to wav, even though the input is flac. The wav header only allows for a 4 GB file, so I'm just wondering if there is any alternative for this final export step, e.g. passing the files to ffmpeg directly with the -i option, or something else? Here's hoping we can find a solution for this, and as always thanks for the great work.

[danielw97]

From some quick research, there appears to be an issue open about this at the pydub repo: https://github.com/jiaaro/pydub/issues/135

[aedocw]

Oooh this is a fun one! Thanks for sharing this detailed output, it's helpful. I will dig in this week and figure out how best to workaround this limitation. Also appreciate that link to the pydub issue, that is great!

[danielw97]

No worries, appreciate your willingness to check into this.

[aedocw]

Looks like this PR would fix the issue: https://github.com/jiaaro/pydub/pull/734

In the mean time I'll take a shot at doing a shell call to ffmpeg to concat the files which should work out.

[aedocw]

I pushed up the linked branch (ffmpeg-concat) that should address this. It no longer transcodes to FLAC along the way, and as long as each individual chapter is less than 4gb I have a feeling this will work.

I also tested the following with three wav files, each 2.8G and it did not choke. ffmpeg -f concat -safe 0 -i list.txt -codec:a aac -b:a 192k -f ipod output.m4a That ended up with an audio file 51 hours long and 2.4G big.

I'm pretty confident this will work out for you now, let me know if you are able to test it and how it goes.

[danielw97]

Amazing, thanks for addressing this so quickly. I'll try it now although I suspect it's sorted and will write back with the result.

[danielw97]

Hi again, I kept the troublesome flacs from earlier, and converted them back to wav so that I could test this more quickly. However, I'm now getting an error which may have to do with the name of my files although not sure: Note this is on wsl.

[concat @ 0x555db56da2c0] Unsafe file name 'Stepping Back 1-2.wav'
filelist.txt: Operation not permitted

[danielw97]

Hi, I see an issue about this from a quick search, not sure if it helps: https://stackoverflow.com/questions/38996925/ffmpeg-concat-unsafe-file-name

[aedocw]

Hah, I did not realize "-safe 0" was missing from what I pushed up to the branch, just added that, please pull and try again. Thanks!

[danielw97]

Amazing, this worked perfectly, and as a bonus is faster than using pydub and uses less ram too. Thanks for fixing this so quickly.

[aedocw]

AWESOME!! Thanks so much for your help, really appreciate the collaboration :)

[danielw97]

Update: this may be something to add when murging this into main, although I don't believe the --bitrate flag would be taken into account as currently it's being specified in the ffmpeg command by the looks of things.

[aedocw]

Oh you're right, thanks for catching that!

[aedocw]

Fixed with https://github.com/aedocw/epub2tts/pull/179