Closed FredSRichardson closed 2 weeks ago
My fix for this was to use soundfile.info()
(as you did in other parts of the same code) to extract all the meta data needed to create a Recording
object (after confirming the audio file exists, the above example won't work). I cached the meta data by path string to ease up on file I/O (so it should be called only once per a unique audio file).
Hello @FredSRichardson :)
fix: #10060
Update the LazyNeMoIterator to use the full duration for the Recordingdel
@neehhaa06 The current code in nemo/collections/common/data/lhotse/nemo_adapters.py
uses soundfile
to avoid I/O overhead of loading the whole file. The related comment is here for the LazyNeMoTarredIterator
class:
raw_audio = tar.extractfile(tar_info).read()
# Note: Lhotse has a Recording.from_bytes() utility that we won't use here because
# the profiling indicated significant overhead in torchaudio ffmpeg integration
# that parses full audio instead of just reading the header for WAV files.
# recording = lhotse.Recording.from_bytes(raw_audio, recording_id=tar_info.path)
meta = soundfile.info(BytesIO(raw_audio))
The meta
variable above returned by soundfile.info()
is very useful. It includes the sample rate, number of samples, number of channels, etc. I think this is the best thing to use to address this bug report in addition to caching the meta data using the file path string using a dict.
Thanks, it looks like there's indeed an issue. Do you mind submitting a PR with your fix?
Hello @pzelasko :) Are you asking me?
I was asking @FredSRichardson since he mentioned he managed to fix it locally. That said you're also welcome to contribute @neehhaa06 :)
I can't contribute directly unfortunately, but I did leave a fairly detailed spec of the change and it's fairly minor. Let me know if you have any questions.
Thanks, appreciate it. I added tests and a fix here https://github.com/NVIDIA/NeMo/pull/10198
Describe the bug These lines of code do not handle a cut's duration correctly when the offset is not zero: From
nemo/collections/common/data/lhotse/nemo_adapters.py
for the classLazyNeMoIterator
:The issue is that the function
_create_recording()
in the same file uses theduration
parameter to compute the number of samples for the whole recording in the returned lhotseRecording
instance. The duration in the nemo manifest is for the cut starting at theoffset
value within the recording. The duration of the whole recording has to be obtained from somewhere else (possibly the recording itself).Steps/Code to reproduce bug
Run
scripts/speech_recognition/estimate_duration_bins.py -b 30 manifest.json
using a manifest file containing non-zero offsets for example:You'll notice that the
mean
duration in the output is a negative number.Expected behavior
The duration of the whole audio file should be using when creating the
Recording
object in the method_create_recording()
innemo/collections/common/data/lhotse/nemo_adapters.py
. The reported mean duration should not be negative.Environment overview (please complete the following information)
Environment details
If NVIDIA docker image is used you don't need to specify these. Otherwise, please provide:
Additional context
Add any other context about the problem here. Example: GPU model