search

QutEcoacoustics / audio-analysis

The audio analysis code (AnalysisPrograms.exe) for the QUT Ecoacoustics Research Group
https://ap.qut.ecoacoustics.info/
Apache License 2.0
52 stars 12 forks source link

Bug with the ArdeaInsignis recognizer #100

Closed atruskie closed 7 years ago

atruskie commented 7 years ago

Only occurs when events are detected:

Arguments:

"C:\Users\Administrator\Desktop\Sensors Analysis\ParallelExecutables\2\AnalysisPrograms.exe" audio2csv -source "Y:\Tshering\WBH_Walaytar\201505 - second deployment\Site2_Waklaytar\24Hours WBH_28032016\WBH12HOURS-D_20160403_120000.wav" -config "C:\Users\Administrator\Desktop\Sensors Analysis\Towsey.ArdeaInsignis.Parallel.yml" -output "Y:\Results\2016Oct31-145124\Tshering\WBH_Walaytar\201505 - second deployment\Site2_Waklaytar\24Hours WBH_28032016\WBH12HOURS-D_20160403_120000.wav" -tempdir F:\2 -m True  -n

Exception:

2016-10-31T13:55:31.5233416+10:00 [1] FATAL AnalysisPrograms.MainEntry - Unhandled exception ->
System.AggregateException: One or more errors occurred. ---> System.IndexOutOfRangeException: Index was outside the bounds of the array.
   at AnalysisPrograms.Recognizers.ArdeaInsignis.Recognize(AudioRecording recording, Object configuration, TimeSpan segmentStartOffset, Lazy`1 getSpectralIndexes, DirectoryInfo outputDirectory, Nullable`1 imageWidth) in C:\Work\Github\audio-analysis\AudioAnalysis\AnalysisPrograms\Recognizers\ArdeaInsignis.cs:line 221
   at AnalysisPrograms.Recognizers.Base.RecognizerBase.Analyze(AnalysisSettings analysisSettings) in C:\Work\Github\audio-analysis\AudioAnalysis\AnalysisPrograms\Recognizers\Base\RecognizerBase.cs:line 72
   at AnalysisBase.AnalysisCoordinator.PrepareFileAndRunAnalysis(FileSegment fileSegment, IAnalyser2 analyser, AnalysisSettings localCopyOfSettings, Boolean parallelised) in C:\Work\Github\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 431
   at AnalysisBase.AnalysisCoordinator.ProcessItem(FileSegment item, IAnalyser2 analysis, AnalysisSettings clonedSettings, Boolean parallelised) in C:\Work\Github\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 668
   at AnalysisBase.AnalysisCoordinator.<>c__DisplayClass29_0.<RunParallel>b__0(FileSegment item, ParallelLoopState state, Int64 index) in C:\Work\Github\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 289
   at System.Threading.Tasks.Parallel.<>c__DisplayClass17_0`1.<ForWorker>b__1()
   at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
   at System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object )
   --- End of inner exception stack trace ---
atruskie commented 7 years ago

@towsey any update on this?

atruskie commented 7 years ago

@towsey any update on this?

towsey commented 7 years ago

yes! I altered the bounds of a loop and committed the change. I did not test the result on the file that failed. Asked Tshering to resubmit just one recording to check the output.

atruskie commented 7 years ago

I'll assume that is commit f25bb4416d82ff9b2245c11c4bd0ab8d7e87c6ba ?

@tsheringde have you tested it?

atruskie commented 7 years ago

log.txt

More IndexOutOfRangeExceptions - this time from TowseyLibrary.DataTools.Subarray. Only occurs in files that have events.

tsheringde commented 7 years ago

@towsey, Can you please fix the error in the Heron Recognizer. Thank you.

atruskie commented 7 years ago

I'm processing the backlog of issues on https://github.com/QutBioacoustics/Ecoacoustics/issues and this bug is blocking. Any progress here @towsey ?

atruskie commented 7 years ago

@towsey has the latest bug with the heron recognizer been resolved? Do you need help? Is something blocking you?

towsey commented 7 years ago

I have been waiting on Tshering, but our paths do not seem to cross. I have asked her to check the latest output. In short, still working on it.

tsheringde commented 7 years ago

Anthony, I got any email from Michael that the bug has been resolved and rerun. I will look at the results shortly after finishing my pilot study report today. I will go through the results tonight and get back tomorrow. Thank you.

tsheringde commented 7 years ago

@towsey , the given path for results seems mistaken. Could you kindly verify please.

atruskie commented 7 years ago

From @towsey - the bug has been fixed in d7eda701d8c6820f51ac230c7499f19c58c878fa bffa66de795e1e07c32d09082e61f8f025152024

atruskie commented 7 years ago

@towsey more bugs, log excerpt (failure at minute 101 (analysis was done in serial this time)):

2016-12-13T13:40:01.7637508+10:00 [1] INFO  CleanLogger - QUT Bioacoustic Analysis Program - version 16.12.3674.0 (RELEASE build, 13/12/2016 13:14) 
Git branch-version: master-d7eda701d8c6820f51ac230c7499f19c58c878fa
Copyright QUT 2016
2016-12-13T13:40:01.8247986+10:00 [1] INFO  LogFileOnly - Executable called with these arguments: 
"C:\Users\Administrator\Desktop\Sensors Analysis\ParallelExecutables\2\AnalysisPrograms.exe" audio2csv -source "Y:\Tshering\WBH_Walaytar\201505 - second deployment\Site2_Waklaytar\24Hours WBH_28032016\WBH12HOURS-D_20160403_120000.wav" -config "C:\Users\Administrator\Desktop\Sensors Analysis\Towsey.ArdeaInsignis.Parallel.yml" -output "Y:\Results\2016Dec13-143946\Tshering\WBH_Walaytar\201505 - second deployment\Site2_Waklaytar\24Hours WBH_28032016\WBH12HOURS-D_20160403_120000.wav" -tempdir F:\2 -m True  -n

2016-12-13T13:40:02.3532296+10:00 [1] INFO  CleanLogger - # PROCESS LONG RECORDING
2016-12-13T13:40:02.3532296+10:00 [1] INFO  CleanLogger - # DATE AND TIME: 13/12/2016 13:40:02
2016-12-13T13:40:02.3542304+10:00 [1] INFO  CleanLogger - # Recording file:      Y:\Tshering\WBH_Walaytar\201505 - second deployment\Site2_Waklaytar\24Hours WBH_28032016\WBH12HOURS-D_20160403_120000.wav
2016-12-13T13:40:02.3542304+10:00 [1] INFO  CleanLogger - # Configuration file:  C:\Users\Administrator\Desktop\Sensors Analysis\Towsey.ArdeaInsignis.Parallel.yml
2016-12-13T13:40:02.3542304+10:00 [1] INFO  CleanLogger - # Output folder:       Y:\Results\2016Dec13-143946\Tshering\WBH_Walaytar\201505 - second deployment\Site2_Waklaytar\24Hours WBH_28032016\WBH12HOURS-D_20160403_120000.wav
2016-12-13T13:40:02.3542304+10:00 [1] INFO  CleanLogger - # Temp File Directory: F:\2
2016-12-13T13:40:02.7345333+10:00 [1] INFO  CleanLogger - # IndexProperties Cfg: C:\Users\Administrator\Desktop\Sensors Analysis\ParallelExecutables\2\ConfigFiles\IndexPropertiesConfigHiRes.yml
2016-12-13T13:40:02.7405377+10:00 [1] INFO  CleanLogger - # Resolved IndexProperties Cfg: C:\Users\Administrator\Desktop\Sensors Analysis\ParallelExecutables\2\ConfigFiles\IndexPropertiesConfigHiRes.yml
2016-12-13T13:40:02.7545498+10:00 [1] INFO  AnalysisPrograms.AnalyseLongRecordings.AnalyseLongRecording - Minimum event threshold has been set to 0.3
2016-12-13T13:40:02.7605538+10:00 [1] WARN  Acoustics.Shared.FileDateHelpers - File date parse cannot parse date 20160403-120000 - a timezone offset hint was not provided to the function for a date format with no timezone information
2016-12-13T13:40:03.0037520+10:00 [1] INFO  AnalysisPrograms.Acoustic - Image tiling is NOT enabled
2016-12-13T13:40:03.0247885+10:00 [1] INFO  CleanLogger - START ANALYSIS ...
2016-12-13T13:40:04.4929552+10:00 [1] INFO  AnalysisBase.AnalysisCoordinator - Analysis started in sequence.
2016-12-13T13:43:42.8693635+10:00 [1] INFO  AnalysisBase.AnalysisCoordinator - Completed segment 97/406
2016-12-13T13:43:45.0821471+10:00 [1] INFO  AnalysisBase.AnalysisCoordinator - Completed segment 98/406
2016-12-13T13:43:47.0527281+10:00 [1] INFO  AnalysisBase.AnalysisCoordinator - Completed segment 99/406
2016-12-13T13:43:49.2454930+10:00 [1] INFO  AnalysisBase.AnalysisCoordinator - Completed segment 100/406
2016-12-13T13:43:50.6106067+10:00 [1] FATAL AnalysisPrograms.MainEntry - Unhandled exception ->
System.IndexOutOfRangeException: Index was outside the bounds of the array.
   at AnalysisPrograms.Recognizers.ArdeaInsignis.Recognize(AudioRecording recording, Object configuration, TimeSpan segmentStartOffset, Lazy`1 getSpectralIndexes, DirectoryInfo outputDirectory, Nullable`1 imageWidth) in C:\Work\Github\audio-analysis\AudioAnalysis\AnalysisPrograms\Recognizers\ArdeaInsignis.cs:line 77
   at AnalysisPrograms.Recognizers.Base.RecognizerBase.Analyze(AnalysisSettings analysisSettings) in C:\Work\Github\audio-analysis\AudioAnalysis\AnalysisPrograms\Recognizers\Base\RecognizerBase.cs:line 58
   at AnalysisBase.AnalysisCoordinator.PrepareFileAndRunAnalysis(FileSegment fileSegment, IAnalyser2 analyser, AnalysisSettings localCopyOfSettings, Boolean parallelised) in C:\Work\Github\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 431
   at AnalysisBase.AnalysisCoordinator.RunSequential(IEnumerable`1 analysisSegments, IAnalyser2 analysis, AnalysisSettings clonedSettings) in C:\Work\Github\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 328
   at AnalysisBase.AnalysisCoordinator.Run(IEnumerable`1 fileSegments, IAnalyser2 analysis, AnalysisSettings settings) in C:\Work\Github\audio-analysis\AudioAnalysis\AnalysisBase\AnalysisCoordinator.cs:line 233
   at AnalysisPrograms.AnalyseLongRecordings.AnalyseLongRecording.Execute(Arguments arguments) in C:\Work\Github\audio-analysis\AudioAnalysis\AnalysisPrograms\AnalyseLongRecordings\AnalyseLongRecording.cs:line 217
   at System.Dynamic.UpdateDelegates.UpdateAndExecuteVoid2[T0,T1](CallSite site, T0 arg0, T1 arg1)
   at PowerArgs.ArgAction`1.Invoke() in c:\Work\Software Dev\Github\PowerArgs\PowerArgs\ArgAction.cs:line 38
   at AnalysisPrograms.MainEntry.Main(String[] args) in C:\Work\Github\audio-analysis\AudioAnalysis\AnalysisPrograms\MainEntry.cs:line 55
2016-12-13T13:43:50.6215989+10:00 [1] INFO  LogFileOnly - ERRORLEVEL: 200
towsey commented 7 years ago

Resolution to issue: See commit: 482ae826783e7cdf4ea1f1ad08181d0f784c4573

Just to let you know I have run the revised heron recognizer on the 7 hour recording that contains 33 calls. Now achieving 100% recall with just one duplicate detection due to the segment overlap. I have reduced the segment overlap to 5 sec. This helps to reduce duplicate hits. The csv file of events appears to contain correct values for starts and ends of calls, relative to start of each segment etc. However, the csv columns having the header names FreqBinCount, FrameDuration, FrameCount, Profile, FileName and HitElements are either filled with zeros or blank. Of these, only FrameCount would change from event to event. Not sure why Profile and HitElements are included as column headings as they have no relevance at this stage.