search

lifewatch / pypam

Python Passive Acoustic Analysis tool for Passive Acoustic Monitoring (PAM)
GNU General Public License v3.0
32 stars 8 forks source link

Status of unit tests #7

Closed carueda closed 1 year ago

carueda commented 1 year ago

Hi @cparcerisas I've just synced my clone and ran the unit tests, python -m unittest. Since there are several errors, I wonder about the status of these tests. Some of the errors are related to missing referenced files (summarized output below); others related with KeyError. Thanks.

(virtenv) ➜  pypam git:(master) ✗ python -m unittest

...
======================================================================
ERROR: test_millidecade_bands (tests.test_acoustic_file.TestAcuFile)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/carueda/github/pypam/tests/test_acoustic_file.py", line 20, in setUp
...
soundfile.LibsndfileError: Error opening 'test_data/67416073.210610033655.wav': System error.

======================================================================
ERROR: test_plots (tests.test_acoustic_file.TestAcuFile)
----------------------------------------------------------------------
...
soundfile.LibsndfileError: Error opening 'test_data/67416073.210610033655.wav': System error.

======================================================================
ERROR: test_generate_dataset (tests.test_dataset.TestDataset)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/carueda/github/pypam/tests/test_dataset.py", line 62, in setUp
    self.ds = dataset.DataSet(summary_path, output_folder, instruments, temporal_features=temporal_features,
...
    handle = open(
FileNotFoundError: [Errno 2] No such file or directory: 'test_data/data_summary.csv'

======================================================================
ERROR: tests.test_utils (unittest.loader._FailedTest)
----------------------------------------------------------------------
...
    handle = open(
FileNotFoundError: [Errno 2] No such file or directory: './test_data/signal_data.csv'
cparcerisas commented 1 year ago

Hi @carueda, all the errors are because you do not have the files downloaded. You can see them here https://github.com/lifewatch/pypam/tree/master/tests/test_data. Can you check if you have them in you cloned repository? They should not be included in the regular distribution (because of the size) but yes when cloning the repository. If you have them then the problem is probably that the relative path is not correctly specified in the tests. Maybe a work around is to change the running directory of your tests?

carueda commented 1 year ago

Hi @cparcerisas – You are right, the files are in the clone, and I just needed to run python -m unittest under tests/ (not under the root of the repo). All the file-related tests are now completing OK.

Output of remaining errors is enclosed below, and I'll try to have a look into them myself as part of the idea of learning more about the internals of pypam and eventually helping with fixes and such. Thanks.

ERROR: test_apply_to_all (test_acoustic_survey.TestASA)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 3803, in get_loc
    return self._engine.get_loc(casted_key)
  File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 146, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index_class_helper.pxi", line 49, in pandas._libs.index.Int64Engine._check_type
KeyError: numpy.datetime64('2021-06-10T03:36:55.000000000')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/carueda/github/pypam/tests/test_acoustic_survey.py", line 61, in test_apply_to_all
    self.asa.apply_to_all('plot_spectrogram')
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/lifewatch_pypam-0.1.8-py3.9.egg/pypam/acoustic_survey.py", line 269, in apply_to_all
    f(sound_file)
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/lifewatch_pypam-0.1.8-py3.9.egg/pypam/acoustic_file.py", line 1152, in plot_spectrogram
    plots.plot_spectrograms(ds_spectrogram, log, db, self.p_ref, save_path)
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/lifewatch_pypam-0.1.8-py3.9.egg/pypam/plots.py", line 72, in plot_spectrograms
    sxx = ds_spectrogram['spectrogram'].loc[time_bin]
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/xarray/core/dataarray.py", line 199, in __getitem__
    return self.data_array.sel(key)
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/xarray/core/dataarray.py", line 1329, in sel
    ds = self._to_temp_dataset().sel(
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/xarray/core/dataset.py", line 2501, in sel
    pos_indexers, new_indexes = remap_label_indexers(
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/xarray/core/coordinates.py", line 421, in remap_label_indexers
    pos_indexers, new_indexes = indexing.remap_label_indexers(
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/xarray/core/indexing.py", line 121, in remap_label_indexers
    idxr, new_idx = index.query(labels, method=method, tolerance=tolerance)
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/xarray/core/indexes.py", line 241, in query
    indexer = self.index.get_loc(label_value)
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 3805, in get_loc
    raise KeyError(key) from err
KeyError: numpy.datetime64('2021-06-10T03:36:55.000000000')

======================================================================
ERROR: test_detect_piling_events (test_acoustic_survey.TestASA)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/carueda/github/pypam/tests/test_acoustic_survey.py", line 78, in test_detect_piling_events
    self.asa.detect_piling_events(max_duration=max_duration, min_separation=min_separation,
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/lifewatch_pypam-0.1.8-py3.9.egg/pypam/acoustic_survey.py", line 452, in detect_piling_events
    df_output = sound_file.detect_piling_events(min_separation=min_separation,
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/lifewatch_pypam-0.1.8-py3.9.egg/pypam/acoustic_file.py", line 1013, in detect_piling_events
    events_df = detector.detect_events(signal, method=method, verbose=verbose, save_path=file_path)
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/lifewatch_pypam-0.1.8-py3.9.egg/pypam/impulse_detector.py", line 74, in detect_events
    df = self.detect_events_snr(signal, verbose, save_path)
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/lifewatch_pypam-0.1.8-py3.9.egg/pypam/impulse_detector.py", line 154, in detect_events_snr
    events_df = self.load_all_times_events(times_events, signal)
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/lifewatch_pypam-0.1.8-py3.9.egg/pypam/impulse_detector.py", line 214, in load_all_times_events
    events_df.at[i, ('temporal', columns_temp)] = [start, end, duration, rms, sel, peak]
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/pandas/core/indexing.py", line 2442, in __setitem__
    return super().__setitem__(key, value)
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/pandas/core/indexing.py", line 2397, in __setitem__
    self.obj._set_value(*key, value=value, takeable=self._takeable)
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/pandas/core/frame.py", line 4207, in _set_value
    icol = self.columns.get_loc(col)
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/pandas/core/indexes/multi.py", line 2899, in get_loc
    self._check_indexing_error(key)
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/pandas/core/indexes/multi.py", line 2582, in _check_indexing_error
    raise InvalidIndexError(key)
pandas.errors.InvalidIndexError: ('temporal', ['start_seconds', 'end_seconds', 'duration', 'rms', 'sel', 'peak'])

======================================================================
ERROR: test_detect_ship_events (test_acoustic_survey.TestASA)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 3803, in get_loc
    return self._engine.get_loc(casted_key)
  File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 144, in pandas._libs.index.IndexEngine.get_loc
TypeError: 'slice(None, None, None)' is an invalid key

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/carueda/github/pypam/tests/test_acoustic_survey.py", line 84, in test_detect_ship_events
    self.asa.detect_ship_events(0.1, 0.5)
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/lifewatch_pypam-0.1.8-py3.9.egg/pypam/acoustic_survey.py", line 488, in detect_ship_events
    df_output = sound_file.detect_ship_events(min_duration=min_duration,
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/lifewatch_pypam-0.1.8-py3.9.egg/pypam/acoustic_file.py", line 1046, in detect_ship_events
    events_df = detector.detect_events(signal, verbose=verbose)
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/lifewatch_pypam-0.1.8-py3.9.egg/pypam/loud_event_detector.py", line 87, in detect_events
    events_df.at[i] = {'start_seconds': start_i, 'end_seconds': end_points[i],
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/pandas/core/indexing.py", line 2442, in __setitem__
    return super().__setitem__(key, value)
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/pandas/core/indexing.py", line 2397, in __setitem__
    self.obj._set_value(*key, value=value, takeable=self._takeable)
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/pandas/core/frame.py", line 4207, in _set_value
    icol = self.columns.get_loc(col)
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 3810, in get_loc
    self._check_indexing_error(key)
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 5966, in _check_indexing_error
    raise InvalidIndexError(key)
pandas.errors.InvalidIndexError: slice(None, None, None)

======================================================================
ERROR: test_nmf (test_acoustic_survey.TestASA)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/carueda/github/pypam/tests/test_acoustic_survey.py", line 47, in test_nmf
    self.asa = ASA(hydrophone=soundtrap, folder_path='./../data', binsize=binsize, nfft=nfft, utc=True,
TypeError: __init__() got an unexpected keyword argument 'utc'

======================================================================
ERROR: test_generate_dataset (test_dataset.TestDataset)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/carueda/github/pypam/tests/test_dataset.py", line 67, in test_generate_dataset
    self.ds()
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/lifewatch_pypam-0.1.8-py3.9.egg/pypam/dataset.py", line 86, in __call__
    self[idx]
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/lifewatch_pypam-0.1.8-py3.9.egg/pypam/dataset.py", line 95, in __getitem__
    deployment = self.generate_deployment(idx=idx)
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/lifewatch_pypam-0.1.8-py3.9.egg/pypam/dataset.py", line 145, in generate_deployment
    asa = acoustic_survey.ASA(hydrophone,
TypeError: __init__() got an unexpected keyword argument 'overlap'

----------------------------------------------------------------------
Ran 20 tests in 349.566s

FAILED (errors=5)
carueda commented 1 year ago

Using the reorg_some_tests branch associated to PR #8 (which basically reflects the same failures as reported above), and focusing on tests.test_dataset here:

$ PYPAM_TEST_NO_PLOTS= python -m unittest tests.test_dataset
...
 File "/Users/carueda/github/pypam/pypam/dataset.py", line 145, in generate_deployment
    asa = acoustic_survey.ASA(hydrophone,
TypeError: __init__() got an unexpected keyword argument 'overlap'

I changed the keyword argument to fft_overlap, which seems to have been introduced in 4133bdaadcbaad1d4a8fcc6b60a8710d69be8869. This "fixes" that error but I note also a bin_overlap was introduced, and I'm not sure which would actually be correct one here.

Retrying the test (with both fft_overlap=self.overlap and bin_overlap=self.overlap as "fix" to the above):

Traceback (most recent call last):
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/xarray/core/dataset.py", line 1394, in _construct_dataarray
    variable = self._variables[name]
KeyError: 'rms'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/carueda/github/pypam/tests/test_dataset.py", line 67, in test_generate_dataset
    self.ds()
  File "/Users/carueda/github/pypam/pypam/dataset.py", line 86, in __call__
    self[idx]
  File "/Users/carueda/github/pypam/pypam/dataset.py", line 95, in __getitem__
    deployment = self.generate_deployment(idx=idx)
  File "/Users/carueda/github/pypam/pypam/dataset.py", line 161, in generate_deployment
    ds = ds.merge(temporal_evo[f])
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/xarray/core/dataset.py", line 1498, in __getitem__
    return self._construct_dataarray(key)
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/xarray/core/dataset.py", line 1396, in _construct_dataarray
    _, name, variable = _get_virtual_variable(
  File "/Users/carueda/github/pypam/virtenv/lib/python3.9/site-packages/xarray/core/dataset.py", line 169, in _get_virtual_variable
    ref_var = variables[ref_name]
KeyError: 'rms'

I'm much less sure about how to fix this.

carueda commented 1 year ago

Hi @cparcerisas - thanks again for merging that PR ; )

As part of that merge (as you've probably already noticed), CI via GitHub actions is now in place: https://github.com/lifewatch/pypam/actions . For now, it only targets python 3.9 while we are able to fix those remaining unit tests. Then we can easily enable other python versions as needed. All of this will play nicely with other overall adjustments, including the use of Poetry #6, moving forward.