ocean-data-factory-sweden / kso

Notebooks to upload/download marine footage, connect to a citizen science project, train machine learning models and publish marine biological observations.
GNU General Public License v3.0
4 stars 12 forks source link

Tut 9: Aggregation of detections in plot detections custom footage issue on cloudina #362

Closed Bergylta closed 3 months ago

Bergylta commented 4 months ago

🐛 Bug

When running custom footage through tutorial 9, the aggregation to plot the detections does not work

To Reproduce (REQUIRED)

Input: Custom footage

Output:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[15], line 1
----> 1 t_utils.plot_processed_detections(
      2     df=dets_df,
      3     thres=10,  # number of seconds for thresholding in interval
      4     int_length=10,  # length in seconds of interval for filtering
      5 )

File ~/kso/kso_utils/tutorials_utils.py:1705, in plot_processed_detections(df, thres, int_length)
   1701 interval = pd.Grouper(key="seconds_since_reference", freq=str(int_length) + "S")
   1703 # Group by species and minute, calculate the count
   1704 max_count_per_species = (
-> 1705     df.groupby(["movie_id", "commonName", interval])["max_n"].max().reset_index()
   1706 )
   1708 # Enable plotting of matplotlib
   1709 try:

File ~/.local/lib/python3.10/site-packages/pandas/core/frame.py:7714, in DataFrame.groupby(self, by, axis, level, as_index, sort, group_keys, squeeze, observed, dropna)
   7709 axis = self._get_axis_number(axis)
   7711 # https://github.com/python/mypy/issues/7642
   7712 # error: Argument "squeeze" to "DataFrameGroupBy" has incompatible type
   7713 # "Union[bool, NoDefault]"; expected "bool"
-> 7714 return DataFrameGroupBy(
   7715     obj=self,
   7716     keys=by,
   7717     axis=axis,
   7718     level=level,
   7719     as_index=as_index,
   7720     sort=sort,
   7721     group_keys=group_keys,
   7722     squeeze=squeeze,  # type: ignore[arg-type]
   7723     observed=observed,
   7724     dropna=dropna,
   7725 )

File ~/.local/lib/python3.10/site-packages/pandas/core/groupby/groupby.py:882, in GroupBy.__init__(self, obj, keys, axis, level, grouper, exclusions, selection, as_index, sort, group_keys, squeeze, observed, mutated, dropna)
    879 if grouper is None:
    880     from pandas.core.groupby.grouper import get_grouper
--> 882     grouper, exclusions, obj = get_grouper(
    883         obj,
    884         keys,
    885         axis=axis,
    886         level=level,
    887         sort=sort,
    888         observed=observed,
    889         mutated=self.mutated,
    890         dropna=self.dropna,
    891     )
    893 self.obj = obj
    894 self.axis = obj._get_axis_number(axis)

File ~/.local/lib/python3.10/site-packages/pandas/core/groupby/grouper.py:882, in get_grouper(obj, key, axis, level, sort, observed, mutated, validate, dropna)
    880         in_axis, level, gpr = False, gpr, None
    881     else:
--> 882         raise KeyError(gpr)
    883 elif isinstance(gpr, Grouper) and gpr.key is not None:
    884     # Add key to exclusions
    885     exclusions.add(gpr.key)

KeyError: 'commonName'

Expected behavior

A clear and concise description of what you expected to happen.

Environment

If applicable, add screenshots to help explain your problem.

Additional context

Add any other context about the problem here.

victor-wildlife commented 3 months ago

@Bergylta what is the custom footage you are trying to run (e.g. is this a folder in SNIC I can try with on my end)?

Bergylta commented 3 months ago

@victor-wildlife , i think it should be the GARV_01.mp4, which on cloudina lies in the concat gobies 1 folder. That being said, i am too getting the artifact download error right now.