Create a flow chart for passive acoustic monitoring data

MathewBiddle commented 11 months ago

Initial attempt at visualizing what is documented at https://www.ncei.noaa.gov/products/passive-acoustic-data#tab-3561

%%{
  init: {
    'theme': 'base',
    'themeVariables': {
      'primaryColor': '#007396',
      'primaryTextColor': '#fff',
      'primaryBorderColor': '#003087',
      'lineColor': '#003087',
      'secondaryColor': '#007396',
      'tertiaryColor': '#CCD1D1'
    },
   'flowchart': { 'curve': 'basis' }
  }
}%%

flowchart TD

A["hydrophone"] 

B{{"PassivePacker"}}

C{{"Darwin Core
Alignment"}}

D[("NCEI")]

E[("IPT
OBIS-USA")]

F[/"MBON 
Data Portal"\]

G([OBIS])

H([GBIF])

P[(IOOS RA ERDDAP)]

I(["IOOS Data Catalog
(data.ioos.us)"])

J([NOAA OneStop])

K([data.gov])

L(["Commerce
Data Hub"])

M[/"IOC-UNESCO Harmful Algae Information System"\]

N[/"Infographics"\]

%% O["GitHub /
%% protocols.io"]

A -- Species Detections --> C
A --raw audio files & 1-min hybrid millidecade bands --> B
B --> D
%% A -- Protocols --> O
C --> E
E --> D
%% O .-> E
E --> G
E --> H

%% NOAA STUFF
%% A -- Environmental Observations --> D
A -- non-acoustic data --> P
P --> I
P -- raw --> D
P -- occurrence --> C
I --> J
I --> K
I --> L
D --> J
D --> K
D --> L

G .-> Q
H .-> Q
F .-> Q
D .-> Q
J .-> Q
K .-> Q
L .-> Q

subgraph Q[Example Products]
M
N
F
end

click C "https://doi.org/10.35035/doc-vf1a-nr22" "GBIF eDNA Manual" _blank
click D "https://www.ncei.noaa.gov" "NCEI" _blank
click F "https://mbon.ioos.us" "MBON" _blank
click G "https://obis.org" "OBIS" _blank
click H "https://gbif.org" "GBIF" _blank
%%click I "https://data.ioos.us" "IOOS Catalog" _blank
%%click J "https://data.noaa.gov/onestop/" "NOAA OneStop" _blank
%%click K "https://data.gov" "data.gov" _blank

non-acoustic data is essentially anything measured in concert with the passive acoustic hydrophone. For example, abiotic measurements from sensors attached to the same platform, biotic observations nearby (camera trap), etc.

MathewBiddle commented 11 months ago

FYI @ioos/pamdata

MathewBiddle commented 11 months ago

Loosely based on the initial biological data flow at https://github.com/ioos/mbon-docs/issues/40#issuecomment-1581262699

MathewBiddle commented 10 months ago

Update to include sound propagation piece.

%%{
  init: {
    'theme': 'base',
    'themeVariables': {
      'primaryColor': '#007396',
      'primaryTextColor': '#fff',
      'primaryBorderColor': '#003087',
      'lineColor': '#003087',
      'secondaryColor': '#007396',
      'tertiaryColor': '#CCD1D1'
    },
   'flowchart': { 'curve': 'basis' }
  }
}%%

flowchart TD

A["hydrophone"] 

B{{"PassivePacker"}}

C{{"Darwin Core
Alignment"}}

D[("NCEI")]

E[("IPT
OBIS-USA")]

F[/"MBON 
Data Portal"\]

G([OBIS])

H([GBIF])

P[(IOOS RA ERDDAP)]

I(["IOOS Data Catalog
(data.ioos.us)"])

J([NOAA OneStop])

K([data.gov])

L(["Commerce
Data Hub"])

M[/"IOC-UNESCO Harmful Algae Information System"\]

N[/"Infographics"\]

%% O["GitHub /
%% protocols.io"]

R[/"NCEI Passive Acoustic Data Portal"\]

A -- Species Detections + sound propagation--> C
A --raw audio files & 1-min hybrid millidecade bands --> B
B --> D
%% A -- Protocols --> O
C --> E
E --> D
%% O .-> E
E --> G
E --> H

%% NOAA STUFF
%% A -- Environmental Observations --> D
A -- non-acoustic data --> P
P --> I
P -- raw --> D
P -- occurrence --> C
I --> J
I --> K
I --> L
D --> J
D --> K
D --> L

G .-> Q
H .-> Q
F .-> Q
D .-> Q
J .-> Q
K .-> Q
L .-> Q

D --> R

subgraph Q[Example Products]
M
N
F
end

MathewBiddle commented 9 months ago

%%{
  init: {
    'theme': 'base',
    'themeVariables': {
      'primaryColor': '#007396',
      'primaryTextColor': '#fff',
      'primaryBorderColor': '#003087',
      'lineColor': '#003087',
      'secondaryColor': '#007396',
      'tertiaryColor': '#CCD1D1'
    },
   'flowchart': { 'curve': 'basis' }
  }
}%%

flowchart TD

pA(("passive\nacoustic\nmonitoring"))

A[("hydrophone")]

B{{"PassivePacker"}}

C{{"Darwin Core
Alignment"}}

D[("National Centers\nfor\nEnvironmental Information\n(NCEI)")]

E[("IPT
OBIS-USA")]

F[/"MBON 
Data Portal"\]

G([OBIS])

H([GBIF])

%%P[(IOOS RA ERDDAP)]

%%I(["IOOS Data Catalog
%%(data.ioos.us)"])

J([NOAA OneStop])

K([data.gov])

L(["Commerce
Data Hub"])

M[/"IOC-UNESCO Harmful Algae Information System"\]

N[/"Infographics"\]

%% O["GitHub /
%% protocols.io"]

R[/"NCEI Passive Acoustic Data Portal"\]

pA --> A
A -- species detections\n+\nsound propagation--> C
A -- raw\nand\nprocessed\ndata --> B
B --> D
%% A -- Protocols --> O
C --> E
E --> D
%% O .-> E
E --> G
E --> H

%% NOAA STUFF
%% A -- Environmental Observations --> D
%%pA -- non-acoustic data --> P
%%P --> I
%%P -- raw --> D
%%P -- occurrence --> C
%%I --> J
%%I --> K
%%I --> L
D --> J
D --> K
D --> L

G --> D

G .-> Q
H .-> Q
F .-> Q
D .-> Q
J .-> Q
K .-> Q
L .-> Q

D --> R

subgraph Q[Example Products]
M
N
F
R
end

MathewBiddle commented 9 months ago

Cleaning up some of the nuances in the diagram. Grouping federal catalogs, and example products.

%%{
  init: {
    'theme': 'base',
    'themeVariables': {
      'primaryColor': '#007396',
      'primaryTextColor': '#fff',
      'primaryBorderColor': '#003087',
      'lineColor': '#003087',
      'secondaryColor': '#007396',
      'tertiaryColor': '#CCD1D1'
    },
   'flowchart': { 'curve': 'basis' }
  }
}%%

flowchart TD

pA(("passive\nacoustic\nmonitoring"))

A[("hydrophone")]

B{{"PassivePacker"}}

C{{"Darwin Core
Alignment"}}

D[("National Centers\nfor\nEnvironmental Information\n(NCEI)")]

E[("IPT
OBIS-USA")]

F[/"MBON 
Data Portal"\]

G([OBIS])

H([GBIF])

J([NOAA OneStop])

K([data.gov])

L(["Commerce
Data Hub"])

M[/"IOC-UNESCO Harmful Algae Information System"\]
N[/"Infographics"\]
R[/"NCEI Passive Acoustic Data Portal"\]

pA --> A
A -- species detections\n+\nsound propagation--> C
A -- raw\nand\nprocessed\ndata --> B
C --> E
E --> D
E --> G
E --> H
B --> D

D --> FC

G .-> EP
H .-> EP
D .-> EP

D --> R

FC .-> EP

subgraph EP [Example Products]
M
N
F
R
end

subgraph FC [Federal Catalogs]
J
K
L
end

MathewBiddle commented 9 months ago

further defining raw and processed data:

Audio, sound level metrics (beyond 1 minute hybrid millidecade, standard metric), detections, sound propagation models, and well organized sound clips

carriecwall commented 8 months ago

The revised version looks much better! And defining processed data is super helpful. Could we make the line from OBIS to NCEI a dashed line since that pathway is not fully defined yet?

MathewBiddle commented 8 months ago

Thanks for taking a look. The pipeline between OBIS-USA's IPT and NCEI is fully operational. See https://www.ncei.noaa.gov/news/archiving-ocean-biodiversity-information-system-usa-obis-usa. The archive package can be found at https://www.ncei.noaa.gov/metadata/geoportal/rest/metadata/item/gov.noaa.nodc%3A0250940/html.

For reference, the Sanctuary Soundscape Monitoring Project (SanctSound) Daily Aggregated Species Detections have been archived via this pathway, from the most recent quarterly pull, at https://www.nodc.noaa.gov/archive/arc0199/0250940/6.6/data/0-data/noaa_sanctsound_daily_species/

ioos / mbon-docs

Create a flow chart for passive acoustic monitoring data #41