search

holoviz-topics / examples

Visualization-focused examples of using HoloViz for specific topics
https://examples.holoviz.org
Creative Commons Attribution 4.0 International
82 stars 24 forks source link

script to get updated ship_traffic from MarineCadastre? #278

Open rsignell-usgs opened 1 year ago

rsignell-usgs commented 1 year ago

@jlstevens, I imagine there is a python script somewhere to download data from https://marinecadastre.gov/ais/ and convert to the proper format for ingesting into the ship_track demo notebook?

jlstevens commented 1 year ago

It's been a while but I am fairly sure the data was already preprocessed by a third party: I don't ever remember running a script and I seem to remember having the data available from the very start of the project (unfortunately, I don't remember where from exactly!)

That said, the data isn't particularly hard to understand so I'm sure it wouldn't be too hard to write such a script.

Is there any particular reason you are trying to learn about how that data was generated?

rsignell-usgs commented 1 year ago

I'm a visiting scientist here for a month at NATO CMRE in Italy, and they are interested in modernizing their tools for working with AIS data!

I was just asking about a script because I had not found the example 2020 CSV data that the notebook is supposed to load.

So getting a data download script would kill two birds with one stone: allow me to update from 2020 to 2022, and show me what the expected CSV files are supposed to look like!

rsignell-usgs commented 1 year ago

OMG, I've been here before: https://github.com/holoviz-topics/examples/issues/138. Let me take look at that thread and figure out the best course of action here.

jlstevens commented 1 year ago

Ah, and I see that issue mentions some of those details about where the data comes from that I had forgotten!

rsignell-usgs commented 1 year ago

Hmm, actually that thread seems just concerned with reading the already-existing parquet data files directly from cloud object storage rather than downloading them.

Still a mystery how the original data files were produced.

Which is a pity, since there is no data in those existing files from the Med. :(