search

GlobalFishingWatch / treniformis

Apache License 2.0
5 stars 6 forks source link

Importing monthly FFA lists #11

Open Bjorn-skytruth opened 8 years ago

Bjorn-skytruth commented 8 years ago

@geowurster I am importing monthly FFA lists into BigQuery. These will be one of the sources used for known fishing vessels. They should also give useful vessel IDs if we get data from the SPC.

Lists are downloaded here, https://www.ffa.int/node/42/. Generally they have been updated every week but I am just taking the last list in each month.

One of the list can be seen here. I had to modify the original excel sheet to bring in some information we want to include (geartype, length, tonnage) into the tab with the mmsi. This first tab will be imported directly into BigQuery. I then plan to aggregate the vessel list and add standardized column names according to our schema.

bitsofbits commented 7 years ago

@davidkroodsma , is this issue superseded by the list work you've been doing? Can I close it?

davidkroodsma commented 7 years ago

Actually, interesting you just found this. I'm trying to get Juan to work on this today.

Bjorn-skytruth commented 7 years ago

@bitsofbits @davidkroodsma Monthly FFA lists were added here world-fishing-827:FFA_monthly_lists with the idea this this could eventually be consolidated to give FFA authorization dates to put in GFW and also make a lot of other detailed registry information available. Data is pulled from monthly lists uploaded to Google sheets like this. Some additional fields like engine power are available but were not imported into BigQuery.

I still think that having this monthly FFA information available and consolidated into a single table would be useful but this requires more work.