search

att-vault / vault

ATT Vault Tech Scenario
2 stars 1 forks source link

Rationalize data access #15

Closed jbednar closed 3 years ago

jbednar commented 3 years ago

Following #12 , made all the Viewing and Hit notebooks access data in the same place, ./data in the same directory as the notebooks. For those notebooks, .data should contain at least the following files:

./index_active/
./Cleaned_AIS/ais_2009.h5
./Cleaned_AIS/ais_2013.h5
./Cleaned_AIS/ais_2017.h5
./Cleaned_AIS/ais_2016.h5
./Cleaned_AIS/Zone10_2014_01
./Cleaned_AIS/Zone10_2014_01/Vessel.csv
./Cleaned_AIS/Zone10_2014_01/Broadcast.csv
./Cleaned_AIS/ais_2012.h5
./Cleaned_AIS/ais_2015.h5
./Cleaned_AIS/ais_2011.h5
./Cleaned_AIS/ais_2010.h5
./Cleaned_AIS/ais_2014.h5
./Indexed_TLE/precomp2.h5
./AIS/AIS_2017_01_Zone01.csv
./AIS/AIS_2017_01_Zone03.csv
./AIS/AIS_2017_01_Zone02.csv
./Cleaned_TLE/tle2017.csv

This set of data and its organization match /data on ec2-3-22-166-135.us-east-2.compute.amazonaws.com, which already had all of these files in those locations apart from needing to unzip AIS/AIS_2017*.zip (which I did).

Some other notebooks were also updated to follow this structure, where the files existed on that ec2 instance. Others refer to data on local hard drives or on another EC2 instance, and those notebooks have not yet been updated.