search

intake / intake-stac

Intake interface to STAC data catalogs
https://intake-stac.readthedocs.io/en/latest/
BSD 2-Clause "Simplified" License
110 stars 25 forks source link

added csv to driver list #106

Open kthyng opened 3 years ago

kthyng commented 3 years ago

I have run this successfully in my local version but I am not sure how to best demonstrate that here. Could I get some help with this process and what you'd like to see?

kthyng commented 3 years ago

Since I am going through this process anyway, I thought I'd add in opendap too. Might come in handy.

scottyhq commented 3 years ago

thanks @kthyng!

If you could please add a small test that demonstrates opening the csv from a STAC item here that would be great https://github.com/intake/intake-stac/blob/e889999d2a3efac835676211ea387ea20eb58a9e/intake_stac/tests/test_catalog.py#L148

You can add an 'item_csv.json' in this folder https://github.com/intake/intake-stac/tree/main/intake_stac/tests/data/1.0.0 with an accompanying data file.

kthyng commented 3 years ago

@scottyhq I added a sample csv file and an accompanying item json file. I tried to set up a test (line 240 in test_catalog.py) but I couldn't guess enough about how it worked to figure it out. Can you provide any guidance on setting up the test?

scottyhq commented 3 years ago

Apologies for leaving this so long @kthyng! I think we can get it merged soon, just need to resolve the conflicts (click 'Resolve Conflicts' button below and just delete the <<< === >>> lines b/c we want to keep both your new test and the recent changes). For the test, here is an example to confirm the correct driver is used and we get back a pandas: dataframe

cat = intake.open_stac_item('item-csv.json')
df = cat['test'].read()

assert cat['test'].metadata['type'] == 'text/csv'
assert isinstance(cat['test'], intake.source.csv.CSVSource)
assert isinstance(df, pandas.dataframe.core.DataFrame)

you'll likely have to modify the path to item-csv.json, see other examples at the top of the test script https://github.com/intake/intake-stac/blob/fe1aaa1bc92f35e088d7fa9103195591cade06bf/intake_stac/tests/test_catalog.py#L20-L23

kthyng commented 3 years ago

Thanks @scottyhq! I should be able to get back to this later next week if not before. I appreciate your help!