search

CSTARS / ckanext-ecosis

Ckan extension adding additional fields and controls for spectral data
0 stars 0 forks source link

Unparssed sheets show up in data #59

Open aidanmazur opened 7 years ago

aidanmazur commented 7 years ago

When there are resource files that are uploaded that I mark as "Ignore: this sheet should not be parsed", they are still incorporated into the data (are seen on data previewer on importer and on the search side data viewer when user is scrolling through data). This occurs when the resource file is an excel, have not had it occur with any other format of resource file. I have found that deleting the resource file and re uploading solves the problem most of the time. This has occurred with multiple sheet resource files (with some sheets i mark as ignore) and with stand alone resource files that i want to ignore. Has happened on both dev and production

jrmerz commented 7 years ago

@aidanmazur can you point me at an example dataset on dev? I am having trouble recreating bug locally.

aidanmazur commented 7 years ago

Now I cannot recreate it either. The datasets that it had happened on was the "Leaf Optical Properties Experiment Database (LOPEX93)" on production and "Time series test" dataset on dev, but I already deleted and reuploaded the resource files on both and it self corrected. I will keep trying and let you know when it pops up again. You can close if you want and I'll reopen if I can recreate the issue

jrmerz commented 7 years ago

Let's leave it open for a bit, then can close if it doesn't popup again in the next couple weeks.

aidanmazur commented 7 years ago

Also sent in email, but this is now occurring on dataset "Spectral Characterization of Microtopographic Patterns in an Arctic Baydjarakh Field: Time Series" (http://data.ecosis.org/dataset/spectral-characterization-of-microtopographic-patterns-in-an-arctic-baydjarakh-field--time-series) on production. When looking at the data previewer, it says that the sheets "Run5RawRefl" and "Run5_Indices" are to be ignored, but they are showing up on the Spectra Preview under the publish tab on importer, and on data viewer on search (https://ecosis.org/#result/453d7970-b2b6-4dfe-b7f6-9f8336810dd5)

aidanmazur commented 7 years ago

Occurring on Dataset "Leaf Optical Properties Experiment Database (LOPEX93)" (http://data.ecosis.org/dataset/leaf-optical-properties-experiment-database--lopex93-) as well

jrmerz commented 7 years ago

Which resource and sheet?

aidanmazur commented 7 years ago

For Lopex dataset: Resource file "Lopex1993_Metadata_Naming_and_Notes.xlsx" is supposed to be ignored

For "Spectral Characterization of Microtopographic Patterns in an Arctic Baydjarakh Field: Time Series": resource file "NDVI&Albedo_Run5_Normalized_ForTest.xlsx" sheets 1 (Run5RawRef) and 3 (Run5_Indices)

jrmerz commented 7 years ago

Ok, I have an idea of what this might be. Thanks

jrmerz commented 7 years ago

Ok, I might need to jump on a screen share to watch you do this, might be a browser bug. I toggled the ignore button off/on and the sheets spectra were actually ignored. So I'm not totally sure on the source of this issue.

aidanmazur commented 7 years ago

No longer having the issue on the "Spectral Characterization of Microtopographic Patterns in an Arctic Baydjarakh Field: Time Series" dataset (which has the resource files formatted as one excel file, multiple sheets).

Still having the issue on the "Leaf Optical Properties Experiment Database (LOPEX93)" dataset, but only on search, not importer (which has the resource files formatted as multiple, separate excel files)