ioos / compliance-checker-web

Web-based front-end for the IOOS Compliance Checker project
https://compliance.ioos.us
MIT License
5 stars 10 forks source link

Valid NetCDF files fail to upload with capital file extension #113

Open daltonkell opened 4 years ago

daltonkell commented 4 years ago

Attempting to upload Stoch_Images.NC results in the following error:

image

The file is, in fact, an HDF5 file:

$ file Stoch_Images.NC 
Stoch_Images.NC: Hierarchical Data Format (version 5) data

The Compliance Checker CLI has no trouble checking the file.

This error is likely due to checking files purely based on extensions, where .NC (all-caps) is not allowed.

Bobfrat commented 4 years ago

you can test that theory by making the extension lower case right?

daltonkell commented 4 years ago

Yeah, I believe it should be an easy fix! There's also the possibility to add in some more robust checking, e.g. looking at the file header.

Bobfrat commented 4 years ago

so it does work if you rename the file?

daltonkell commented 4 years ago

It does work, yeah

benjwadams commented 4 years ago

You might want to check the first few bytes of the file (aka "magic" in Unix) instead of or in addition to checking the file extension.

For example:

[badams@localhost]~/devel/glider-dac% file ~/Downloads/UW157-20141116T2118_f7fa_a808_6f48.nc
/home/badams/Downloads/UW157-20141116T2118_f7fa_a808_6f48.nc: NetCDF Data Format data
[badams@localhost]~/devel/glider-dac% head -c20 ~/Downloads/UW157-20141116T2118_f7fa_a808_6f48.nc
CDF
daltonkell commented 4 years ago

@benjwadams this is exactly what I had in mind!