search

spectralpython / spectral

Python module for hyperspectral image processing
MIT License
573 stars 139 forks source link

Ignore comment lines in ENVI headers, and accept blank values. #7

Closed donm closed 10 years ago

donm commented 10 years ago

Fixes #6.

donm commented 10 years ago

I confirmed that ENVI will load a header file with empty values. If the header says description = then the description field is blank, and if wavelength units = then wavelength units are set to "Unknown". If the header says custom_field =, and if you read that file and apply the header to another file, then the new header that is written will not have the custom_field entry. But it also won't have the field if the origin header said custom_field = 100; that is, it skips custom entries either way.

This is probably bad behavior on ENVI's part, since you're allowed to add whatever keys you want to the header file, but the important thing is that loading (and saving) headers with empty values won't cause incompatibility issues.

The reason I made the change was because some data I was working with had an empty wavelength units = field straight from the sensor.

About the semicolon in the middle of a multi-line value question: I did interpret the docs (http://www.exelisvis.com/docs/ENVIHeaderFiles.html) as saying it could be in the middle of an array, since they explicitly say "anywhere within a header file". But I just tested to see what happens (I have ENVI 5.0) and it looks like the code doesn't match the docs. A line starting with a semicolon in a text field (e.g., description) will be included in the value, and a line starting with a semicolon in a numeric field (wavelength) will cause the first number on that line to be read as 0.

So I'll report this to VIS tomorrow. I'm sure they'll jump right on it, and either the docs or the code will be fixed for the 2019 release or something.

In the meantime, what are your thoughts? As is, the patch follows the docs. If you want identical parsing behavior in both programs then one of us could comment out line #98.

tboggs commented 10 years ago

If you're going to report it to them tomorrow, let's wait a day or so to see if you get a response. If it takes longer than that, I'm content with the patch the way it is now. I can't think of a realistic case where an array-valued parameter would have a value at the beginning of a line where the value actually starts with a semicolon. So commenting out such lines shouldn't be a problem.

If you get a response from Exelis later on, we can always update the code to match what they specify as the correct behavior.

donm commented 10 years ago

The tech support rep I emailed is going to report the issue. But she seemed to think that the documentation should be updated to indicate that comments are not allowed in the middle of key/value pair, rather than update the code to match the docs.

tboggs commented 10 years ago

I'm not too concerned about a semicolon inside a <key> = <value> statement. If it is a parameter that is expected to be a numeric value, then it will fail anyway. Did she have any feedback regarding a comment line in the middle of a multi-line array definition?

donm commented 10 years ago

She didn't. I'll try to keep an eye on the incident in their tracking system (I already have a few other bugs reported to them that I'm watching) and will let you know what the outcome is. But I'm sure this has, like, negative priority for them.

tboggs commented 10 years ago

I'll go ahead and merge this since either way, it is better than the current behavior. If we get an authoritative response from Exelis later, we can always update it to be consistent.