Closed Yazhou-Sun closed 3 years ago
The error tells me that the image does not have a "history" key in its metadata. A workaround might be to simply add a "history" tag to the metadata, even if it is blank or just has a dummy variable. Can you provide more of your code? What is the image type you have loaded into spatial_mod
? Can you share the contents of your metadata? I'll try to help you out if I can. Thanks!
The error tells me that the image does not have a "history" key in its metadata. A workaround might be to simply add a "history" tag to the metadata, even if it is blank or just has a dummy variable. Can you provide more of your code? What is the image type you have loaded into
spatial_mod
? Can you share the contents of your metadata? I'll try to help you out if I can. Thanks!
My code is about the same as section 6. Tutorial: spatial_mod: The metadata seems not defined. But I am not quite sure how to create the metadata.
import geopandas as gpd
import os
from hs_process import hsio
from hs_process import spatial_mod
fname_in = r'E:\hyperspectral_imagery1.hdr'
fname_gdf = r'E:\plotboundary.geojson'
gdf = gpd.read_file(fname_gdf)
io = hsio(fname_in)
my_spatial_mod = spatial_mod(io.spyfile)
dir_out = os.path.join(io.base_dir, 'spatial_mod', 'crop_many_gdf')
if not os.path.isdir(os.path.join(io.base_dir, 'spatial_mod')): # create a new folder named "spatial_mod" if it does not exist
os.mkdir(os.path.join(io.base_dir, 'spatial_mod'))
if not os.path.isdir(dir_out): # create a new folder named "crop_many_gdf" if it does not exist
os.mkdir(dir_out)
name_append = '-crop-many-gdf'
df_plots = my_spatial_mod.crop_many_gdf(spyfile=io.spyfile, gdf=gdf)
array_crop, metadata = my_spatial_mod.crop_single(plot_id=1)
Thanks for the information. Can you open up the .hdr file (E:\hyperspectral_imagery1.hdr
) with a text editor and look at the contents? Here is a sample of the image I used in the example/tutorial:
ENVI
samples = 3
lines = 3
bands = 240
header offset = 0
file type = ENVI Standard
data type = 4
interleave = bip
byte order = 0
reflectance scale factor = 1.0
map info = {UTM, 1.0, 1.0, 441371.00707299996, 4855941.7317699995, 0.04, 0.04, 15, T, WGS-84, units meters, rotation 0.000}
history = input cube -> Radiance Conversion<SpecGroup>[<SpecFilename label:'Imager Calibration' value:J:\\\\Shared Work\\\\Data\\\\PikaImagery2_Radiance\\\\Pika II 100106_11 Radiometric Calibration September 2016\\\\8mm\\\\PikaII_100106_11_8mm_RadiometricCalibration_Sept2016.icp>, <SpecCube label:'Dark Noise Cube' value:None>, <SpecBool label:'Auto Remove Dark Noise?' value:True>, <SpecBool label:'Return floating point?' value:False>] -> Georectify Airborne Datacube<SpecGroup>[[<SpecBool label:'Use Flat Earth Altitude?' value:False>, <SpecFloatText label:'Flat Earth Alt (m)' value:340>, <SpecFloatText label:'Field of View (deg)' value:33.0>, <SpecInfo label:'Calculated Resolution (m)' value:Not Available in Batch Mode>, <SpecFloatText label:'Map Resolution (m)' value:0.04>, <SpecBool label:'Select DEM File' value:False>], [<SpecFloatText label:'Sync Offset (s)' value:-0.05>, <SpecFloatText label:'Imager Roll Offset (deg)' value:0>, <SpecFloatText label:'Imager Pitch Offset (deg)' value:2.25>, <SpecFloatText label:'Imager Heading Offset (deg)' value:0>], [<SpecBool label:'Crop Cube to Start/End Lines?' value:False>, <SpecIntText label:'Start Line' value:0>, <SpecIntText label:'End Line' value:1906>], [<SpecBool label:'Select Folder to Save Output' value:False>, <SpecBool label:'Save Products in Source Folder?' value:False>, <SpecBool label:'Generate Full Datacube?' value:True>, <SpecBool label:'Generate KML of Render?' value:True>, <SpecBool label:'Generate GeoTIFF Render?' value:True>, <SpecBool label:'Generate Swath Outline?' value:True>, <SpecChoice label:'Interpolate Image?' value:nearest>, <SpecChoice label:'Interpolate Datacube?' value:linear>, <SpecChoice label:'Render To Use in KML/GeoTIFF' value:NA>, <SpecBool label:'Apply Stretch/Filter?' value:True>, <SpecBool label:'Save Interpolated LCF?' value:False>, <SpecBool label:'Save Mask of Interpolated Pixels?' value:False>]] -> Convert Radiance Cube to Reflectance from Measured Reference Spectrum<SpecGroup>[<SpecSpectrum label:'ROI Spec.' value:spec28>, <SpecBool label:'Measured reflectivity in Percentage?' value:False>, <SpecChoice label:'Scale 100% Reflectivity To:' value:1.0 (floats)>] -> hs_process.crop_single[<SpecPyFloatText label: 'pix_e_ul?' value:342; SpecPyFloatText label: 'pix_n_ul?' value:75; SpecPyFloatText label: 'pix_e_lr?' value:345; SpecPyFloatText label: 'pix_n_lr?' value:78>]
label = Wells_rep2_20180628_16h56m_test_pika_gige_7-Radiance Conversion-Georectify Airborne Datacube-Convert Radiance Cube to Reflectance from Measured Reference Spectrum.bip
wavelength = {394.6, 396.6528, 398.7056, 400.7584, 402.8112...}
band names = {1, 2, 3, 4, 5...}
Notice the "history" tag that describes the history this particular image has gone through to get where it is now. All of the images I work with have this history tag, but if yours do not, then that is probably the cause of the problem.
Thanks for the information. Can you open up the .hdr file (
E:\hyperspectral_imagery1.hdr
) with a text editor and look at the contents? Here is a sample of the image I used in the example/tutorial:ENVI samples = 3 lines = 3 bands = 240 header offset = 0 file type = ENVI Standard data type = 4 interleave = bip byte order = 0 reflectance scale factor = 1.0 map info = {UTM, 1.0, 1.0, 441371.00707299996, 4855941.7317699995, 0.04, 0.04, 15, T, WGS-84, units meters, rotation 0.000} history = input cube -> Radiance Conversion<SpecGroup>[<SpecFilename label:'Imager Calibration' value:J:\\\\Shared Work\\\\Data\\\\PikaImagery2_Radiance\\\\Pika II 100106_11 Radiometric Calibration September 2016\\\\8mm\\\\PikaII_100106_11_8mm_RadiometricCalibration_Sept2016.icp>, <SpecCube label:'Dark Noise Cube' value:None>, <SpecBool label:'Auto Remove Dark Noise?' value:True>, <SpecBool label:'Return floating point?' value:False>] -> Georectify Airborne Datacube<SpecGroup>[[<SpecBool label:'Use Flat Earth Altitude?' value:False>, <SpecFloatText label:'Flat Earth Alt (m)' value:340>, <SpecFloatText label:'Field of View (deg)' value:33.0>, <SpecInfo label:'Calculated Resolution (m)' value:Not Available in Batch Mode>, <SpecFloatText label:'Map Resolution (m)' value:0.04>, <SpecBool label:'Select DEM File' value:False>], [<SpecFloatText label:'Sync Offset (s)' value:-0.05>, <SpecFloatText label:'Imager Roll Offset (deg)' value:0>, <SpecFloatText label:'Imager Pitch Offset (deg)' value:2.25>, <SpecFloatText label:'Imager Heading Offset (deg)' value:0>], [<SpecBool label:'Crop Cube to Start/End Lines?' value:False>, <SpecIntText label:'Start Line' value:0>, <SpecIntText label:'End Line' value:1906>], [<SpecBool label:'Select Folder to Save Output' value:False>, <SpecBool label:'Save Products in Source Folder?' value:False>, <SpecBool label:'Generate Full Datacube?' value:True>, <SpecBool label:'Generate KML of Render?' value:True>, <SpecBool label:'Generate GeoTIFF Render?' value:True>, <SpecBool label:'Generate Swath Outline?' value:True>, <SpecChoice label:'Interpolate Image?' value:nearest>, <SpecChoice label:'Interpolate Datacube?' value:linear>, <SpecChoice label:'Render To Use in KML/GeoTIFF' value:NA>, <SpecBool label:'Apply Stretch/Filter?' value:True>, <SpecBool label:'Save Interpolated LCF?' value:False>, <SpecBool label:'Save Mask of Interpolated Pixels?' value:False>]] -> Convert Radiance Cube to Reflectance from Measured Reference Spectrum<SpecGroup>[<SpecSpectrum label:'ROI Spec.' value:spec28>, <SpecBool label:'Measured reflectivity in Percentage?' value:False>, <SpecChoice label:'Scale 100% Reflectivity To:' value:1.0 (floats)>] -> hs_process.crop_single[<SpecPyFloatText label: 'pix_e_ul?' value:342; SpecPyFloatText label: 'pix_n_ul?' value:75; SpecPyFloatText label: 'pix_e_lr?' value:345; SpecPyFloatText label: 'pix_n_lr?' value:78>] label = Wells_rep2_20180628_16h56m_test_pika_gige_7-Radiance Conversion-Georectify Airborne Datacube-Convert Radiance Cube to Reflectance from Measured Reference Spectrum.bip wavelength = {394.6, 396.6528, 398.7056, 400.7584, 402.8112...} band names = {1, 2, 3, 4, 5...}
Notice the "history" tag that describes the history this particular image has gone through to get where it is now. All of the images I work with have this history tag, but if yours do not, then that is probably the cause of the problem.
Yes, here it is:
ENVI
description = {HEADWALL Hyperspec III[HEADWALL Hyperspec III],[HEADWALL Hyperspec III OR]}
samples = 720
lines = 11182
bands = 274
header offset = 0
file type = ENVI Standard
data type = 12
interleave = bsq
sensor type = Unknown
byte order = 0
wavelength units = nm
map info = {Geographic Lat/Lon,1,1,-89.38681637930115,43.29648597737143,3.080735373867722e-7,2.250256219444636e-7,WGS84,units=Degrees}
wavelength = {
399.309
,401.521
,403.732
,405.944
... ...
,998.646
,1000.86
,1003.07
}
;Correct Position = 1
;Firmware = build: Oct 10 2019_09:59:10
;GSDO = 0.025
;HO = 307.0516052246094
;Invert Columns = 1
;Latitude Offset (m) = 0
;Longitude Offset (m) = 0
;Ortho DEM File = C:\Headwall\dem\usgs_ned_13_n44w090_gridfloat.hdr
;Ortho GPS Offset Altitude Offset(m) = 33
;Ortho GPS Offset Front Up Positive = 0
;Ortho GPS Offset Geoid Correction = 1
;Ortho GPS Offset North-East Positive = 1
;Ortho GPS Offset Pitch = -0.4
;Ortho GPS Offset Right Positive = 1
;Ortho GPS Offset Roll = 0
;Ortho GPS Offset Time Offset(ms) = 0
;Ortho GPS Offset Yaw = 3
;Ortho Lems EFL (mm) = 12
;Ortho Lens Pixel Pitch(um) = 7.4
;Ortho Lens Spatial Binning = 1
;Ortho Resolution (m) = 0.025
;Ortho Running Avg. of Altitude = 0
;Ortho Running Avg. of Latitude = 0
;Ortho Running Avg. of Longitude = 0
;Ortho Running Avg. of Pitch = 0
;Ortho Running Avg. of Raw = 0
;Ortho Running Avg. of Roll = 0
;Ortho Running Avg. of X = 0
;Ortho Running Avg. of Y = 0
;Ortho Running Avg. of Yaw = 0
;Ortho Running Avg. of Z = 0
;Ortho Sensor Alpha = 0
;Ortho Sensor Beta = 0
;Ortho Sensor Gamma = 0
;Ortho Sensor Invert Columns = 1
;Row binning = 1
;Serial Number = nHS-213
;averagePixelDisp = 2.21157549723685
;bBinaryMap = 000007FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFE
;framingCamera = 0
;gain = 3
;hsiii_version = Hyperspec v3.1.0
;invertColumns = 1
;left = 0
;pixel0 = 158.247496218906
;top = 88
As I suspected, it looks to be missing the "history" tag. For now until I am able to fix the code, can you manually add history = none
into the .hdr file?
I see all the extra metadata with a leading semicolon - I'm also not sure how hs_process
will behave with these metadata tags. It would be good to get a sample image from you to debug. I've only tested with Resonon imagery.
@Yazhou-Sun, would you mind uploading a single hyperspectral image to this cloud directory? I can use it as a test image to make sure the code all works with your images. Thanks!
@Yazhou-Sun, would you mind uploading a single hyperspectral image to this cloud directory? I can use it as a test image to make sure the code all works with your images. Thanks!
Thank you so much for your help! I don't have permission to upload files to this drive. would you mind adding me (ysun382@wisc.edu) as an editor or so?
@Yazhou-Sun, would you mind uploading a single hyperspectral image to this cloud directory? I can use it as a test image to make sure the code all works with your images. Thanks!
Thank you so much for your help! I don't have permission to upload files to this drive. would you mind adding me (ysun382@wisc.edu) as an editor or so?
Okay, give it a try now - sorry!
The fixes to this issue are merged into the dev branch. Give it a try and let me know how it goes with your Headwall imagery!
Hi, I am cropping UAV hyperspectral imagery with plot boundary files, there is one issue with this line of code:
Any idea how to solve the issue? Thanks.