Closed dbuscombe-usgs closed 2 years ago
makedatasets needs to be updated so it can store the image as an ND array
train model
needs to be updated so it can use the ND array (and detect automatically / generalize to any N)
seg_images
needs a utility pre-processing script to get multiple coincident bands into an input npz
function that
3 options
all of this made sense in the call (!) - evan is wishing for a solution that won't break his current workflow ... see below... this is more complicated than previously though, because of the a) need to pad disparate sized images, and b) the extreme limitations of keras' built0in image generator, that is not amenable to applying the same augmentations on multiple coincident and co-occuring images
the solution is a single function that combines all 3 elements listed above
having spent a couple of hours playing with ideas, i conclude The only sane way to deal with this is to have one "makedatasets" function that deals with all cases
currently, only 1 and 2 implemented and tested
However, I will implement a "make_nd_datasets" function first that will generalize all 4 cases. Case 1 users (above) can keep using 'make_datasets' for now
ideally, we move away from the keras imagedatagenerator routine entirely
makedatasets should eventually
I notice that all augmentation and image rescaling options used so far can introduce quite horrible interpolation artifacts for the discrete label images, even rescale
from skimage.io with zero-order interpolation, anti-aliasing, and clipping to original range because it first recodes to float .... grrr.
Labels dont ever need interpolation, only resizing through nearest neighbour (up-pad, then infilling, or a spatial replacement operation for downsizing). By definition, interpolation only functions on continuous variables stored as floats, never on discrete arrays
Labels will now therefore adopt this approach which does not create floats or use interpolation
def scale(im, nR, nC):
'''
for reszing 2d integer arrays
'''
nR0 = len(im) # source number of rows
nC0 = len(im[0]) # source number of columns
tmp = [[ im[int(nR0 * r / nR)][int(nC0 * c / nC)]
for c in range(nC)] for r in range(nR)]
return np.array(tmp).reshape((nR,nC))
lab2 = scale_label(lab,TARGET_SIZE[0],TARGET_SIZE[1])
modified from here
corresponding rgb images can be dealt with like this:
def scale_rgb(img, nR, nC, nD):
'''
for reszing 3d integer arrays
'''
imgout = np.zeros((nR, nC, nD))
for k in range(3):
im = img[:,:,k]
nR0 = len(im) # source number of rows
nC0 = len(im[0]) # source number of columns
tmp = [[ im[int(nR0 * r / nR)][int(nC0 * c / nC)]
for c in range(nC)] for r in range(nR)]
imgout[:,:,k] = np.array(tmp).reshape((nR,nC))
return imgout
also, padded images and labels will be stored as lossless pngs, instead of jpegs
imsave(..., check_contrast=False, compression=0)
before resize
after resize with no interpolation
"makedatasets ND":
new function make_nd_dataset,py
is for N-D imagery that is stored across multiple files, each of which are a separate band or combination of bands from the same sensor, or otherwise coincident and concurrent bands stored in different files (e.g. orthomosaics and digital elevation models). The program has been tested with a set of 3-band rgb images, a coincident set of 1-band NIR images, and a set of 1-band SWIR images, fused to form a 5-band raster stack
do_resize
will be set of False, and images will not be padded or resized, otherwise do_resize
will be True, and all images smaller than TARGET_SIZE will be padded and all images larger than TARGET_SIZE will be resized. Note that this version produces much better labels, due to the use of remapping rather than interpolation of 2d label arrays larger than TARGET_SIZEdo_resize
is True, lists of padded images and labels are compiled, otherwise lists of the original unpadded/unresized imagery are compiled (not yet tested the condition do_resize
is False)do_resize
is True, these will be padded/resized, otherwise the original sizedo_resize
is True, these will be padded/resized, otherwise the original sizenew function make_nd_dataset,py
is supposed to be generic and should eventually replace make_dataset.py
, now for the N_DATA_BANDS <= 3 situation only
some random augmented examples with REMAP_CLASSES
SOme ranodm non-augmented samples with REMAP_CLASSES
The next stage of this task is to create a utility that a user can merge (and optionally pad/resize) ND sample imagery for use in deployment/prediction mode
New function utils\merge_nd_inputs4pred.py
. It is essentially the same as make_nd_datasets
except:
Prompts the user for
root = Tk()
root.filename = filedialog.askdirectory(initialdir = os.getcwd(),title = "Select directory for storing OUTPUT files")
output_data_path = root.filename
print(output_data_path)
root.withdraw()
root = Tk()
root.filename = filedialog.askdirectory(initialdir = os.getcwd(),title = "Select FIRST directory of IMAGE files")
data_path = root.filename
print(data_path)
root.withdraw()
W=[]
W.append(data_path)
result = 'yes'
while result == 'yes':
result = messagebox.askquestion("More directories of images?", "More directories of images?", icon='warning')
if result == 'yes':
root = Tk()
root.filename = filedialog.askdirectory(initialdir = os.getcwd(),title = "Select directory of image files")
data_path = root.filename
print(data_path)
root.withdraw()
W.append(data_path)
## neeed resizing?
szs = [imread(f).shape for f in files[:,0]]
szs = np.vstack(szs)[:,0]
if len(np.unique(szs))>1:
do_resize=True
else:
do_resize=False
from tkinter import simpledialog
application_window = Tk()
TARGET_X = simpledialog.askinteger("Imagery are different sizes and will be resized.",
"What is the TARGET_SIZE (X) of the intended model?",
parent=application_window,
minvalue=32, maxvalue=8192)
TARGET_Y = simpledialog.askinteger("Imagery are different sizes and will be resized.",
"What is the TARGET_SIZE (Y) of the intended model?",
parent=application_window,
minvalue=32, maxvalue=8192)
TARGET_SIZE = [TARGET_X,TARGET_Y]
the same padding/resiizng commands are used as make_nd_dataset
. Padded imagery is created and put into folders
why is this important? Imagery of the same spatial extent may have different grid sizes. Take these two Landsat scenes, downloaded using CoastSat
This one is 906x792 pixels
and this is the same spatial extent, but 830 x 790 px
the same two images padded/resized
If I apply the same procedure to all bands, then I can stack all bands into coincident N-D rasters. Non-augmented imagery is simply stored in npz files for prediction
This is now implemented as utils\merge_nd_inputs4pred.py
Next, I need to modify seg_images_in_folder.py
and ensemble_seg_images_in_folder
to accept the new file formats when N_DATA_BANDS > 3
seg_images_in_folder.py
and ensemble_seg_images_in_folder
seg_images_in_folder.py
with 5D hatteras / solo and ensemble modelsmake_dataset
and make_nd_dataset
now, 3- and 4-band images are accommodated, but needs to be generalized to ND images, stored as nparrays or multiband tiffs