BenthicSubstrateMapping / PyHum

Python code to read, display, export and analyse Humminbird sidescan sonar data
Other
68 stars 28 forks source link

PyHum.map_texture #11

Closed LZarri closed 8 years ago

LZarri commented 8 years ago

Error message while trying to get spatially referenced texture maps for PyHum on Windows 7 using Spyder IDE: "TypeError: coercing to Unicode: need string or buffer, NoneType found". Here is full error output:

Traceback (most recent call last):

File "", line 1, in runfile('C:/Users/user/CodingWorkingFiles/Tutorial/BuscombeFix00219.py', wdir='C:/Users/user/CodingWorkingFiles/Tutorial')

File "C:\Users\user\Anaconda2\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 699, in runfile execfile(filename, namespace)

File "C:\Users\user\Anaconda2\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 74, in execfile exec(compile(scripttext, filename, 'exec'), glob, loc)

File "C:/Users/user/CodingWorkingFiles/Tutorial/BuscombeFix00219.py", line 93, in dotest()

File "C:/Users/user/CodingWorkingFiles/Tutorial/BuscombeFix00219.py", line 77, in dotest PyHum.texture(humfile, sonpath, win, shift, doplot, density, numclasses, maxscale, notes)

File "C:\Users\user\Anaconda2\lib\site-packages\PyHum_pyhum_texture.py", line 389, in texture wc = get_kclass(class_fp[p].copy(), numclasses)

File "C:\Users\user\Anaconda2\lib\site-packages\PyHum_pyhum_texture.py", line 456, in get_kclass wc, values = humutils.cut_kmeans(Sk,numclasses+1)

File "C:\Users\user\Anaconda2\lib\site-packages\PyHum\utils.py", line 492, in cut_kmeans wc = da.from_array(wc, chunks=1000) #dask implementation

File "C:\Users\user\Anaconda2\lib\site-packages\dask\array\core.py", line 1428, in from_array name = name or 'from-array-' + tokenize(x, chunks)

File "C:\Users\user\Anaconda2\lib\site-packages\dask\base.py", line 240, in tokenize return md5(str(tuple(map(normalize_token, args))).encode()).hexdigest()

File "C:\Users\user\Anaconda2\lib\site-packages\dask\utils.py", line 479, in call return lkcls

File "C:\Users\user\Anaconda2\lib\site-packages\dask\base.py", line 205, in normalize_array return x.filename, os.path.getmtime(x.filename), x.dtype, x.shape

File "C:\Users\user\Anaconda2\lib\genericpath.py", line 62, in getmtime return os.stat(filename).st_mtime

TypeError: coercing to Unicode: need string or buffer, NoneType found

*

Here is full script to run PyHum:

-- coding: utf-8 --

""" """

import PyHum

def dotest():

humfile = 'E:\Sturgeon\2015Substrate\Humminbird\BeenUploaded\R00219.DAT' sonpath = 'E:\Sturgeon\2015Substrate\Humminbird\BeenUploaded\R00219'

doplot = 1 #yes

reading specific settings

cs2cs_args = "epsg:32100" #Montana bedpick = 1 # auto bed pick c = 1450 # speed of sound fresh water t = 0.108 # length of transducer draft = 0.3 # draft in metres flip_lr = 0 # flip port and starboard model = 998 # humminbird model cog = 1 # GPS course-over-ground used for heading calc_bearing = 1 #no filt_bearing = 0 #no chunk = 65 # distance, 65m

chunk = 'p1000' # pings, 1000

chunk = 'h10' # heading deviation, 10 deg

correction specific settings

maxW = 1000 # rms output wattage dofilt = 0 # 1 = apply a phase preserving filter (WARNING!! takes a very long time for large scans) correct_withwater = 0 # don't retain water column in radiometric correction (1 = retains water column for radiomatric corrections) ph = 7.0 # acidity on the pH scale temp = 10.0 # water temperature in degrees Celsius salinity = 0.0

for shadow removal

shadowmask = 0 #automatic shadow removal

for texture calcs

win = 100 # pixel window shift = 10 # pixel shift density = win/2 numclasses = 4 # number of discrete classes for contouring and k-means maxscale = 20 # Max scale as inverse fraction of data length (for wavelet analysis) notes = 4 # Notes per octave (for wavelet analysis)

for mapping

dogrid = 1 #yes res = 0.2 # grid resolution in metres

if res==99, the program will automatically calc res from the spatial res of the scans

mode = 1 # gridding mode (simple nearest neighbour)

mode = 2 # gridding mode (inverse distance weighted nearest neighbour)

mode = 3 # gridding mode (gaussian weighted nearest neighbour)

dowrite = 0 #disable writing of point cloud data to file

nn = 128 #number of nearest neighbours for gridding (used if mode > 1) influence = 1 #Radius of influence used in gridding. Cut off distance in meters numstdevs = 5 #Threshold number of standard deviations in sidescan intensity per grid cell up to which to accept

for downward-looking echosounder echogram (e1-e2) analysis

beam = 20.0 transfreq = 200.0 # frequency (kHz) of downward looking echosounder integ = 5 numclusters = 3 # number of acoustic classes to group observations

read data in SON files into PyHum memory mapped format (.dat)

PyHum.read(humfile, sonpath, cs2cs_args, c, draft, doplot, t, bedpick, flip_lr, model, calc_bearing, filt_bearing, cog, chunk)

correct scans and remove water column

PyHum.correct(humfile, sonpath, maxW, doplot, dofilt, correct_withwater, ph, temp, salinity)

remove acoustic shadows (caused by distal acoustic attenuation or sound hitting shallows or shoreline)

PyHum.rmshadows(humfile, sonpath, win, shadowmask, doplot)

Calculate texture lengthscale maps using the method of Buscombe et al. (2015)

PyHum.texture(humfile, sonpath, win, shift, doplot, density, numclasses, maxscale, notes)

grid and map the scans

PyHum.map(humfile, sonpath, cs2cs_args, res, dowrite, mode, nn, influence, numstdevs)

res = 0.5 # grid resolution in metres numstdevs = 5

grid and map the texture lengthscale maps

PyHum.map_texture(humfile, sonpath, cs2cs_args, dogrid, res, mode, nn, influence, numstdevs)

calculate and map the e1 and e2 acoustic coefficients from the downward-looking sonar

PyHum.e1e2(humfile, sonpath, cs2cs_args, ph, temp, salinity, beam, transfreq, integ, numclusters, doplot)

if name == 'main': dotest()

*

Thank you!

dbuscombe-usgs commented 8 years ago

It's a dask array issue that arises in a call by the texture module to a function within the utils module. I don't know what is causing it but I have implemented a workaround. Please install from github using

pip uninstall PyHum
pip install git+https://github.com/dbuscombe-usgs/PyHum.git;

and let me know if that doesn't fix it