OptimizeRasters is a set of tools for converting raster data to optimized Tiled TIF or MRF files, moving data to cloud storage, and creating Raster Proxies.
Apache License 2.0
127
stars
50
forks
source link
Hangs on "Converting" when run from script edit - don't call gdal #171
When I try to call OptimizeRasters from a script it hangs on "Converting". When I try from ArcGIS Pro Toolbox, it runs on the same folder correctly. I am on ArcGIS Pro 3.3.1
Any tricks on how to debug this?
#tool for creating RPAS products
#Requires ESRI OptimizeRasters
import sys
import os
from osgeo import gdal
import shutil
import arcpy
#LIST OF FILES TO OPTIMIZE
fList = [
r'C:\Users\will.mcinnes\Desktop\Scratch\OptimizeTesting\Products\Kiixin_AGISOFT_MOS_U1230.tif'
]
OptimizeRastersRoot = 'C:/Image_Mgmt_Workflows/OptimizeRasters' # default intallation location for OptimizeRasters.
sys.path.append(OptimizeRastersRoot) # Set the root folder where the OptimizeRasters module is found unless the client app is loaded from the same root.
try:
import OptimizeRasters # import OptimizeRasters/OR module.
except ImportError as e:
print ('Err. {}'.format(str(e)))
exit(1)
def messages(msg, status):
print ('**{} - {}'.format(msg, status))
return True
def callOptimize(inPath, outPath,f):
args = {
'config': 'C:/Image_Mgmt_Workflows/OptimizeRasters/Templates/Imagery_to_TIF_LZW.xml',
'output': outPath.replace("\\","/")+"/",
'input': inPath.replace("\\","/")+"/",
'subs': 'false', # Do we included subfolders?
'resume':'True',
'cloudupload': 'false',
'op': 'report',
'pyramids': 'true'
}
rpt = OptimizeRasters.Report(OptimizeRasters.Base())
writeToPath = OptimizeRastersRoot
print("writeToPath", writeToPath)
ORJobFile = os.path.join(writeToPath,"logs", '{}{}'.format(rpt.getUniqueFileName(), rpt.CJOB_EXT))
rpt.init(ORJobFile)
for key in args.keys():
rpt.addHeader(key, args[key]) # add necessary headers
rpt.addFile(f) # eg. c:/input/mydata/readme.txt
print('added ', f)
# please note, when adding files into the job file, it's important that all entries should have the same parent folder. In this case, it's (c:/input/mydata/)
rpt.write() # create the OR job/.orjob file.
print('rpt:', rpt)
args['input'] = ORJobFile # input now points to the newly created OptimizeRasters Job file.
app = OptimizeRasters.Application(args) # The args{} can contain any valid cmd-line argument name without the prefix '-'
app.registerMessageCallback(messages) # Optional. If messages need to be brought back onto the caller's side.
if (not app.init()):
return False
print('running')
try:
app.run() # Do processing..
rpt = app.getReport () # Get report/log status
except ImportError as e:
print ('Err. {}'.format(str(e)))
exit(1)
isSuccess = False
if (rpt and
not rpt.hasFailures()): # If log has no failures, consider the processing as successful.
isSuccess = True
print ('Results> {}'.format(str(isSuccess)))
return isSuccess
def ensure_path_exists(path):
if not os.path.exists(path):
os.mkdir(path)
print(f"Created directory: {path}")
else:
print(f"Directory already exists: {path}")
for f in fList:
prodDir = os.path.dirname(f)
print("(proddir(inPath): ", prodDir)
outPath = os.path.join(prodDir,"optimized").replace("\\","/")
ensure_path_exists(outPath)
print ("OutPath: ", outPath)
print ("f: ", f)
# RUN OPTIMIZATION!
callOptimize(prodDir, outPath,f)
Output:
PS C:\Users\will.mcinnes\Documents\hakai-gis-tools> & "c:/Program Files/ArcGIS/Pro/bin/Python/envs/arcgispro-py3/python.exe" c:/Users/will.mcinnes/Documents/hakai-gis-tools/rpas-tools/CreateRPASProduct.py
(proddir(inPath): C:\Users\will.mcinnes\Desktop\Scratch\OptimizeTesting\Products
Directory already exists: C:/Users/will.mcinnes/Desktop/Scratch/OptimizeTesting/Products/optimized
OutPath: C:/Users/will.mcinnes/Desktop/Scratch/OptimizeTesting/Products/optimized
f: C:\Users\will.mcinnes\Desktop\Scratch\OptimizeTesting\Products\Kiixin_AGISOFT_MOS_U1230.tif
writeToPath C:/Image_Mgmt_Workflows/OptimizeRasters
added C:\Users\will.mcinnes\Desktop\Scratch\OptimizeTesting\Products\Kiixin_AGISOFT_MOS_U1230.tif
rpt: <OptimizeRasters.Report object at 0x0000018A9B81DD90>
**OptimizeRasters.py v2.0.8/20231006 - 0
**Convert raster formats to a valid output format through GDAL_Translate.
Please Note:
OptimizeRasters.py is entirely case-sensitive, extensions/paths in the config file are case-sensitive and the program will fail if the correct path/case is not entered at the cmd-line or in the config file.
- 0
**ORJob> Reading/Preparing data. Please wait.. - 0
**ORJob> Done. - 0
Log-path set to (C:\Image_Mgmt_Workflows\OptimizeRasters\logs)
log-msg:version=v2.0.8/20231006
log-msg:-config=C:/Image_Mgmt_Workflows/OptimizeRasters/Templates/Imagery_to_TIF_LZW.xml -output=C:/Users/will.mcinnes/Desktop/Scratch/OptimizeTesting/Products/optimized/ -input=C:/Image_Mgmt_Workflows/OptimizeRasters\logs\OR_20240807T094202275517.orjob -subs=false -resume=True -cloudupload=false -op=report -pyramids=true
log-msg:Mode=tif
log-msg:RasterFormatFilter=['tif', 'tiff', 'TIF', 'TIFF', 'JPG', 'jpg', 'img', 'IMG', 'sid', 'SID']
log-msg:ExcludeFilter=['tmp', 'rrd', 'idx', 'lrc', 'mrf_cache', 'pjp', 'ppng', 'pft', 'pzp', 'ovr', 'aux.xml', 'aux', 'tfw', 'TFW', 'pjg']
log-msg:IncludeSubdirectories=true
log-msg:Compression=LZW
log-msg:Interleave=Pixel
log-msg:BuildPyramids=true
log-msg:PyramidSampling=average
log-msg:PyramidCompression=LZW
log-msg:BlockSize=512
log-msg:KeepExtension=false
log-msg:Threads=4
log-msg:CloudUpload=false
log-msg:Out_S3_DeleteAfterUpload=true
**Checking for updates.. - 0
**'NewVersion' - 0
running
**[Resume] Using job (C:/Image_Mgmt_Workflows/OptimizeRasters\logs\OR_20240807T094202275517.orjob) - 0
**Copying non rasters/aux files (C:/Users/will.mcinnes/Desktop/Scratch/OptimizeTesting/Products/=>C:/Users/will.mcinnes/Desktop/Scratch/OptimizeTesting/Products/optimized/).. - None
**Done. - None
**Converting.. - 0
**Converting (C:/Users/will.mcinnes/Desktop/Scratch/OptimizeTesting/Products/Kiixin_AGISOFT_MOS_U1230.tif).. - 0
When I try to call OptimizeRasters from a script it hangs on "Converting". When I try from ArcGIS Pro Toolbox, it runs on the same folder correctly. I am on ArcGIS Pro 3.3.1 Any tricks on how to debug this?
Output: