cant open grid-to-point-converter

[danhamill]

I cant open grid-to-point-converter using the .bat or .exe file in vortex 0.10.20.

When I try and open the program with the .bat file two .dll files appear in the bin directory. Highlighted below:

the command prompt shows:

C:\workspace\git_clones\vortex-0.10.20\bin>SET "PATH=.\gdal;C:\Program Files\Common Files\Oracle\Java\javapath;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program Files\Tumbleweed\Desktop Validator\;C:\Program Files\Tumbleweed\Desktop Validator\x86;C:\WINDOWS\System32\OpenSSH\;C:\Program Files (x86)\HID Global\ActivClient\;C:\Program Files\HID Global\ActivClient\;C:\WINDOWS\CCM;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\drivers;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\rtools40\usr\bin;C:\gradle\gradle-6.7.1\bin;C:\Program Files\R\R-4.0.3\bin;C:\Program Files\Git\cmd;C:\Users\RDCRLDDH\AppData\Local\Microsoft\WindowsApps;C:\Users\RDCRLDDH\AppData\Local\atom\bin;C:\Users\RDCRLDDH\AppData\Local\Programs\Microsoft VS Code\bin"

C:\workspace\git_clones\vortex-0.10.20\bin>SET "GDAL_DATA=.\gdal\gdal-data"

C:\workspace\git_clones\vortex-0.10.20\bin>SET "PROJ_LIB=.\gdal\projlib"

C:\workspace\git_clones\vortex-0.10.20\bin>"..\jre\bin\java.exe" --module-path "..\jmods" --add-modules javafx.controls,javafx.fxml -Djavafx.cachedir=. -Djava.library.path=".;.\gdal" -cp ..\lib\grid-to-point-converter.jar;..\lib\* converter.ConverterWizard

[tombrauer]

The two dlls are extracted by the UI library JavaFX. No concerns there.

Are you on a USACE ACE-IT machine? There's a chance antivirus could be interfering with the process. Do you have a VM or personal machine you can test on?

[danhamill]

I am on a RDE machine, which I have admin rights on. I see this warning when I try and launch the bat file:

I click run and the terminal pops up, but eventually hangs up,

If I build from the source, I see:

C:\workspace\git_clones\Vortex>gradlew grid-to-point-converter:run

> Configure project :
Inferred project: vortex, version: 0.11.0-dev.0+fa48d61

> Task :grid-to-point-converter:run
Native library load failed.
java.lang.UnsatisfiedLinkError: org.gdal.gdal.gdalJNI.AllRegister()V
java.lang.UnsatisfiedLinkError: no gdalalljni in java.library.path: [C:\workspace\git_clones\Vortex/bin, C:\workspace\git_clones\Vortex/bin/gdal]
<===========--> 88% EXECUTING [10s]
> IDLE
> :grid-to-point-converter:run

[tombrauer]

I just tested launching on my ACE-IT and dev machines and didn't have any trouble. I'd check anti-virus logs if you have them to make sure you're not getting interference.

For running from source, try running gradlew refreshNatives. That should download natives to a bin directory in the project root. I can see from your log gdalalljni is looking for a native link in Vortex/bin.

[danhamill]

Any idea where to find those? 

I tested on another computer and it worked.

I really don’t understand why I can run some vortex utilities and this one fails.  I tried and old version 0.10.10 and it wouldn’t run either.  I know I used grid to point in 0.10.0 before. 

[danhamill]

I am trying to come up with a scripting workaround to my computer issues, but I am getting the following error:

Traceback (most recent call last):
  File "C:\workspace\git_clones\vortex_scripting\UA_to_DSS_IMPORT\grid_to_point_Converter.py", line 22, in <module>
    myImport = GridToPointConverter.builder()\
TypeError: pathToFeatures(): 1st arg can't be coerced to java.nio.file.Path

From this jython script:

from mil.army.usace.hec.vortex.convert import GridToPointConverter
from mil.army.usace.hec.vortex import Options
from mil.army.usace.hec.vortex.io import DataReader
import os
from glob import glob

d_files = glob(r"G:\UA\*_noData.dss")
output_dss = r"G:\UA\ts\UA_SWE_Depth_MoRiverBasin.dss"
clip_shp = r"C:\workspace\Mo River\shp\MissouriRiverBasin_alb.shp"

for dss_file in d_files:

    sourceGrids = DataReader.getVariables(dss_file)
    basin = 'MISSOURI RIVER BASIN'
    ds = 'UA_sanitized'

    write_options = Options.create()
    write_options.add('partF', ds )
    write_options.add('partA', 'SHG')
    write_options.add('partB', basin)
    myImport = GridToPointConverter.builder()\
            .pathToGrids(dss_file)\
            .pathToFeatures(clip_shp)\
            .field('NAME')\
            .destination(output_dss)\
            .writeOptions(write_options).build()
    myImport.process() 

Is there something special I need to do to get it to read the shapefile path?

[tombrauer]

On my machine the logs are most commonly found in C:\ProgramData\McAfee\Host Intrusion Prevention. I usually have an IT person help walk me through them. Based on your description it sounds like McAfee is interfering with the process and shutting down native libraries. I'd have an IT person try to change your policies so that you can run the program without interference.

As a work-around, you could try using the 0.9.* series. They used an older version of the JavaFX library.

[tombrauer]

It looks like pathToFeatures is expecting a java.nio.file.Path and you are supplying a String. You'll need to wrap the String in a path, e.g. clip_shp = Paths.get("C:\workspace\Mo River\shp\MissouriRiverBasin_alb.shp"). You probably need to add an import statement for Path e.g. from java.nio.file import Path. You may have to add one for Paths as well, e.g. from java.nio.file import Paths.

[danhamill]

Progress!! After wraping the clip_shp and output_dss file paths, I am now seeing:

Traceback (most recent call last):
  File "C:\workspace\git_clones\vortex_scripting\UA_to_DSS_IMPORT\grid_to_point_Converter.py", line 30, in <module>
    myImport.process()
AttributeError: 'mil.army.usace.hec.vortex.convert.GridToPointConve' object has no attribute 'process'

If I change it to .convert(), I get the following traceback:

Traceback (most recent call last):
  File "C:\workspace\git_clones\vortex_scripting\UA_to_DSS_IMPORT\grid_to_point_Converter.py", line 30, in <module>
    myImport.convert()
        at mil.army.usace.hec.vortex.convert.GridToPointConverter.convert(GridToPointConverter.java:94)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
java.lang.NullPointerException: java.lang.NullPointerException

[tombrauer]

Ah, I think I see the issue. GridToPointConverter is expecting a set of variables, i.e. DSS pathnames, as an argument as well. If you're looking to convert all records in a HEC-DSS file, you'd need to use some Jython HEC-DSS commands to get the list of records from an HEC-DSS file so that it could then be passed to the GridToPointConverter. I'm sure it can be done; I'm not sure of the commands off hand. Seeing what you are trying to do I'll try to make the GridToPointConverter a little bit easier to script against.

[danhamill]

Can we use DataReader? I found that from an example you created

from mil.army.usace.hec.vortex.io import DataReader

[tombrauer]

Yes! That's what you will need to add.

[tombrauer]

Once you get this script working, do you mind creating a grid-to-point-converter example similar to the others? I'll see if I can get you access to the page.

[danhamill]

Sure!

[tombrauer]

I added you to the repo. I hope that does the trick.

[danhamill]

I got the invite. Do we want to put them in examples or on the wiki?

[tombrauer]

Start with the wiki, if you want to add in the examples, that's cool too. Thanks!

[danhamill]

If I find anything useful about why grid-to-point-converter won't run on my desktop, I'll post it here.

Otherwise, I added a sanitizer and grid-to-point-converter scripts to examples and the wiki.

Thanks for your help!