search

CIRDLES / Squid

Squid3 is being developed by the Cyber Infrastructure Research and Development Lab for the Earth Sciences (CIRDLES.org) at the College of Charleston, Charleston, SC and Geoscience Australia as a re-implementation in Java of Ken Ludwig's Squid 2.5. - please contribute your expertise!
http://cirdles.org/projects/squid/
Apache License 2.0
12 stars 24 forks source link

Data Audit is finicky but I don't know why (i thought it was Java, but its not) #598

Closed NicoleRayner closed 3 years ago

NicoleRayner commented 3 years ago

Squid3 Version: 1.7.4 Java Version: 1.8.0_261 OS: Windows 10 10.0

Issue details: Using the Java version listed above (_261), I was having problems getting Data Audit to work for some .xml files (but not all, seemingly just ones that had SBM problems but my testing was not exhaustive here). I could process the data fully, I just could not get Data Audit to appear (nothing would happen. I waited a couple of minutes, normally this is virtually instantaneous).

This was driving me crazy because I KNOW I had processed these same files before. It occurred to me that I was not working on my usual computer, so I tried a different machine. That computer was able to Data Audit the same file with no problem and I noted that it had Java 1.8.0_281. I've since updated Java on the other machine, but the problem persists. Both are Windows 10 machines (one is Win 10 Pro, the other Win 10 Enterprise - it works on Enterprise normally. I haven't found another Win10 Pro machine to test with).

Attached is the file that was giving me grief

983_1_2020_Dec_14_15.33.zip

bowring commented 3 years ago

I am able to open your file and view the audit. Can you supply more info about the operating environment? Which flavor of Java are you using Oracle or Bell? Are you using JDK or JRE?

NicoleRayner commented 3 years ago

Using Oracle. Just the JDK/JRE distinction apply to Oracle? Not sure how to get that information. Java is pushed by our organization so I'm not sure its possible that different machines have fundamentally different flavours (updates could get out of sync but that's about it)

image

On Win10 Enterprise machines it works fine, my machine in Win10 Pro but I don't know if that would be the culprit.

bowring commented 3 years ago

What is the exact symptom - what shows when you attempt audit from menu?

NicoleRayner commented 3 years ago

It's so weird because absolutely NOTHING happens, it just hangs. Whatever screen I am on remains, but if I select something else (from ANOTHER menu other than Data, e.g. Project/Manage or Task) it proceeds normally.

bowring commented 3 years ago

Does that include the expression manager works normally? I am trying to figure out if this is related to JavaFX, the GUI part of Java.

NicoleRayner commented 3 years ago

Yes - expression manager works normally.

bowring commented 3 years ago

This may be a memory issue - try it on a smaller file. The example you sent was 8.7 MB.

bowring commented 3 years ago

also, run Squid3 in -verbose mode from the command line to get error messages if any:

java -jar Squid3JarFile -v

NicoleRayner commented 3 years ago

Looks like it may be a memory thing. Under Data - Manage Spots, I deleted the last 155 analyses then saved as a new Prawn Data file that was 1.3MB. Loaded up the new slimmed down one and can Data Audit without trouble.

I did another test with a different file that was also giving me grief, a file size somewhere between 5.4 and 5.0 MB will process properly on my laptop.

bowring commented 3 years ago

Do you have a suggestion for how we could show a "birds-eye" audit if memory issue is detected and then provide for viewing one row at a time, for example, in more detail?

NicoleRayner commented 3 years ago

To my mind, the single most important/valuable thing to audit is Primary Beam (or SBM if we institute that as I suggested in response to @sbodorkos issue #597). If that is what you mean by "birds-eye", aka something greatly simplified shown when a memory issue is detected, that would be my pick. Subsequent to that, the one row (aka species) at a time would work, or maybe 2 at a time, so that you can do the "delta" comparison.