XLSForm / pyxform

A Python package to create XForms for ODK Collect.
BSD 2-Clause "Simplified" License
82 stars 138 forks source link

return_code: 3221226505 with ODK_Validate.jar #737

Open khalifeserge opened 1 month ago

khalifeserge commented 1 month ago

Software and hardware versions

pyxform v2.2.0, Python 3.12.7

Problem description

On windows 11 with python 3.12 I tried pyxform 2.2.0 and 2.2.1 with Java 8 and Java 21 and got the same error where the validation would fail for a valid xlsform but does not produce any error message. I dug into the code a little and was able to find the source of the problem. In C:\Program Files\Python312\Lib\site-packages\pyxform\validators\util.py under function "run_popen_with_timeout" there the following call p = Popen( command, env=env, stdin=PIPE, stdout=PIPE, stderr=PIPE, startupinfo=startup_info ) which return a return_code: 3221226505 when the env variable and command variables are as follow:

env: {'TEMP': 'C:\Users\serge\AppData\Local\Temp', 'TMP': 'C:\Users\serge\AppData\Local\Temp', 'TMPDIR': 'C:\Users\serge\AppData\Local\Temp'} command: ['java', '-Djava.awt.headless=true', '-jar', 'C:\Program Files\Python312\Lib\site-packages\pyxform\validators\odk_validate\bin\ODK_Validate.jar', 'C:\Users\serge\AppData\Local\Temp\tmpila5ae4r']

If I call call p = Popen( command, env=None, stdin=PIPE, stdout=PIPE, stderr=PIPE, startupinfo=startup_info ) then the return code is 0 and everything works normally

If you need me to test other code to figure out why this is happening I can.

lindsay-stevens commented 1 month ago

Here are a few troubleshooting ideas:

It may be an issue with ODK Validate. It was last updated in pyxform v2.11 so try pyxform v2.10. If that doesn't work, let us know what pyxform version was in use before this problem started.

Environment variables are needed by the Popen call you mentioned, to ensure that Java correctly locates the user (rather than system) temp directory for writing files. Confirm that the directory C:\Users\serge\AppData\Local\Temp exists (or whichever directory is shown by python -c 'import tempfile; print(tempfile.gettempdir())').

It could be a Java issue, so try prefixing the PATH environment variable with the location of the intended Java version binaries folder, e.g. C:\Program Files (x86)\Java\jre1.8.0_71\bin. If that doesn't work, perhaps try reinstalling Java.

If none of the above addresses it, please provide a XLSForm (XLS or markdown) which can be used to reproduce the issue.

khalifeserge commented 4 weeks ago

I tried many version of ODK all the way back to 1.9 with the same results, I also checked the existence of temp files and all looks good. I had Java 8 installed when I first started having this issue, I then uninstalled it and installed Java 21 using winget. I ran

  1. java --version and got java 21.0.5 2024-10-15 LTS
  2. then I ran where javaand I gotC:\Program Files\Common Files\Oracle\Java\javapath\java.exewhich was not what I expected as java is installed inC:\Program Files\Java\jdk-21\bin\java.exe`

I followed your advice and change the command in \pyxform\validators\odk_validate__init__.py under the _call_validator function as follows:

  1. return run_popen_with_timeout( [r"C:\Program Files\Common Files\Oracle\Java\javapath\java.exe", "-Djava.awt.headless=true", "-jar", bin_file_path, path_to_xform], 100 ) which failed as before
  2. I then ran it with the JDK path return run_popen_with_timeout( [r"C:\Program Files\Java\jdk-21\bin\java.exe", "-Djava.awt.headless=true", "-jar", bin_file_path, path_to_xform], 100 ) which worked

I have to do some more research on why this is the case and why also removing the env from the call causes it to work as well but I think this issue can be closed unless or maybe some additional checks can be made to provide a better error message as in this particular case the output is as follow which is not helpful:

ODKValidateError during conversion.
Traceback (most recent call last):
  File "C:\Program Files\Python312\Lib\site-packages\pyxform\xls2xform.py", line 263, in main_cli
    warnings = xls2xform_convert(
               ^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python312\Lib\site-packages\pyxform\xls2xform.py", line 139, in xls2xform_convert
    result = convert(
             ^^^^^^^^
  File "C:\Program Files\Python312\Lib\site-packages\pyxform\xls2xform.py", line 113, in convert
    xform = survey.to_xml(
            ^^^^^^^^^^^^^^
  File "C:\Program Files\Python312\Lib\site-packages\pyxform\survey.py", line 1242, in to_xml
    xml = self.print_xform_to_file(
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python312\Lib\site-packages\pyxform\survey.py", line 1208, in print_xform_to_file
    warnings.extend(odk_validate.check_xform(path))
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python312\Lib\site-packages\pyxform\validators\odk_validate\__init__.py", line 94, in check_xform
    raise ODKValidateError(
pyxform.validators.odk_validate.ODKValidateError: ODK Validate Errors:
khalifeserge commented 4 weeks ago

The TLDR is if you are having ODK Validate Errors: message without any errors showing up it could be because the Java environment variable is set to C:\Program Files\Common Files\Oracle\Java\javapath instead of C:\Program Files\Java\jdk-XX\bin