google / jsonnet

Jsonnet - The data templating language
http://jsonnet.org
Apache License 2.0
6.98k stars 440 forks source link

Change Python module build (setup.py / setuptools config) to make it more widely compatible #1137

Closed johnbartholomew closed 7 months ago

johnbartholomew commented 8 months ago

This moves us toward a Windows & MSVC-compatible Python module build, and should obsolete #784, and move toward a resolution for #734.

Things not in this PR:

Is it tested? Kinda. I don't have a Windows machine to test on, so I'm relying on the GitHub workflow to build & run the Python test.

johnbartholomew commented 8 months ago

Adjusted the workflow configuration to give cibuildwheel a test command to run - this just runs the _jsonnet_test.py script. Currently it's failing for the Windows wheel build. Looks like a problem with relative import paths (perhaps the same or partially the same problem as #867, though that particular report is a little unclear).

Unfortunately it seems all the path handling code only expects / as a directory separator.

   E...EEEE..
  ======================================================================
  ERROR: test_double_import (__main__.JsonnetTests)
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "D:\a\jsonnet\jsonnet\python\_jsonnet_test.py", line 172, in test_double_import
      json_str = _jsonnet.evaluate_snippet(
  RuntimeError: RUNTIME ERROR: couldn't open import "trivial.jsonnet": File not found
    D:\a\jsonnet\jsonnet\python\testdata\test.jsonnet:1:11-35   thunk <x>
    D:\a\jsonnet\jsonnet\python\testdata\test.jsonnet:3:1   

  ======================================================================
  ERROR: test_import_binary_encode (__main__.JsonnetTests)
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "D:\a\jsonnet\jsonnet\python\_jsonnet_test.py", line 136, in test_import_binary_encode
      json_str = _jsonnet.evaluate_snippet(
  RuntimeError: RUNTIME ERROR: couldn't open import "binary123.bin": File not found
    D:\a\jsonnet\jsonnet\python\testdata\test.jsonnet:1:1-26    

  ======================================================================
  ERROR: test_import_binary_sentinel_encode (__main__.JsonnetTests)
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "D:\a\jsonnet\jsonnet\python\_jsonnet_test.py", line 145, in test_import_binary_sentinel_encode
      json_str = _jsonnet.evaluate_snippet(
  RuntimeError: RUNTIME ERROR: couldn't open import "binary1230123.bin": File not found
    D:\a\jsonnet\jsonnet\python\testdata\test.jsonnet:1:1-30    

  ======================================================================
  ERROR: test_import_encode (__main__.JsonnetTests)
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "D:\a\jsonnet\jsonnet\python\_jsonnet_test.py", line 118, in test_import_encode
      json_str = _jsonnet.evaluate_snippet(
  RuntimeError: RUNTIME ERROR: couldn't open import "trivial.jsonnet": File not found
    D:\a\jsonnet\jsonnet\python\testdata\test.jsonnet:1:1-25    

  ======================================================================
  ERROR: test_import_no_eol_encode (__main__.JsonnetTests)
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "D:\a\jsonnet\jsonnet\python\_jsonnet_test.py", line 127, in test_import_no_eol_encode
      json_str = _jsonnet.evaluate_snippet(
  RuntimeError: RUNTIME ERROR: couldn't open import "trivial_no_eol.jsonnet": File not found
    D:\a\jsonnet\jsonnet\python\testdata\test.jsonnet:1:1-32    

  ----------------------------------------------------------------------
  Ran 10 tests in 0.625s

  FAILED (errors=5)
Error: Command python D:\a\jsonnet\jsonnet/python/_jsonnet_test.py

(Build action log: https://github.com/johnbartholomew/jsonnet/actions/runs/8087521792/job/22099656631)

johnbartholomew commented 8 months ago

I did the naive thing and just added an #if switch on _WIN32 to additionally look for backslash as a directory separator. We can consider switching to the C++17 lib, but I'd prefer to start with a smaller change like this that should be no worse than what we have right now.

In GitHub actions the Python test suite now passes on Windows: https://github.com/johnbartholomew/jsonnet/actions/runs/8098334188/job/22131487672

  + pip install 'C:\Users\runneradmin\AppData\Local\Temp\cibw-run-e8_jxk3n\cp312-win_amd64\repaired_wheel\jsonnet-0.20.0-cp312-cp312-win_amd64.whl'
  Processing c:\users\runneradmin\appdata\local\temp\cibw-run-e8_jxk3n\cp312-win_amd64\repaired_wheel\jsonnet-0.20.0-cp312-cp312-win_amd64.whl
  Installing collected packages: jsonnet
  Successfully installed jsonnet-0.20.0
  + python D:\a\jsonnet\jsonnet/python/_jsonnet_test.py
  ..........
  ----------------------------------------------------------------------
  Ran 10 tests in 0.505s
  OK