[bug] The fastavro dependency is unavailable on Windows

[BewareMyPower]

Currently the avro support of the Python client depends on the fastavro 0.24.0 dependency, which is unavailable on Windows.

> py -m pip install fastavro==0.24.0
...

  × Running setup.py install for fastavro did not run successfully.
  │ exit code: 1
  ╰─> [53 lines of output]
      running install
      C:\Users\xuyunze\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-310
      creating build\lib.win-amd64-cpython-310\fastavro
      copying fastavro\const.py -> build\lib.win-amd64-cpython-310\fastavro
      copying fastavro\json_read.py -> build\lib.win-amd64-cpython-310\fastavro
      copying fastavro\json_write.py -> build\lib.win-amd64-cpython-310\fastavro
      copying fastavro\logical_writers.py -> build\lib.win-amd64-cpython-310\fastavro
      copying fastavro\read.py -> build\lib.win-amd64-cpython-310\fastavro
      copying fastavro\schema.py -> build\lib.win-amd64-cpython-310\fastavro
      copying fastavro\six.py -> build\lib.win-amd64-cpython-310\fastavro
      copying fastavro\validation.py -> build\lib.win-amd64-cpython-310\fastavro
      copying fastavro\write.py -> build\lib.win-amd64-cpython-310\fastavro
      copying fastavro\_logical_writers_py.py -> build\lib.win-amd64-cpython-310\fastavro
      copying fastavro\_read_common.py -> build\lib.win-amd64-cpython-310\fastavro
      copying fastavro\_read_py.py -> build\lib.win-amd64-cpython-310\fastavro
      copying fastavro\_schema_common.py -> build\lib.win-amd64-cpython-310\fastavro
      copying fastavro\_schema_py.py -> build\lib.win-amd64-cpython-310\fastavro
      copying fastavro\_timezone.py -> build\lib.win-amd64-cpython-310\fastavro
      copying fastavro\_validate_common.py -> build\lib.win-amd64-cpython-310\fastavro
      copying fastavro\_validation_py.py -> build\lib.win-amd64-cpython-310\fastavro
      copying fastavro\_write_py.py -> build\lib.win-amd64-cpython-310\fastavro
      copying fastavro\__init__.py -> build\lib.win-amd64-cpython-310\fastavro
      copying fastavro\__main__.py -> build\lib.win-amd64-cpython-310\fastavro
      creating build\lib.win-amd64-cpython-310\fastavro\io
      copying fastavro\io\binary_decoder.py -> build\lib.win-amd64-cpython-310\fastavro\io
      copying fastavro\io\binary_encoder.py -> build\lib.win-amd64-cpython-310\fastavro\io
      copying fastavro\io\json_decoder.py -> build\lib.win-amd64-cpython-310\fastavro\io
      copying fastavro\io\json_encoder.py -> build\lib.win-amd64-cpython-310\fastavro\io
      copying fastavro\io\parser.py -> build\lib.win-amd64-cpython-310\fastavro\io
      copying fastavro\io\symbols.py -> build\lib.win-amd64-cpython-310\fastavro\io
      copying fastavro\io\__init__.py -> build\lib.win-amd64-cpython-310\fastavro\io
      running build_ext
      building 'fastavro._read' extension
      creating build\temp.win-amd64-cpython-310
      creating build\temp.win-amd64-cpython-310\Release
      creating build\temp.win-amd64-cpython-310\Release\fastavro
      "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\xuyunze\AppData\Local\Programs\Python\Python310\include -IC:\Users\xuyunze\AppData\Local\Programs\Python\Python310\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\cppwinrt" /Tcfastavro/_read.c /Fobuild\temp.win-amd64-cpython-310\Release\fastavro/_read.obj
      _read.c
      fastavro/_read.c(5832): warning C4146: unary minus operator applied to unsigned type, result still unsigned
      fastavro/_read.c(23105): warning C4013: '_PyGen_Send' undefined; assuming extern returning int
      fastavro/_read.c(23105): warning C4047: '=': 'PyObject *' differs in levels of indirection from 'int'
      fastavro/_read.c(23110): warning C4047: '=': 'PyObject *' differs in levels of indirection from 'int'
      fastavro/_read.c(23194): warning C4047: '=': 'PyObject *' differs in levels of indirection from 'int'
      "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\xuyunze\AppData\Local\Programs\Python\Python310\libs /LIBPATH:C:\Users\xuyunze\AppData\Local\Programs\Python\Python310 /LIBPATH:C:\Users\xuyunze\AppData\Local\Programs\Python\Python310\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\um\x64" /EXPORT:PyInit__read build\temp.win-amd64-cpython-310\Release\fastavro/_read.obj /OUT:build\lib.win-amd64-cpython-310\fastavro\_read.cp310-win_amd64.pyd /IMPLIB:build\temp.win-amd64-cpython-310\Release\fastavro\_read.cp310-win_amd64.lib
         Creating library build\temp.win-amd64-cpython-310\Release\fastavro\_read.cp310-win_amd64.lib and object build\temp.win-amd64-cpython-310\Release\fastavro\_read.cp310-win_amd64.exp
      _read.obj : error LNK2001: unresolved external symbol _PyGen_Send
      build\lib.win-amd64-cpython-310\fastavro\_read.cp310-win_amd64.pyd : fatal error LNK1120: 1 unresolved externals
      error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.29.30133\\bin\\HostX86\\x64\\link.exe' failed with exit code 1120
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  Rolling back uninstall of fastavro
  Moving to c:\users\xuyunze\appdata\local\programs\python\python310\lib\site-packages\fastavro-1.7.0.dist-info\
   from C:\Users\xuyunze\AppData\Local\Programs\Python\Python310\Lib\site-packages\~astavro-1.7.0.dist-info
  Moving to c:\users\xuyunze\appdata\local\programs\python\python310\lib\site-packages\fastavro\
   from C:\Users\xuyunze\AppData\Local\Programs\Python\Python310\Lib\site-packages\~astavro
  Moving to c:\users\xuyunze\appdata\local\programs\python\python310\scripts\fastavro.exe
   from C:\Users\xuyunze\AppData\Local\Temp\pip-uninstall-pkl1sik6\fastavro.exe
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> fastavro

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

[erichare]

@BewareMyPower Am i mistaken or are there windows wheels available now from here? https://pypi.org/project/fastavro/0.24.0/#files

[BewareMyPower]

@erichare It might be caused by my Python version is too high (3.10). You can see the wheels are only for Python 2.7, 3.5 to 3.8.

[erichare]

Yep, good point @BewareMyPower .

[BewareMyPower]

I tried the local fastavro dependency but it's incompatible. We need to solve the incompatibility first.

[ghost]

I tried the local fastavro dependency but it's incompatible. We need to solve the incompatibility first.

@BewareMyPower Does this mean there is no path to pip install 'pulsar-client[avro]' on Python greater than 3.8?

I'm getting a similar error on Python 3.10.9 on a macOS Monterey 12.6.

Is there a reason pulsar-client is dependent on such an ancient version of fastavro?

[BewareMyPower]

Is there a reason pulsar-client is dependent on such an ancient version of fastavro?

No. Even if yes, we can still modify the code to make it work.

[BewareMyPower]

I tried upgrading the fastavro to 1.7.3 and here is the workflow to build wheels: https://github.com/BewareMyPower/pulsar-client-python/actions/runs/4741336587

Let's see if the wheels will work well. Currently it's a bit complicated to verify the dependency upgrade locally.