reloc/build_reloc.sh: `AttributeError: cython_sources` when building PyYAML

[tchaikov]

$ cat /etc/fedora-release 
Fedora release 40 (Forty)
$ cython --version
Cython version 3.0.9
$ reloc/build_reloc.sh --version scylla-5.5-1234 --nodeps         
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
poetry 1.7.1 requires build<2.0.0,>=1.0.3, but you have build 0.10.0 which is incompatible.      
* Creating virtualenv isolated environment...
* Installing packages in isolated environment... (Cython, setuptools>=42, setuptools_scm[toml]>=6.2, wheel)
* Getting build dependencies for sdist...
Compiling cqlshlib/copyutil.py because it changed.                                                                     
[1/1] Cythonizing cqlshlib/copyutil.py
/tmp/build-env-fi88_xxy/lib/python3.12/site-packages/setuptools_scm/version.py:102: UserWarning: tag 'v6.0.16-scylla' will be stripped of its suffix '-scylla'
  warnings.warn(f"tag {tag!r} will be stripped of its suffix {suffix!r}")
* Building sdist...                                                                                                    
/tmp/build-env-fi88_xxy/lib/python3.12/site-packages/setuptools_scm/version.py:102: UserWarning: tag 'v6.0.16-scylla' will be stripped of its suffix '-scylla'
  warnings.warn(f"tag {tag!r} will be stripped of its suffix {suffix!r}")                              
'cqlshlib' not a regular file -- skipping
Successfully built scylla_cqlsh-6.0.17.dev5+g3baa678.tar.gz                                                            
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1                                           
  ╰─> [55 lines of output]                            
      running dist_info                                                                                                
      creating /tmp/pip-modern-metadata-yjpfnhok/PyYAML.egg-info
      writing /tmp/pip-modern-metadata-yjpfnhok/PyYAML.egg-info/PKG-INFO
      writing dependency_links to /tmp/pip-modern-metadata-yjpfnhok/PyYAML.egg-info/dependency_links.txt
      writing top-level names to /tmp/pip-modern-metadata-yjpfnhok/PyYAML.egg-info/top_level.txt
      writing manifest file '/tmp/pip-modern-metadata-yjpfnhok/PyYAML.egg-info/SOURCES.txt'
      Traceback (most recent call last):
        File "/usr/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()                                      
        File "/usr/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
        File "/usr/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)                                                             
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/lib/python3.12/site-packages/setuptools/build_meta.py", line 366, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/usr/lib/python3.12/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 288, in <module>
        File "/usr/lib/python3.12/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/usr/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/usr/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/usr/lib/python3.12/site-packages/setuptools/dist.py", line 963, in run_command
          super().run_command(command)
        File "/usr/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/usr/lib/python3.12/site-packages/setuptools/command/dist_info.py", line 91, in run
          self.egg_info.run()
        File "/usr/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 321, in run
          self.find_sources()
        File "/usr/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 329, in find_sources
          mm.run()
        File "/usr/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 551, in run
          self.add_defaults()
        File "/usr/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 589, in add_defaults
          sdist.add_defaults(self)
        File "/usr/lib/python3.12/site-packages/setuptools/command/sdist.py", line 112, in add_defaults
          super().add_defaults()
        File "/usr/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
          self._add_defaults_ext()
        File "/usr/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext 
          self.filelist.extend(build_ext.get_source_files())
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<string>", line 204, in get_source_files
        File "/usr/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
          raise AttributeError(attr)
      AttributeError: cython_sources
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

looks like https://github.com/yaml/pyyaml/issues/601 . but i still have the same error after unpinning or bumping pyyaml to 6.0.1 in requirement.txt.

[fruch]

@tchaikov it's need to be pinned also in the pyproject.toml

[tchaikov]

$ git diff
diff --git a/pyproject.toml b/pyproject.toml
index ad9691e..154a301 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,5 +1,6 @@
 [build-system]
 requires = [
+    "PyYAML==6.0.1",
     "setuptools>=42",
     "wheel",
     "Cython",
@@ -9,4 +10,4 @@ build-backend = "setuptools.build_meta"

 [tool.setuptools_scm]
 write_to = "pylib/cqlshlib/_version.py"
-tag_regex = '^(?P<prefix>v)?(?P<version>[^\+-]+)(?P<suffix>-scylla)?$'
\ No newline at end of file
+tag_regex = '^(?P<prefix>v)?(?P<version>[^\+-]+)(?P<suffix>-scylla)?$'
diff --git a/requirements.txt b/requirements.txt
index 3b3c264..e2e12ab 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,5 +1,5 @@
 scylla-driver==3.26.5
 geomet==0.2.1.post1
-PyYAML==6.0
+PyYAML==6.0.1
 click==8.1.3
$ reloc/build_reloc.sh --version scylla-5.5-1234 --nodeps
# ...
        File "/usr/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
          self._add_defaults_ext()
        File "/usr/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
          self.filelist.extend(build_ext.get_source_files())
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<string>", line 205, in get_source_files
        File "/usr/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
          raise AttributeError(attr)
      AttributeError: cython_sources
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[tchaikov]

i must be missing something?

[fruch]

i must be missing something?

I've managed to reproduce it

when uninstalling cython completely it worked also when doing pip install 'cython<3' it worked

so seems like we need to make sure the env we are running doesn't have cython or have one lower then 3

[tchaikov]

yeah, but IMHO, to remove Cython from system is the recipe to bad performance for the applications which use Cython. not to mention some of them rely on Cython to build. i knew, there are some workaround in this regard. but i always prefer using the packaged Cython.

and use Cython < 3, is exactly one of the workarounds proposed by https://github.com/yaml/pyyaml/issues/601 .

anyway, we need to be prepared for this.

[fruch]

one more detail the failing command is not the build of the package, but the download command that need to fetch all dependencies

it uses --no-build-isolation which mean it doesn't download it's own set of versions, and uses what there. the reason it was used since we are using --no-binary :all: to download the source packages

but that's affect also the build dependency which would make it extremely slow, compiled cython from source