pip install with 8.4.0 tar.gz (without git) doesn't work

[thebaptiste]

Hello I'm trying to pip install esmf 8.4.0 with the tar.gz file, I mean not using git. It doesn't work, I have this error :

Processing /home/baptiste/mfextaddon_scientific/src/layers/layer4_python3_scientific/0630_esmf/build/esmf-8.4.0/src/addon/esmpy
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [33 lines of output]
      Traceback (most recent call last):
        File "/home/baptiste/mfext/build/opt/python3_core/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/home/baptiste/mfext/build/opt/python3_core/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/baptiste/mfext/build/opt/python3_core/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 130, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-c9e7rw70/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "/tmp/pip-build-env-c9e7rw70/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 320, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-c9e7rw70/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 335, in run_setup
          exec(code, locals())
        File "<string>", line 1, in <module>
        File "/tmp/pip-build-env-c9e7rw70/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 87, in setup
          return distutils.core.setup(**attrs)
        File "/tmp/pip-build-env-c9e7rw70/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 147, in setup
          _setup_distribution = dist = klass(attrs)
        File "/tmp/pip-build-env-c9e7rw70/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 476, in __init__
          _Distribution.__init__(
        File "/tmp/pip-build-env-c9e7rw70/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 282, in __init__
          self.finalize_options()
        File "/tmp/pip-build-env-c9e7rw70/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 900, in finalize_options
          ep(self)
        File "/tmp/pip-build-env-c9e7rw70/overlay/lib/python3.10/site-packages/setuptools_git_versioning.py", line 290, in infer_version
          version = version_from_git(dist.metadata.name, **config, root=root)
        File "/tmp/pip-build-env-c9e7rw70/overlay/lib/python3.10/site-packages/setuptools_git_versioning.py", line 645, in version_from_git
          return _sanitize_version(version)
        File "/tmp/pip-build-env-c9e7rw70/overlay/lib/python3.10/site-packages/setuptools_git_versioning.py", line 504, in _sanitize_version
          result = str(Version(sanitized_version))
        File "/tmp/pip-build-env-c9e7rw70/overlay/lib/python3.10/site-packages/packaging/version.py", line 266, in __init__
          raise InvalidVersion(f"Invalid version: '{version}'")
      packaging.version.InvalidVersion: Invalid version: 'changelog_start'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

My esmf.mk file starts with (see ESMF_VERSION_STRING and ESMF_VERSION_STRING_GIT)

# ESMF application makefile fragment
#
# Use the following ESMF_ variables to compile and link
# your ESMF application against this ESMF build.
#
# !!! VERY IMPORTANT: If the location of this ESMF build is   !!!
# !!! changed, e.g. libesmf.a is copied to another directory, !!!
# !!! this file - esmf.mk - must be edited to adjust to the   !!!
# !!! correct new path                                        !!!
#
# Please see end of file for options used on this ESMF build
#

#----------------------------------------------
ESMF_VERSION_STRING=8.4.0
# Not a Git repository
ESMF_VERSION_STRING_GIT=NoGit
#----------------------------------------------

ESMF_VERSION_MAJOR=8
ESMF_VERSION_MINOR=4
ESMF_VERSION_REVISION=0
ESMF_VERSION_PATCHLEVEL=0
ESMF_VERSION_PUBLIC='T'
ESMF_VERSION_BETASNAPSHOT='F'

ESMF_APPSDIR=/home/baptiste/mfext/build/opt/scientific/esmf/bin
ESMF_LIBSDIR=/home/baptiste/mfext/build/opt/scientific/esmf/lib
ESMF_ESMXDIR=/home/baptiste/mfext/build/opt/scientific/esmf/include/ESMX
...

I'm on python 3.10.7 with pip 22.2.2

I think pyproject.toml is not ok for an usage without git, it can't set the version. May be it should use in this case ESMF_VERSION_STRING in esmf.mk. It would be great that both usages with and without git work.

Regards

[billsacks]

My initial thinking on this is that we should stop trying to use setuptools-git-versioning and rely on a hard-coded version string like the starting_version that is already in pyproject.toml. The starting_version string is supposed to be a fall-back, but that fall-back doesn't appear to work robustly so I wonder if we should just stop trying to get the version dynamically and always use this hard-coded string. I think our nightly ESMPy testing would detect if we've forgotten to update that hard-coded string.

I'm not positive, but I think that using the version from esmf.mk wouldn't be as robust, in that you might be inadvertently pointing to an ESMF build from a different ESMF version than the one from which you're installing ESMPy: I think the current logic would detect that but using a version from esmf.mk would not.

What do others think about this?

[theurich]

@billsacks I wonder, do we understand why it is that "that fall-back doesn't appear to work robustly"? I agree it does seem to have issues, but I don't understand enough about the implemented mechanism to say what is causing it, or how it could be fixed. In general though, I like the idea of trying to figure out the version from Git if possible, and if not have a fall-back. In a way that is what we do on the ESMF library side. That said, I am open to a simpler solution if it is more robust. It would be nice to hear from @rokuingh for his thoughts on this, too.

[billsacks]

I don't understand it, but @rokuingh and I found that it wasn't working (at least on my Mac) when there were problems with a different scenario - problems installing setuptools-git-versioning. I do agree that getting the version from git is nice in that it gives more fine-grained version info; the main downside I see (besides possible added complexity) is that it means that our testing isn't covering this fall-back (unless we add a test that specifically tests this fall-back mechanism).

[billsacks]

We have been unable to reproduce this issue: Our Docker container is built using the tarball and has no problems installing ESMPy in this usage (see https://github.com/esmf-org/esmf-containers/blob/feature/esmf-buildenv/esmf-build-release/linux-ubuntu/Dockerfile#L52). I also just tried a local build & install of ESMF and ESMPy on my Mac from the 8.4.0 tarball (using python 3.10.7, pip 22.3 and setuptools-git-versioning 1.12.0) and had no problems: the ESMPy installation correctly picked up this version info from pyproject.toml:

starting_version = "8.4.0" # this is a backup for pip <= 22.0 where git-versioning doesn't work

Since we can't reproduce this issue here, I'm going to close it. If you need further support, please reach out to esmf_support@ucar.edu and include any details you think would be helpful for us to be able to reproduce your issue. However, our lead ESMPy developer is on leave so we may not be able to give a quick reply.