ecmwf-ifs / loki

Freely programmable source-to-source translation for Fortran
https://sites.ecmwf.int/docs/loki/
Apache License 2.0
29 stars 12 forks source link

Fix installation without git checkout #302

Closed reuterbal closed 5 months ago

reuterbal commented 5 months ago

We are using setuptools_scm to determine the version number from the latest tag. This prevents installation if there is no .git worktree information, e.g., when installing from a tarball. The recommended workaround is to provide the variable SETUPTOOLS_SCM_PRETEND_VERSION, which this PR adds to the install script as well as the CMake.

Apparently, I forgot to update the VERSION file for a while, so that's fixed with this as well...

github-actions[bot] commented 5 months ago

Documentation for this branch can be viewed at https://sites.ecmwf.int/docs/loki/302/index.html

awnawab commented 5 months ago

Many many thanks for looking into this and implementing this so quickly πŸ™ Whilst this does allow the installation to complete successfully, I get a ModulenotFoundError when trying to import the frontend:

>>> from loki import frontend
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/naan/Documents/ifs-source/loki/loki/__init__.py", line 13, in <module>
    from loki.frontend import *  # noqa
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/naan/Documents/ifs-source/loki/loki/frontend/__init__.py", line 8, in <module>
    from loki.frontend.preprocessing import *  # noqa
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/naan/Documents/ifs-source/loki/loki/frontend/preprocessing.py", line 15, in <module>
    import pcpp
ModuleNotFoundError: No module named 'pcpp'
codecov[bot] commented 5 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 94.97%. Comparing base (5673795) to head (355471b).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #302 +/- ## ======================================= Coverage 94.96% 94.97% ======================================= Files 153 151 -2 Lines 32427 32407 -20 ======================================= - Hits 30795 30779 -16 + Misses 1632 1628 -4 ``` | [Flag](https://app.codecov.io/gh/ecmwf-ifs/loki/pull/302/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ecmwf-ifs) | Coverage Ξ” | | |---|---|---| | [lint_rules](https://app.codecov.io/gh/ecmwf-ifs/loki/pull/302/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ecmwf-ifs) | `96.39% <ΓΈ> (ΓΈ)` | | | [loki](https://app.codecov.io/gh/ecmwf-ifs/loki/pull/302/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ecmwf-ifs) | `95.13% <ΓΈ> (ΓΈ)` | | | [transformations](https://app.codecov.io/gh/ecmwf-ifs/loki/pull/302/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ecmwf-ifs) | `92.17% <ΓΈ> (ΓΈ)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ecmwf-ifs#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

awnawab commented 5 months ago

I've done a bit more digging. When doing a cmake install without the .git directory, the contents of build/loki_env/lib/python3.10/site-packages/loki are as follows:

config.py  dimension.py  __init__.py  logging.py  module.py  program_unit.py  __pycache__  scope.py  sourcefile.py  subroutine.py  types.py

When installing with the .git directory, we get the following instead:

analyse  batch  config.py     expression  __init__.py  lint module.py       __pycache__  sourcefile.py  tests  transform backend  build dimension.py  frontend    ir logging.py  program_unit.py  scope.py   subroutine.py  tools  types.py

The former is missing all the subdirectories that one would expect in theloki dir e.g. frontend, backend.

This can however easily be worked around by installing loki as an editable package, and I think for the purposes of achieving ecmwf-actions/build-package compatibility that would be enough. So unless the fix is quick, this discrepancy can probably be mopped up in a subsequent pull-request.

awnawab commented 5 months ago

As loki is not yet compatible with python3.12, for macos cmake installations we often want to specify Python3_ROOT_DIR so that we can use the correct python installation rather than the system installation or even the homebrew version which is now at 3.12 because of other projects that depend on it. As such, the find_python_venv function can fail to find the newly created virtual env and keeps finding the python installation pointed to by Python3_ROOT_DIR. This can be remedied by adding set( Python3_ROOT_DIR "${VENV_PATH}" ) just after unset( Python3_EXECUTABLE ) in find_python_venv. Please let me know if you agree and think that's not too hacky.