radical-cybertools / radical.utils

Utility classes and tools for various radical projects
Other
8 stars 6 forks source link

simplify deployment #404

Closed andre-merzky closed 4 months ago

andre-merzky commented 5 months ago

Cleaner deployment and versioning, prompted by setuptools upgrade.

codecov[bot] commented 4 months ago

Codecov Report

Attention: Patch coverage is 56.09756% with 18 lines in your changes are missing coverage. Please review.

Project coverage is 61.81%. Comparing base (3a782df) to head (8c53239). Report is 14 commits behind head on devel.

Files Patch % Lines
src/radical/utils/heartbeat.py 0.00% 15 Missing :warning:
src/radical/utils/get_version.py 85.71% 2 Missing :warning:
src/radical/utils/stack.py 87.50% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## devel #404 +/- ## ========================================== - Coverage 62.26% 61.81% -0.46% ========================================== Files 61 61 Lines 6777 6764 -13 ========================================== - Hits 4220 4181 -39 - Misses 2557 2583 +26 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

mtitov commented 4 months ago

@andre-merzky updated CI is in place, can you please look into check-runs and specifically what is reported by radical-stack e.g., (1) within the linting job

  python               : /home/runner/work/radical.utils/radical.utils/testenv/bin/python3
  pythonpath           : 
  version              : 3.7.17
  virtualenv           : /home/runner/work/radical.utils/radical.utils/testenv

  radical.gtod         : ?
  radical.pilot        : ?
  radical.saga         : ?
  radical.utils        : 80c813e@

(2) within the testing job

  python               : /home/runner/work/radical.utils/radical.utils/testenv/bin/python3
  pythonpath           : 
  version              : 3.7.17
  virtualenv           : /home/runner/work/radical.utils/radical.utils/testenv

  radical.gtod         : not enough values to unpack (expected 6, got 4)
  radical.pilot        : not enough values to unpack (expected 6, got 4)
  radical.saga         : not enough values to unpack (expected 6, got 4)
  radical.utils        : 80c813e@
andre-merzky commented 4 months ago

@andre-merzky updated CI is in place, can you please look into check-runs and specifically what is reported by radical-stack

Thanks @mtitov - I'll check the stack output for RU. The other (non-RU) modules will only report correctly once they are also updated.

andre-merzky commented 4 months ago

@andre-merzky updated CI is in place, can you please look into check-runs and specifically what is reported by radical-stack

Thanks @mtitov - I'll check the stack output for RU. The other (non-RU) modules will only report correctly once they are also updated.

This now shows as h3f4kjh@detached - does that work for you?

mtitov commented 4 months ago

This now shows as h3f4kjh@detached - does that work for you?

can we have something as what we had before? for example, that was for one of the branches 1.48.0-v1.48.0-1-g93b634f26@fix-env_inheritance, thus if we can keep "targeted_version" (future release) and "branch name" that would be great (maybe adding counter of commits within the devel -> 1.48.0-1-g93b634f26@fix-env_inheritance). That helps to track which devel branches are used for runs.

andre-merzky commented 4 months ago

This now shows as h3f4kjh@detached - does that work for you?

can we have something as what we had before? for example, that was for one of the branches 1.48.0-v1.48.0-1-g93b634f26@fix-env_inheritance, thus if we can keep "targeted_version" (future release) and "branch name" that would be great (maybe adding counter of commits within the devel -> 1.48.0-1-g93b634f26@fix-env_inheritance). That helps to track which devel branches are used for runs.

The versioning scheme did not really change:

$ python3 -c 'import radical.utils as ru; print(ru.get_version())'
('v1.52.0', 'v1.52.0-56-g1620a1e3@feature-deployment', '-56', '@feature-deployment')

The short string in the unit test output is caused by the shallow git clone - most git information (tags, branches) are not available. I can fiddle with that further, but I am not sure that special case is worth the effort...

andre-merzky commented 4 months ago

@mtitov , @mturilli : this should be ready now.

mtitov commented 4 months ago

The versioning scheme did not really change:

$ python3 -c 'import radical.utils as ru; print(ru.get_version())'
('v1.52.0', 'v1.52.0-56-g1620a1e3@feature-deployment', '-56', '@feature-deployment')

The short string in the unit test output is caused by the shallow git clone - most git information (tags, branches) are not available. I can fiddle with that further, but I am not sure that special case is worth the effort...

(1) radical-stack doesn't show that 2nd name, it should show either the 1st one if it is a release or the 2nd one if it is from some branch, here what it shows now

  python               : /home/runner/work/radical.utils/radical.utils/testenv/bin/python3
  pythonpath           : 
  version              : 3.7.17
  virtualenv           : /home/runner/work/radical.utils/radical.utils/testenv

  radical.utils        : 1427a04@detached

(2) can we print version without v, sorry, that could sound as nitpicking, but v is used in tags, version itself is without v

andre-merzky commented 4 months ago

I rewrote this now. Instead of creating version detail and parsing again, setup.py now stores the individual pieces in VERSION and get_version() get's them unaltered. This is simpler, less error prone and makes fewer assumptions about tag structure etc.

andre-merzky commented 4 months ago

Thanks @mtitov, I iterated the PR as suggested.