esmf-org / esmf-profiler

ESMF Profiler application converts binary traces into a web based, dynamic report.
0 stars 2 forks source link

Repeating profile push twice in a row results in a error #98

Closed rsdunlapiv closed 2 years ago

rsdunlapiv commented 2 years ago

Describe the bug

Running the esmf-profiler twice in a row with the push option results in an error:

First push is successful:

[venv] [Rocky.Dunlap@hfe05 esmf-profiler]$ esmf-profiler -t tests/fixtures/test-traces -n 'profilepushtest' -o ./out -p git@github.com:esmf-org/app-profiles.git
profiler.main : Processing trace: tests/fixtures/test-traces
profiler.main : Generating 1 profiles
Cloning into '/tmp/tmp3aqd5bpc'...
remote: Enumerating objects: 512, done.
remote: Counting objects: 100% (512/512), done.
remote: Compressing objects: 100% (413/413), done.
remote: Total 512 (delta 232), reused 351 (delta 79), pack-reused 0
Receiving objects: 100% (512/512), 5.43 MiB | 5.98 MiB/s, done.
Resolving deltas: 100% (232/232), done.
Enumerating objects: 8, done.
Counting objects: 100% (8/8), done.
Delta compression using up to 80 threads.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 814 bytes | 814.00 KiB/s, done.
Total 6 (delta 2), reused 0 (delta 0)
remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
To github.com:esmf-org/app-profiles.git
   5d1e847..b53d837  main -> main

Second push results in an error:

[venv] [Rocky.Dunlap@hfe05 esmf-profiler]$ esmf-profiler -t tests/fixtures/test-traces -n 'profilepushtest' -o ./out -p git@github.com:esmf-org/app-profiles.git
profiler.main : Processing trace: tests/fixtures/test-traces
profiler.main : Generating 1 profiles
Cloning into '/tmp/tmp863h5t37'...
remote: Enumerating objects: 518, done.
remote: Counting objects: 100% (518/518), done.
remote: Compressing objects: 100% (418/418), done.
remote: Total 518 (delta 235), reused 355 (delta 80), pack-reused 0
Receiving objects: 100% (518/518), 5.44 MiB | 8.78 MiB/s, done.
Resolving deltas: 100% (235/235), done.
profiler.git : Subprocess error: Command '['git', 'commit', '-a', '-m', "''Commit profile Rocky.Dunlap/profilepushtest''"]' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/scratch2/NCEPDEV/stmp1/Rocky.Dunlap/esmf-profiler/venv/bin/esmf-profiler", line 11, in <module>
    load_entry_point('profiler', 'console_scripts', 'esmf-profiler')()
  File "/scratch2/NCEPDEV/stmp1/Rocky.Dunlap/esmf-profiler/src/profiler/main.py", line 229, in main
    push_profile_to_repo(input_path=output_path, name=args.name, url=args.push)
  File "/scratch2/NCEPDEV/stmp1/Rocky.Dunlap/esmf-profiler/src/profiler/main.py", line 105, in push_profile_to_repo
    _commit_profile(username, name, _temp)
  File "/scratch2/NCEPDEV/stmp1/Rocky.Dunlap/esmf-profiler/src/profiler/main.py", line 75, in _commit_profile
    return git.commit(f"'Commit profile {username}/{name}'", repopath)
  File "/scratch2/NCEPDEV/stmp1/Rocky.Dunlap/esmf-profiler/src/profiler/git.py", line 57, in commit
    return _command_safe(cmd, repopath)
  File "/scratch2/NCEPDEV/stmp1/Rocky.Dunlap/esmf-profiler/src/profiler/git.py", line 24, in _command_safe
    cmd, cwd=cwd, check=True, stdout=subprocess.PIPE, encoding="utf-8"
  File "/usr/lib64/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['git', 'commit', '-a', '-m', "''Commit profile Rocky.Dunlap/profilepushtest''"]' returned non-zero exit status 1.

The issue might be that there are no changes to commit the second time, so the git commit command is failing. In that case, the tool should still exit cleanly with a message saying there are no changes to the profile, so there is nothing to commit.

To Reproduce Run the command above twice in a row.

Expected behavior User should be able to repeat the push and succeed. If there are no changes, then the command should still not fail, even if no push is required.

rsdunlapiv commented 2 years ago

@ryanlong1004 I update the test procedure to include a "repeat push" step: https://github.com/esmf-org/esmf-profiler/wiki/Testing-Procedures#push-profile-to-github

rsdunlapiv commented 2 years ago

The theory about there being no update to push does not seem to make sense on second thought. At a mimumum, the timestamp in the site.json will be different each time -- so it seems that there should always be at least some diff that needs to be pushed. Needs further investigation into the error.

ryanlong1004 commented 2 years ago

See screenshot below.

Git only returns 0 on success and 1 on failure (even a successful failure).

If not stderr is returned, send stdout to stdout and continue as no error.

If there is stderr, reraise the exception and send a message to stderr.

image

ryanlong1004 commented 2 years ago

https://github.com/esmf-org/esmf-profiler/commit/24d5777c903d1ec5bb8c6d4a6d058202d8da2555