tinuous setup on smaug lacks .dmg for releases starting with 8.20210630 release

[yarikoptic]

has only

$> ls releases/8.*/*dmg
releases/8.20210330/git-annex_8.20210330_x64.dmg@  releases/8.20210621/git-annex_8.20210621_x64.dmg@
releases/8.20210428/git-annex_8.20210428_x64.dmg@

which might be due to the race condition discussed in #81 since workflow fails in

Run if git rev-parse refs/tags/"$INPUT_COMMITISH" &> /dev/null
[INFO] Building a tag; uploading assets to release ...
[INFO] Creating release
HTTP 422: Validation Failed (https://api.github.com/repos/datalad/git-annex/releases/48976539)
Release.tag_name already exists
Error: Process completed with exit code 1.

? and thus lacking that .dmg "attached" to the release, as e.g. in https://github.com/datalad/git-annex/releases/tag/8.20210903 and thus con/tinuous not fetching it.

But then, even though https://github.com/datalad/git-annex/actions/runs/1198660194 has built git-annex-macos-dmg_8.20210903_x64.zip artifact attached, and inside of it I see git-annex_8.20210903_x64.dmg - I don't see that file anywhere among build artifacts on

(git)smaug:/mnt/datasets/datalad/ci/git-annex[master]git
$> find builds/2021/09 -iname *20210903*dmg  

may be con/tinuous skips download of artifacts from failed builds? (we should probably "fix" that and download even if run fails)

@jwodder WDYT?

[jwodder]

@yarikoptic For the releases affected by #81, the DMG is an asset of the draft releases, and tinuous ignores drafts.

I'm not sure why the workflow artifact wasn't downloaded; tinuous doesn't skip artifacts from failed builds.

[yarikoptic]

I'm not sure why the workflow artifact wasn't downloaded; tinuous doesn't skip artifacts from failed builds.

could you please investigate?

@yarikoptic For the releases affected by #81, the DMG is an asset of the draft releases, and tinuous ignores drafts.

rright! so we do have that draft release https://github.com/datalad/git-annex/releases/tag/untagged-0aa0e40c74c0d6696a8c which has that .dmg -- good, at least we could fetch dmg's from those drafts "manually" for now.

Back on that nicely detailed https://github.com/cli/cli/issues/4270 you filed. I do not get -- OSX workflow was the last one to run according to the times you have, but then why it did try to create a release instead of uploading to an existing one? it smells like may be gh release list | grep -q "^$INPUT_COMMITISH\b" incorrectly worked out on OSX -- or what am I missing?

FWIW, tried that grep on an osx and it seems to work as it should

yoh@datalads-imac2 ~ % echo 123 | grep -q "^$INPUT_COMMITISH\b" && echo yes || echo no      
yes

[jwodder]

@yarikoptic Regarding the gh bug, I don't get it either; that's why I filed the bug report (though I do now realize the problem is more likely in gh release list somewhere than in gh release create).

[jwodder]

@yarikoptic tinuous isn't downloading build artifacts for release builds because those are triggered "manually", and your tinuous.yaml file has types: [cron, pr, push]. Either add "manual" to the list or remove the setting altogether to fetch runs with all supported triggers. Note that doing this will only add future manual runs; if you want to collect old runs as well, you need to delete .tinuous.state.json, optionally set since: in tinuous.yaml to something closer to the present, and rerun tinuous (which may take a while).

[yarikoptic]

re "manual": Thank you @jwodder! Added, redid "since" yesterday and it worked out perfectly and I got manual-20210903/build-macos.yaml-414-7052974a-failed//git-annex-macos-dmg_8.20210903_x64/git-annex_8.20210903_x64.dmg. I don't think I would bother with rerunning for previous releases.

re "gh bug" - I guess might be worth crafting a small workflow which would shows (or better uploads into artifacts ) the output of gh release list across OSes and compare how that output differs?

[jwodder]

@yarikoptic

re "gh bug" - I guess might be worth crafting a small workflow which would shows (or better uploads into artifacts ) the output of gh release list across OSes and compare how that output differs?

Tried that; the output's all the same. (Interestingly, the output doesn't include the draft releases, while they are included if I run the command on my computer. Must be a token permissions issue, though I doubt it's responsible for the bug.)

[yarikoptic]

odd... could you "simulate" the release (push some new tag which we would later remove etc) while also dumping that output from release list and exact value for INPUT_COMMITISH -- I still have no idea how/why that situation has happened... of cause we could also adjust workflow script to remove list entirely : so if gh release create reports that it is already known -- we go to gh release upload and if reports something else -- error out.

[jwodder]

@yarikoptic After adding debugging to the workflow, running it on a new tag on both Ubuntu and macOS at once failed to reproduce the error.

[yarikoptic]

We could then just merge that PR and keep debug diagnostic output in deployment to troubleshoot when/if next failure happens on a real release.

[yarikoptic]

so, seems that we are doing ok for now

(git)smaug:/mnt/datasets/datalad/ci/git-annex[master]git
$> /bin/ls -t1d releases/*/*dmg
releases/10.20220127/git-annex_10.20220127_x64.dmg
releases/8.20211123/git-annex_8.20211123_x64.dmg
releases/8.20211117/git-annex_8.20211117_x64.dmg
releases/8.20211028/git-annex_8.20211028_x64.dmg
releases/8.20210907a1/git-annex_8.20210903+git12-ged789059b_x64.dmg
releases/8.20210621/git-annex_8.20210621_x64.dmg
releases/8.20210428/git-annex_8.20210428_x64.dmg
releases/8.20210330/git-annex_8.20210330_x64.dmg