Decide what to do about bintray

[codefromthecrypt]

We are severely over quota on bintray. It may be a better idea to switch to direct uploading to sonatype, using GPG keys we used before at ASF or new keys. Reason being that when we go over quota and are locked out of uploading.. it is a severe issue and they've already extended our quota beyond everyone else. Plus we can't even delete old releases as you aren't allowed to delete too far back.

[codefromthecrypt]

another thing we could do is setup a separate bintray thing for openzipkin-contrib. While we don't do many releases in contrib, it could possibly help with the quota issue as we can delete the whole projects from openzipkin (because you can only delete old versions up to a year back)

[codefromthecrypt]

Drat.. you can't delete packages created over a year ago either. Not sure how to solve the not enough space issue if we aren't allowed to delete things..

[abesto]

Remind me, what's the value we get out of using Bintray?

Assuming it's still significant, I'll reach out to their support, see if we can ask them to remove old stuff beyond the normal age limit. Otherwise I might just throw money at them. That would also get us access to some nicer APIs for quickstart.sh or whatever, IIRC.

On Tue, Jun 25, 2019, 03:11 Adrian Cole notifications@github.com wrote:

Drat.. you can't delete packages created over a year ago either. Not sure how to solve the not enough space issue if we aren't allowed to delete things..

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/openzipkin/zipkin-release/issues/2?email_source=notifications&email_token=AAAOUTU43KFZ7CHNH3YGXJLP4F5ELA5CNFSM4H3DWHE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYOX4OY#issuecomment-505249339, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAOUTVARNVBVEFVORNQWULP4F5ELANCNFSM4H3DWHEQ .

[codefromthecrypt]

oh heh I already contacted them anyway about deleting old stuff. they are looking into it.

we dont use bintray as anything except an intermediary to maven central. it provides 2 major things we would notice if we stopped using it.

1. implicit gpg signing of artifacts
2. api to release repo to maven central (though often this times out it mostly works)

it may be possible to automate both of these things client side with a combo of key storage in travis and some automation of sonatype artifactory. otherwise the latter at least would need to be done manually.

On Wed, Jun 26, 2019, 5:41 AM Zoltán Nagy notifications@github.com wrote:

Remind me, what's the value we get out of using Bintray?

Assuming it's still significant, I'll reach out to their support, see if we can ask them to remove old stuff beyond the normal age limit. Otherwise I might just throw money at them. That would also get us access to some nicer APIs for quickstart.sh or whatever, IIRC.

On Tue, Jun 25, 2019, 03:11 Adrian Cole notifications@github.com wrote:

Drat.. you can't delete packages created over a year ago either. Not sure how to solve the not enough space issue if we aren't allowed to delete things..

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub < https://github.com/openzipkin/zipkin-release/issues/2?email_source=notifications&email_token=AAAOUTU43KFZ7CHNH3YGXJLP4F5ELA5CNFSM4H3DWHE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYOX4OY#issuecomment-505249339 , or mute the thread < https://github.com/notifications/unsubscribe-auth/AAAOUTVARNVBVEFVORNQWULP4F5ELANCNFSM4H3DWHEQ

.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/openzipkin/zipkin-release/issues/2?email_source=notifications&email_token=AAAPVV7T2JSOISXA2AACALLP4KGJ3A5CNFSM4H3DWHE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYRVWYI#issuecomment-505633633, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAPVV3TFF4PLL6XNM6ERRDP4KGJ3ANCNFSM4H3DWHEQ .

[codefromthecrypt]

so bintray helped by marking the stale packages such that I could delete them. probably good reason they wouldn't delete directly. delete requests take a very long time to complete, but at least when deleting at package level, there's only one click.

There are a bunch of old releases in active projects. brute force clicking of several hundred versions might be quite a chore, but it could be done. I can ask bintray next about making "click delete" on those possible...

[abesto]

I feel like there should be an API to do that. Will look into it, target is today evening.

On Tue, Jul 2, 2019, 01:02 Adrian Cole notifications@github.com wrote:

so bintray helped by marking the stale packages such that I could delete them. probably good reason they wouldn't delete directly. delete requests take a very long time to complete, but at least when deleting at package level, there's only one click.

There are a bunch of old releases in active projects. brute force clicking of several hundred versions might be quite a chore, but it could be done. I can ask bintray next about making "click delete" on those possible...

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/openzipkin/zipkin-release/issues/2?email_source=notifications&email_token=AAAOUTTIYXZJX7GDLDGVBLDP5KLKJA5CNFSM4H3DWHE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODY7VOBY#issuecomment-507467527, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAOUTTVJLRUHM7UFXSCHDLP5KLKJANCNFSM4H3DWHEQ .

[abesto]

I can ask bintray next about making "click delete" on those possible...

This sentence is confusing me. I read the rest of the comment to mean that deleting old versions has now been enabled, so clicking delete should be possible?

In other news, there's a "delete version" API endpoint (https://bintray.com/docs/api/#url_delete_version): DELETE /packages/:subject/:repo/:package/versions/:version, so if you can easily compile a list of versions to clean up, then I should be able to easily clean them up via the API (after making a backup just in case, I guess).

Also, the "list versions" API seems to return when the package was uploaded, so I could also set up a script (probably executed manually at this point) that'd list versions nearing the "you won't be able to delete these anymore" age, so that we can clean them up if needed (which could then be fed to the script wrapping the above API). How does that sound?

I also took a look at the pricing; AFAICT it starts at $150 per month plus storage and transfer, which is out of the "whatever I'll just throw money at them so we don't need to think" range for me. I'm getting the feeling that we may be better off automating (or at least 1-click-ifying) GPG signing and releasing directly to Maven central / Sonatype, but let's first see where we get with the cleanup methinks.

[codefromthecrypt]

I can ask bintray next about making "click delete" on those possible...

This sentence is confusing me. I read the rest of the comment to mean that deleting old versions has now been enabled, so clicking delete should be possible?

sorry they did allow me to delete the old apps like spark streaming. however versions of say zipkin that are old are not deletable unless we escalate iiuc

In other news, there's a "delete version" API endpoint (

https://bintray.com/docs/api/#url_delete_version): DELETE /packages/:subject/:repo/:package/versions/:version, so if you can easily compile a list of versions to clean up, then I should be able to easily clean them up via the API (after making a backup just in case, I guess).

maven central is our backup heh ;)

Also, the "list versions" API seems to return when the package was

uploaded, so I could also set up a script (probably executed manually at this point) that'd list versions nearing the "you won't be able to delete these anymore" age, so that we can clean them up if needed (which could then be fed to the script wrapping the above API). How does that sound?

this is good. what I was thinking was if we keep anything old we keep last subminor of each minor and no copies of anything pre 1.0. the only thing that uses these links are docker images until recently when we switched to download from maven central. this means non docker like brave and zipkin reporter.. we really dont need any of the old versions. if someone wants a faster maven central they can use Google's mirror

I also took a look at the pricing; AFAICT it starts at $150 per month plus

storage and transfer, which is out of the "whatever I'll just throw money at them so we don't need to think" range for me.

me too plus it is an account to manage..

I'm getting the feeling that we may be better off automating (or at least

1-click-ifying) GPG signing and releasing directly to Maven central / Sonatype, but let's first see where we get with the cleanup methinks.

I suspect we may end up here but not with as much time criticality.

thanks for the eyes on this zoltan!

—

You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/openzipkin/zipkin-release/issues/2?email_source=notifications&email_token=AAAPVV7DNRXCQXV6S6L5JBLP5OS7RA5CNFSM4H3DWHE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZCJJDY#issuecomment-507810959, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAPVV64PJJIJ7Y42NSC6ATP5OS7RANCNFSM4H3DWHEQ .

[abesto]

Did a little searching; others have faced this issue before, and seem to have come to roughly the same conclusion. https://github.com/getgauge/gauge/blob/master/build/publish_nightly_to_bintray.sh#L144 even implements something similar to what we're considering. I'll try my hands at implementing this in a nice way without making it too Zipkin specific (unless that needs significant extra effort), see where that takes us.

[abesto]

Current state: https://twitter.com/abesto/status/1148349430980055041

I won't reboot to Linux, I'll just write this quick script on Windows 10. Two hours later: ./venv/bin is ./venv/Scripts on win32, so simple Makefiles don't work. pipenv hangs, doesn't work. poetry just plain doesn't work. Nothing works. Just wanted to vent. Thanks for tuning in

To be continued. On Linux.

[codefromthecrypt]

thanks for the story. cant wait to see the next episode

On Tue, Jul 9, 2019, 5:54 AM Zoltán Nagy notifications@github.com wrote:

Current state: https://twitter.com/abesto/status/1148349430980055041

I won't reboot to Linux, I'll just write this quick script on Windows 10. Two hours later: ./venv/bin is ./venv/Scripts on win32, so simple Makefiles don't work. pipenv hangs, doesn't work. poetry just plain doesn't work. Nothing works. Just wanted to vent. Thanks for tuning in

To be continued. On Linux.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/openzipkin/zipkin-release/issues/2?email_source=notifications&email_token=AAAPVVY33U27NGEHR3XXHTTP6OZRVA5CNFSM4H3DWHE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZOO3KQ#issuecomment-509406634, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAPVV7XHE6GFEJCOTVQJLTP6OZRVANCNFSM4H3DWHEQ .

[abesto]

As for the story, I managed to install enough random stuff to get a WSL-based devenv working (brand-new WSL Ubuntu, WSL plugin for VSCode, a metric shitton of apt install, pyenv, much .bashrc tweaking, and finally poetry for dependency / virtualenv management).

I've pushed a commit into a branch (https://github.com/openzipkin/zipkin-release/commit/226a9406384665620c8a9ef908a2a0ed0f5491b7) with what I have so far. If you want to try it out, make sure you have Python 3.7 and Poetry (curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python), then:

cd bintray-cleanup
export BINTRAY_API_KEY=...
export BINTRAY_USERNAME=...
poetry run bintray_cleanup/main.py list-old-versions openzipkin maven zipkin-dependencies 365

The end of the output looks kinda like this:

Next steps are (1) documentation, I guess (2) add code to delete old versions (with confirmation, dry-run, and an option to limit the number of versions to delete). As always, maybe (3) Docker, though for this that might be overkill.

FWIW:

$ poetry run bintray_cleanup/main.py --help
Usage: main.py [OPTIONS] COMMAND [ARGS]...

Options:
  --api-base-url TEXT
  --api-username TEXT  [required]
  --api-key TEXT       [required]
  --help               Show this message and exit.

Commands:
  clear-cache
  list-old-versions
  list-versions

[abesto]

Fixed by #3 (a years worth of artifacts is ~2.7 GB, we should be fine for quite a while)