Poll/question: Naming of PySide2 wheels

[fredrikaverpil]

This is for you who uses these wheels.

With yesterday's version of PySide2 (branch 5.6), the following arguments can now be queried:

__version__         = "2.0.0~alpha0"
__version_info__    = (2, 0, 0, "alpha", 0)

__build_date__ = '2017-08-21T11:46:13+00:00'

__build_commit_hash__ = '8aca0dec116c6b321373f5636b34036530ac6e4b'
__build_commit_hash_described__ = '2.0.0.dev0-4855-g8aca0de'

QtC is not yet maintaining PySide2.__version__ (it doesn't change) and currently all wheels built will be called PySide2-5.6-*.whl. They use the setup.py's __version__ value (which is 5.6).

So next time a commit happens here in the repository, and wheels are re-built, the old ones will be overwritten. This is not good, as previous versions will simply be overwritten – or if older than 180 days, they cannot be overwritten and the wheel upload will fail.

I'm also afraid that something might stop working during build in older versions of e.g. Ubuntu (this actually already happened).

So, we need to do something about this naming. And we can use these new arguments if we wish.

Please note that the reason wheels are now built as PySide2-5.6-*.whl is because the version management is a mess and that in setup.py, the current __version__ is said to be 5.6. There seems to be no reason at all to define versions 5.6., 2.0.0~alpha0 and 2.0.0.dev0 in different places, like today. QtC should really confirm to ONE version string and stick to it, as this is just confusing everyone at this point. It almost seems like they haven't bothered or decided what the current version of PySide2 is actually called. But I digress.

My proposal is that we name them on this form: PySide2-[git branch]-[git commit date YYMMDD]-*.whl where the date comes from the commit at HEAD (of the selected git branch). Meaning, I get both these attributes directly from git.

This would make the wheels appear in chronological order in Bintray and if a certain commit is re-built, the wheel is replaced (unless older than 180 days, in which we have a problem).

What say you?

[leycec]

👍 💛 🆗

My proposal is that we name them on this form: PySide2-[git branch]-[git commit date YYMMDD]-*.whl where the date comes from the commit at HEAD (of the selected git branch). Meaning, I get both these attributes directly from git.

Since this promotes sane sorting, conforms to Bintray constraints, and circumvents the "...if older than 180 days, they cannot be overwritten and the wheel upload will fail." trap, this is a good. ^{wut were they thinking}

That said, this otherwise elegant scheme also increases the installation burden on downstream consumers. By which I mean me.

Unless I'm blithely unaware of a Bintray facility to automatically install from the newest wheel, bumping the basename of published wheels will now require downstream consumers to manually bump the contents of installation scripts and instructions. Previously, users could reliably install the newest wheel specific to the desired Python and platform combination merely by reusing a single Bintray-hosted URL constant. The newest Python 3.5 + Ubuntu 16.04 wheel is always guaranteed to be installable regardless of wheel version as follows, for example:

sudo --set-home pip3 install https://dl.bintray.com/fredrikaverpil/pyside2-wheels/ubuntu16.04/PySide2-5.6-cp35-cp35m-linux_x86_64.whl

That's super nice.

If we can help it, preserving this URL guarantee across the proposed nomenclature change would be even nicer. In addition to distributing PySide2-[git branch]-[git commit date YYMMDD]-*.whl wheels, would it be feasible to also distribute reliably named PySide2-[git branch]-*.whl wheels lacking the git commit date but embedding all other metadata – assuming we refresh the latter at least once every 180 days to satisfy arbitrary, senseless, and hate-filled Bintray constraints? The latter wheels would effectively act as symbolic links to the newest versions of the former wheels.

We've authored a Python 3.5 + Ubuntu 16.04 shell script installer for our application embedding your excellent installation instructions for that combination. We'd rather not manually modify this script on each wheel bump. Mostly because we'll never actually do it – and then our clients (impoverished ramen-swilling grad students) will have outdated and eventually broken PySide2 installations and unjustifiably blame our lack of work ethic.

Maybe? My face has gone numb with fatigue, so what do I know! 🤕

[fredrikaverpil]

Previously, users could reliably install the newest wheel specific to the desired Python and platform combination merely by reusing a single Bintray-hosted URL constant. The newest Python 3.5 + Ubuntu 16.04 wheel is always guaranteed to be installable regardless of wheel version

Actually, that's true but with a huge caveat. If the wheel is older than 180 days, Bintray does not allow you to delete it or overwrite it with a newer one. They consider this becoming something downstream users (like you!) depend on, and won't allow me to remove it.

So you'd be stuck forever on that wheel unless the wheel was (probably automatically) re-generated every <180 days. Which I don't think can happen because of maintainability reasons (meaning; I won't be able to make sure that happens). There are in fact stale wheels older than 180 days in Bintray right now which I'm trying to get removed by the Bintray staff since I hit the 10 GB storage limit yesterday. This incidentally triggered a "rate limit" on their end which caused ALL wheels to not download at all. I guess it's their way to get the author's attention... 🍌

I totally get the whole "it automatically is always the latest version" thing. Believe me, I spend countless hours discussing this at work in many regards. It's sometimes nice, and it's sometimes not desirable. In this case, I don't think we have much choice.

Down the line, I'm also going to have to find a solution to the 10 GB disk space limit at Bintray. But that's for a different day.

In addition to distributing PySide2-[git branch]-[git commit date YYMMDD]-.whl wheels, would it be feasible to also distribute reliably named PySide2-[git branch]-.whl

Absolutely. However, that wheel would very likely become stale and impossible to overwrite with a newer one.

I don't think it's a feasible solution to call Bintray everytime that happens.

I'm not too DNS savvy, but perhaps it's possible to create some sort of URL redirect which will always point to the latest wheel of a certain branch, which your build scripts could use. Then, at each new build of the wheels, this redirect would have to be updated to reflect the new URL. Well, it's an idea... albeit sounding a bit sketchy and half-baked.

---

I actually am considering starting to use something other than Bintray. The problem is I need to find something which can handle automatic uploads by Travis-CI and AppVeyor, like Bintray does, and is completely free for open source projects. If all things fail, I might actually publish the wheels (as unofficial PySide2 wheels, not claiming the PySide2 namespace) on PyPi, although that would be a last resort since these wheels have so special prerequisites. I'm not even sure you can differentiate wheels on different Linux distros with PyPi. But if I ever store them on PyPi, I wouldn't be able to maintain a branch-specific, always latest, wheel.

😞

[fredrikaverpil]

Ok, I just want to mention here that I can delete versions (or files, packages etc) via the Bintray REST API (link to version removal). So the 180 day thing can be manually fixed quite easily by me. Not sure I can make Travis and AppVeyor do it though.

# Example
curl -X DELETE -ufredrikaverpil:<apiKey> "https://api.bintray.com/packages/fredrikaverpil/pyside2-wheels/development/versions/2.0.0.dev0

That solves the storage capacity issue for quite some time, which is really good.

[leycec]

Thanks for the copious response, Fred. I sympathize with your numerous plights.

If the wheel is older than 180 days, Bintray does not allow you to delete it

makes sense

or overwrite it with a newer one.

does not make sense

They consider this becoming something downstream users (like you!) depend on, and won't allow me to remove it.

...because Daddy Bintray knows best. ಠ_ಠ

It's hard to complain when we're getting something for free, but you and I shall do so anyway.

I totally get the whole "it automatically is always the latest version" thing. Believe me, I spend countless hours discussing this at work in many regards. It's sometimes nice, and it's sometimes not desirable.

I am lazy to a pathetic, disgusting, and nauseating degree. Since video games will not play themselves and corn chips will not eat themselves, automatic is always the desirable thing.

Even if it breaks everything. I'll pay that price – and so will my clients! ^{what choice do they have}

Not sure I can make Travis and AppVeyor do it though.

The need to explicitly publicize a private API key complicates things, maybe? If that's the only issue blocking automated wheel replacement (it's probably not), the Travis-CI Encryption API would like to show you a good time tonight. 🍹

Using this API, so-called "secure" environment variables may be:

• Encrypted either:
  • Locally at the command-line via the travis encrypt subcommand.
  • Remotely via the Repository Settings UI.
• Subsequently expanded throughout the .travis.yml file like any normal (i.e., insecure) environment variable.

Under the safe assumption that your private Bintray key uniformly applies to all branches of this repository (it probably does, but whadda I know), encrypting this key as an environment variable via the Repository Settings UI might be the sensible bet.

In theory, that should then permit you to reliably delete the desired versionless wheel hosted at Bintray before uploading the latest revision of that wheel. And theory is half the battle.

[leycec]

Unrelatedly, thanks a bushel-full of bananas for the PySide2 conda feedstock. ^{...that was you, right}

Leveraging that as an upstream dependency should enable us to push out our own conda feedstock(s), which now appears to be the best-of-breed solution for portable installation of PySide2-based applications. Unless my eerily blue eyes do deceive me. 👁 👁

[fredrikaverpil]

To sum this up, I think I need to revise the version to conform with what the pyside2-feedstock is doing. So, instead of calling the version 5.6, I'll change this into 2.0.0dev0.

Having this said, I don't have to delete the "5.6" wheels right now due to disk space limitations or anything. But they'll go away eventually to free up some space when needed.

Unrelatedly, thanks a bushel-full of bananas for the PySide2 conda feedstock. ...that was you, right

Not really, I just made the author (who is also the author of the pyside-feedstock) aware of some patches I really wanted to be in there (which they are).

I would strongly advise to base your stuff on the conda install instead, if possible. It's way more complete than these wheels, which I feel is more of a hobby project now, mainly because I'm using upstream PPAs etc, rather than having 100% control by building Qt from source – which happens with conda.

should enable us to push out our own conda feedstock(s)

Yep, I believe so. Having deployed Qt.py as a feedstock has opened up my eyes to the strengths of conda-forge. I've got a whole bunch of ideas lined up for software I would love to have on Anaconda Cloud now (not PySide2-related). This is the way forward. I hate to say it, but I really want to leave the whole "portable wheel"-situation to QtC now... it doesn't make sense for me to keep on with this crusade.