search

boostorg / release-tools

5 stars 24 forks source link

docs-only and source-only archives #50

Open sdarwin opened 10 months ago

sdarwin commented 10 months ago

@alandefreitas you had mentioned at some point the idea of creating docs-only and source-only archives for boost.

could you add that in release-tools?

@glenfe just informed us the jfrog downloads are taking 200TB of bandwidth per month.

Switching to source-only downloads would save $1000's per month.

These are the hosted bundles https://boostorg.jfrog.io/artifactory/main/

beta/     07-Apr-2021 22:52    -
develop/  07-Apr-2021 22:48    -
master/   07-Apr-2021 22:44    -
release/  07-Apr-2021 20:44    -
staging/  04-Aug-2023 13:51    -

There are two places in release-tools where docs-only and source-only archives might be generated.

  1. https://github.com/boostorg/release-tools/blob/develop/ci_boost_release.py
  2. https://github.com/boostorg/release-tools/blob/develop/publish_release.py

There are pros and cons to having the logic in one or the other. Most of the complexity has usually been contained in ci_boost_release.py so that is the likely place. Create subdirectores in jfrog under master/ and develop/. 

develop/boost_1_83_0-snapshot.tar.gz   (keep the same for now)
develop/full/boost_1_83_0-snapshot.tar.gz   
develop/docs/boost_1_83_0-docs-snapshot.tar.gz   
develop/source/boost_1_83_0-snapshot.tar.gz   

master/boost_1_83_0-snapshot.tar.gz    (keep the same for now)
master/full/boost_1_83_0-snapshot.tar.gz   
master/docs/boost_1_83_0-docs-snapshot.tar.gz   
master/source/boost_1_83_0-snapshot.tar.gz

Arguably, the name of the snapshots should always stay the same, such as boost_1_83_0-snapshot.tar.gz , and the determining factor is the directory where it is hosted. This makes it easier in other scripts which consume the archives, to simply switch the location (from full/ to source/) .

Possibly, the docs-only archive could have a -docsadded to the name.

ci_boost_release.py generates and uploads the various snapshot to jfrog.

publish_release.py at the time of a release, downloads snapshots, renames them to release/1.83.0/source/boost_1_83_0.tar.gz, and uploads again.

release/1.83.0/source/
release/1.83.0/docs/
release/1.83.0/full/

Unfortunately, the current archives are using the name "source" already.

I was hoping the first iteration of this feature would publish to multiple folders, but somehow keep the current behavior for a short time, since the old website depends on /source/ actually being /full/.

Perhaps temporarily then:

release/1.83.0/source-only/
release/1.83.0/source/   (actually full)
release/1.83.0/docs/
release/1.83.0/full/
alandefreitas commented 10 months ago

Sure. I just have to merge the code from cppalliance/boost and review it, as we did for the antora functionality.

That's good news. I didn't know this change was also that important. I actually thought there would be a lot of debate over it.

The defaults you suggested are already similar to the names I used in cppalliance/boost so that should be fine.