Closed mdedetrich closed 1 year ago
Alternatively, here's a script draft to upload generated docs and apidocs to https://github.com/apache/incubator-pekko-site:
#!/bin/sh
set -ex
# Expects generated docs in these folders
# target/javaunidoc
# target/scala-2.13/unidoc
# docs/target/paradox
TARGET_REPO="https://github.com/apache/incubator-pekko-site"
REPO="https://github.com/jrudolph/incubator-pekko-site"
TMPDOCS="/tmp/docstmp"
BRANCH="new-docs"
TARGET_BRANCH="asf-staging"
GITOPTS="--git-dir $TMPDOCS/.git --work-tree $TMPDOCS"
mkdir $TMPDOCS
git clone --depth 1 -b $TARGET_BRANCH $TARGET_REPO $TMPDOCS
git $GITOPTS remote add my $REPO
git $GITOPTS checkout -b $BRANCH
git $GITOPTS rm -r content/api content/japi content/docs
cp -r target/javaunidoc $TMPDOCS/content/japi
cp -r target/scala-2.13/unidoc $TMPDOCS/content/api
cp -r docs/target/paradox $TMPDOCS/content/docs
git $GITOPTS add -A content
git $GITOPTS commit -m "publish docs"
git $GITOPTS push my $BRANCH
echo "Now open $REPO/pull/new/$BRANCH to open PR"
Thanks! Because it makes sense to upload docs for snapshots to nightly (along with an expiry due to their size) I will go ahead and do this anyways. Since the flow is the same we can then also put docs for releases into nightlies and point to that, if there is an ASF process/incubator issue then we can look into using this script.
I also think that the script is not enough, we would also have to either have to document or automate changing the relevant paradox/unidoc settings in sbt to update the absolute link host (since its going to be in a different location), this is one advantage that using nightlies has is that we can just (for now) hardcode the URI in the sbt settings
One consideration regarding regular site vs. nightlies is also how these domains are served and how CDNs get involved.
One consideration regarding regular site vs. nightlies is also how these domains are served and how CDNs get involved.
True, I would assume that since Flink (one of the most popular ASF TLP's) is doing it its a non concern but thats a bold assumption
As discussed in https://the-asf.slack.com/archives/C0491A12EP2/p1675850262146849, at least initially there is a preliminary decision to upload generated paradox/scala+java docs into the apache nightlies (i.e. https://nightlies.apache.org/). The idea is the main website (which is separate and hosted at http://pekko.apache.org/) would point to the docs uploaded to nightlies. The reasons why to do this are
index.html
in a root folder. This makes it really trivial to host generated docs (see https://nightlies.apache.org/flink/flink-ml-docs-master/ as an example)sbt unidoc paradox
and change theLOCAL_PATH
andREMOTE_PATH
)/pekko
folder, pekko-http would sit in/pekko-http
)1.0.0
would sit under/pekko/1.0.0
. Same deal with snapshots but they would have an expiry./pekko/current/<content>
folder which will always point to the latest releaseThe Flink TLP project uses this method (go to https://flink.apache.org/ and on the left menu navigate to Documentation), so there is prior art for this and we can also ask them any questions. The reason why I am advocating for this solution is that its by far the simplest and easiest to get going, its quite similar to what other Scala/OS projects do (especially Scala projects that use Paradox), in fact Lightbend used almost exactly the same method with Akka (thats why Akka/Pekko contains https://github.com/lightbend/sbt-publish-rsync). Its more principled in design especially when taking into account the fact that docs generated from a release (i.e. paradox/scala+java doc) have a different life/update cycle compared to the main website at https://github.com/apache/incubator-pekko-site. The main site would be updated on the https://github.com/apache/incubator-pekko-site git repo where as the doc updating would be handed by a github action/release manager within sbt.
The potential drawbacks are the size of the generated docs (~510 megabytes) although there isn't much we can do about this (scala/javadocs take up almost all of the size and its just how they are generated) and whether putting release documentation in Apache nightlies is problematic (considering that Flink does this I don't think its a concern 🤷 ). Irregardless we should also look into this.