lread
commented
1 year ago For lein seems like they are currently testing on codeberg via woodpecker. I'll effectively carry out the same tests but against Pomegranate master head.
Closed lread closed 1 year ago
lread
commented
1 year ago For lein seems like they are currently testing on codeberg via woodpecker. I'll effectively carry out the same tests but against Pomegranate master head.
lread
commented
1 year ago Leiningen tests needed one correction to pass. See: https://codeberg.org/lread/leiningen/commit/d2fdfcba577964f4bba28821ad1deab026f198d4
The test was using an invalid value for the :checksum option.
See below for details.
Leiningen is testing on codeberg via woodpecker with this config:
pipeline:
build:
image: debian:stable-slim
commands:
- apt-get update -qq && apt-get install -y gnupg leiningen netcat-openbsd openssh-client
- cd leiningen-core && lein bootstrap
- cd .. && bin/lein testI've gone ahead and forked leiningen over at codeberg: https://codeberg.org/lread/leiningen I'll be using this fork in tests.
To reduce test time, I created a docker image via make-docker-image.sh:
#!/usr/bin/env bash
set -xeou pipefail
POM_PATH=.m2/repository/clj-commons/pomegranate/1.2.22
# Install pomegranate to local m2 repo, will be version 1.2.22
git clone https://github.com/clj-commons/pomegranate pomegranate-testing
cd pomegranate-testing
bb install
cd ..
rm -rf pomegranate-testing
# docker can't copy from absolute paths
mkdir -p ${POM_PATH}
cp -r ~/${POM_PATH}/* ${POM_PATH}
cat <<EOF > Dockerfile
FROM debian:stable-slim
# Basic setup (added git)
RUN apt-get update -qq \
&& apt-get install -y gnupg leiningen netcat-openbsd openssh-client \
&& apt-get install -y git
# Clone my fork of leiningen
RUN git clone https://codeberg.org/lread/leiningen.git
# Copy over pomegranate local deployment
RUN mkdir -p /root/${POM_PATH}
COPY ${POM_PATH} /root/${POM_PATH}
# Copy over our test runner
COPY testing123.sh /testing123.sh
EOF
docker build --no-cache -t lread/leinpomtest .Which incorporates testing123.sh:
#!/usr/bin/env bash
set -xeou pipefail
TEST_BRANCH=$1
# dump image os info
cat /etc/*-release
# select lein branch to test against
cd leiningen
git switch ${TEST_BRANCH}
# sanity - make sure we are testing what we think we are
git diff main
# continue with woodpecker cmds to test
cd leiningen-core
lein bootstrap
cd ..
bin/lein test
# deps tree is interesting, lets dump it
echo "dumping deps"
bin/lein deps :treeAnd then I used lein-pom-test.sh to launch tests:
#!/usr/bin/env bash
set -xeou pipefail
TEST_BRANCH=$1
docker run -i lread/leinpomtest /bin/bash /testing123.sh ${TEST_BRANCH}./lein-pom-test.sh mainGood, this works.
For this run, to get tests to pass, I made 3 changes to leiningen:
Bump pomegranate to my test version 1.2.22
Remove :pedantic? :abort from leningen project.clj
Overcome failures in tests
leiningen.core.test.pedantic/top-level-overrides-transative-laterleiningen.core.test.pedantic/range-causes-other-transative-to-ignore-top-levelThese tests rely on setup with an invalid option of :checksum false.
This now fails because of a fix for https://issues.apache.org/jira/browse/MRESOLVER-151.
Looks like :checksum false would have happened to translate to :checksum :warn, although I think the intent was probably :checksum :ignore.
You can see the changes here: https://codeberg.org/lread/leiningen/compare/main...lread-test-pomegranate-fix1
To test:
./lein-pom-test.sh lread-test-pomegranate-fix1This passes.
I figured it would be interesting to try again but this time without turning pedantic off. I can't easily tell what is an override of a transitive dep vs an explicitly used dep in leiningen. And I don't have any real understanding of leiningen source. But I can guess, I suppose.
So I created a new branch lread-test-pomegranate-fix2 which naively tries to use pomegranate dep versions.
Changes are: https://codeberg.org/lread/leiningen/compare/main...lread-test-pomegranate-fix2
To test:
./lein-pom-test.sh lread-test-pomegranate-fix2This passes. I expect the leiningen team has their own ideas of when to update deps, but nice to know everything passes using all latest pomegranate deps.
lread
commented
1 year ago Detected no problems, no changes seem to be required except the pomegranate bump.
It uses CircleCI, we'll mimic what it does in our verification.
I'll use my fork of mranderson for testing. I'll run these tests from my host OS (which is Linux) using a babashka script. I tested under jdk17.
(require '[babashka.tasks :as t]
'[babashka.fs :as fs]
'[clojure.string :as string])
(defn del-tree [dir]
(when (fs/exists? dir)
(println "Deleting" (str dir))
(fs/delete-tree dir)))
(defn wipe-artifacts [project]
(let [m2-repo (fs/file (fs/home) ".m2" "repository" project)]
(del-tree m2-repo)))
(defn clone [project]
(let [dir (-> project
(string/split #"/")
last
(str "-test"))]
(println "Cloning" project "to dir" dir)
(del-tree dir)
(t/shell (str "git clone https://github.com/" project) dir)
dir))
(defn switch-branch [dir branch]
(t/shell {:dir dir} "git switch" branch)
dir)
(defn git-diff [dir]
(println "<- begin diff from master")
(t/shell {:dir dir} "git diff master")
(println ">- end diff from master")
dir)
(defn deps-tree [dir]
(t/shell {:dir dir} "lein deps :tree"))
(defn install-test-pomegranate []
(let [dir (clone "clj-commons/pomegranate")]
(t/shell {:dir dir} "bb install")))
(defn run-mranderson-tests
"mimic what mranderson is running as tests on circleci"
[dir]
(let [opts {:dir dir}]
(t/shell opts "lein test")
(t/shell opts ".circleci/integration_test.sh")
(t/shell opts "lein with-profile +eastwood eastwood")
(t/shell opts "lein kaocha-coverage --codecov")
dir))
(let [mranderson-branch (first *command-line-args*)]
(or mranderson-branch (do (println "* must specify mranderson branch to test against")
(System/exit 1)))
;; make sure I'm not using local test versions of these libs
(wipe-artifacts "clj-commons/pomegranate")
(wipe-artifacts "thomasa/mranderson")
(install-test-pomegranate)
(-> (clone "lread/mranderson")
(switch-branch mranderson-branch)
(git-diff)
(run-mranderson-tests)
(deps-tree)))bb test.clj masterTests pass.
This lread-test-pomegranate-fix1 branch ended up to only need to bump the pomegranate dep.
https://github.com/lread/mranderson/compare/master...lread-test-pomegranate-fix1
bb test.clj lread-test-pomegranate-fix1Tests pass.
lread
commented
1 year ago I took a more manual approach with this one.
MrAnderson uses pomegranate to resolve dependencies. I don't think I'm helping to verify Pomegranate by verifying MrAnderson inlined deps source code generation. I instead simply look at the deps tree MrAnderson dumps to the console when inlining deps for cider-nrepl.
I compared make inline-deps console stderr/stdout for cider-nrepl the following:
Scenario 1: current deps generation
Scenario 2: fudge cider-nrepl to have MrAnderson generate :unresolved-tree true deps
This all looks good. Dumped deps are unchanged with the new Pomegranate.
lread
commented
1 year ago I think we are good here.
Leiningen will require one fix to its tests for that invalid :checksum value.
It does expose/describe :checksum via its sample project.clj but only documents valid Pomegranate values.
I'll do some sanity testing before the next release.