Separate script source and the binary products

[richardtop]

As of right now, this repository is about 800 MB and can take a long time to clone/copy. Most of the space is occupied by the binary artifacts of which only the last one is usually used.

So, in order to use 50MB of data, the user has to clone 800MB. This is not a one-off operation and it might have to be repeated when the package cache is reset, as the Swift Package Manager doens't support shallow clone.

I suggest splitting the repositories with the scripts and with the package manifests. The best way would be to store the actual binary artifacts on the platform such as Artifactory and make the repository with the package manifests only point to the right binary artifact.

This would significantly increase cloning / package resolution speed, as resolving 50MB is way faster than downloading the full 800+MB of data.

[simonbility]

I was in the process of building something to address this for us internally.

The idea is inspired by how lottie maintains a separate repo which only vends a package providing a binaryTarget.

https://github.com/airbnb/lottie-spm

what i got so far is this script:

#!/bin/sh -e

asset_name="OpenSSL.xcframework.zip"
release_json=$(gh api repos/krzyzanowskim/OpenSSL/releases/latest)
asset_url=$(echo $release_json | jq -r ".assets[] | select(.name == \"$asset_name\") | .url")
newest_tag=$(echo $release_json | jq -r ".tag_name")
current_tag=$(cat .version)

if [[ $newest_tag == $current_tag ]]; then
    echo "Already up-to-date"
else
    echo "Mirroring new tag $newest_tag"

    gh release download $newest_tag --repo krzyzanowskim/OpenSSL -p $asset_name --clobber  

    checksum=$(swift package compute-checksum "$asset_name")

    cat << EOF > Package.swift
// swift-tools-version: 5.8
// The swift-tools-version declares the minimum version of Swift required to build this package.

import PackageDescription

let package = Package(
    name: "OpenSSL",
    products: [
        .library(
            name: "OpenSSL",
            targets: ["OpenSSL"]
        )
    ],
    targets: [
        .binaryTarget(
            name: "OpenSSL",
            url: "$asset_url.zip",
            checksum: "$checksum"
        )
    ]
)
EOF
    echo $newest_tag > .version

    git commit -am "Update to $newest_tag"
    git push
    git tag $newest_tag
    git push --tags
fi

The idea was to have a github action run this periodically checks the main-repo for new releases.

If there is interest i could try to open-source this somehow.

[richardtop]

The problem with this approach is that there is already so much artifacts committed that it won't reduce the footprint of this repo. Otherwise, a great idea.

[simonbility]

Yes it would not decrease the size of this repo. But thats not really the goal though.

This would introduce a separate repo that does not have any code checked in and therefore is really fast to clone.

This new repo only includes a Package manifest that includes OpenSSL as a binaryTarget.

@richardtop I thought thats also what you where suggesting

I suggest splitting the repositories with the scripts and with the package manifests.

[richardtop]

Yep clear. Yes, I agree, this will ease the problem.

I am already doing so, in my own implementation, I use a fork of this repo only for scripts and not to store the artifacts.

Cloned OpenSSL is now only 50 MB

[krzyzanowskim]

Now you can update your setup to use https://github.com/krzyzanowskim/OpenSSL-Package which is a drop-in replacement URL for the same package