Needs Hook to Provide Alternative Storage Adapters

[timnolte]

@bradyvercher right, I recognized that this topic is more about the packages.json in my post revision. We haven't quite seen the issue with the packages.json yet. Our issues has been the SatisPress server going down somehow and failing all sorts of builds because they can't pull the plugins.

I did start poking around and saw the Storage interface. However, it seems like there isn't any sort of hook to provide a custom storage adapter which is what I think would be required here: https://github.com/cedaro/satispress/blob/12923cf5bbe56f8d233924e85b0a35d7865860ad/src/ServiceProvider.php#L313-L316

Perhaps there is another way to provide an adapter that I'm just not finding.

Originally posted by @timnolte in https://github.com/cedaro/satispress/issues/81#issuecomment-674130842

[bradyvercher]

You should be able to do something like this to replace services in the container:

add_action( 'satispress_compose', function( $satispress, $container ) {
    $container['storage'] = new AwsS3Adapter();
}, 10, 2 );

As for the server going down, if your build server doesn't cache artifacts between builds, it's probably downloading multiple files at once. I believe everything is being served through PHP, so the connections are kept open until the file is finished downloading, which could be exhausting your SatisPress server resources. Depending on your HTTP server software, you might look into something like X-SendFile to see if that helps, but I can't guarantee that will fix whatever issue it is you're running into.

If you do want to go the S3 route, here's what I was working with as a concept if it'll give you a head start (you'll probably want to inject the client and bucket name as arguments in the constructor, this was just for testing purposes):

use Aws\S3\S3Client;
use SatisPress\Exception\FileNotFound;
use SatisPress\HTTP\Response;
use SatisPress\HTTP\ResponseBody\NullBody;
use SatisPress\Storage\Storage;

/**
 * Amazon S3 storage class.
 *
 * @package SatisPress
 * @since   1.0.0
 */
class AwsS3Adapter implements Storage {
    protected $bucket = 'satispress-packages';
    protected $client;

    public function __construct() {
        $this->client = S3Client::factory( [
            'credentials' => [
                'key'    => AWS_ACCESS_KEY_ID,
                'secret' => AWS_SECRET_ACCESS_KEY,
            ],
            'region'  => 'us-west-2',
            'version' => 'latest',
        ] );
    }

    public function checksum( string $algorithm, string $file ): string {
        try {
            $result = $this->client->headObject( [
                'Bucket' => $this->bucket,
                'Key'    => $file,
            ] );

            $checksum = $result->search( "Metadata.{$algorithm}" );
            // $checksum = $result->search( sprintf( '"@metadata".headers."x-amz-meta-%s"', $algorithm ) );

        } catch ( \Exception $e ) {
            throw FileNotFound::forInvalidChecksum( $filename );
        }

        return $checksum;
    }

    public function delete( string $file ): bool {
        return true;
    }

    public function exists( string $file ): bool {
        return $this->client->doesObjectExist( $this->bucket, $file );
    }

    public function list_files( string $path = '' ): array {
        $args = [ 'Bucket' => $this->bucket ];
        if ( ! empty( $path ) ) {
            $args['Prefix'] = $path;
        }

        $objects = $this->client->getIterator( 'ListObjects', $args );

        $files = [];
        foreach ( $objects as $object ) {
            if ( empty( $object['Size'] ) ) {
                continue;
            }

            $files[] = $object['Key'];
        }

        return $files;
    }

    public function move( string $source, string $destination ): bool {
        $this->client->upload(
            $this->bucket,
            $destination,
            fopen( $source, 'r+' ),
            'private',
            [
                'params' => [
                    'ContentType'   => 'application/zip',
                    'ContentSHA256' => hash_file( 'sha256', $source ),
                    'Metadata'      => [
                        'sha1' => hash_file( 'sha1', $source ),
                    ],
                ],
            ]
        );

        unlink( $source );

        return true;
    }

    public function send( string $file ): Response {
        $command = $this->client->getCommand( 'GetObject', [
            'Bucket' => $this->bucket,
            'Key'    => $file,
        ] );

        $url = $this->client->createPresignedRequest( $command, '+1 hours' );

        return new Response(
            new NullBody(),
            302,
            [ 'Location' => (string) $url->getUri() ]
        );
    }
}

[timnolte]

@bradyvercher yeah, I did bring up with them also the fact that there is no Composer dependency caching going on in the build process. In our case we have multiple web heads serving up the site and if one of those web heads has a problem we can have intermittent read failures from the SatisPress site on a single build.

[bradyvercher]

I'm gonna close this out, but let me know if that didn't help get you what you need.