Did you guys just change it to a tar.gz after it being a zip file forever?

[tobsn]

... and put it into a completely new folder?

https://s3.amazonaws.com/elasticache-downloads/ClusterClient/PHP-7.0/latest-64bit

It changed from being a .zip with an artifact/ folder to a .tar.gz with a folder named AmazonElastiCacheClusterClient-2.0.1-PHP70-64bit-libmemcached-1.0.18

Our production env was rebuilt due to a ELB failure and it crashed pretty much all our instances because they now couldn't finishing building.

[atarsha]

Yes, they've changed it to a tar.gz which just broke our elastic beanstalk deploy config scripts because you can't unzip a tar.gz

Do we need to pull the latest and package it in our repo to prevent these kind of issues in the future?

[tobsn]

In case anyone needs it fixed and doesn't know how - this is the old one:

command: curl -o clusterclient-aws-php7.zip https://s3.amazonaws.com/elasticache-downloads/ClusterClient/PHP-7.0/latest-64bit && tar -xzf clusterclient-aws-php7.zip && cp -f artifact/amazon-elasticache-cluster-client.so "$(php -r 'echo ini_get("extension_dir");')"

This is the NEW ONE as of current:

command: curl -o clusterclient-aws-php7.tar.gz https://s3.amazonaws.com/elasticache-downloads/ClusterClient/PHP-7.0/latest-64bit && tar -xzf clusterclient-aws-php7.tar.gz && cp -f AmazonElastiCacheClusterClient-2.0.1-PHP70-64bit-libmemcached-1.0.18/amazon-elasticache-cluster-client.so "$(php -r 'echo ini_get("extension_dir");')"
test: "php -r \"exit((extension_loaded('memcached') ? 1 : 0));\""
cwd: /tmp

[malyeyev-AMZN]

Yes we did. We were not aware people were relying on this for automation and will be more mindful of this sort of thing next time

[atarsha]

@malyeyev-AMZN So what is the expectation for users? Should we be pulling the latest and storing it internally for use with say, Elastic Beanstalk where it needs to be installed each time to avoid future issues like this?

What is your expectation for how we would do that?

[QuChen88]

@atarsha There are a couple of options there I think.

1. You can store the latest artifact locally to avoid downloading it each time. And only refresh it when needed (i.e. new updates/releases come out etc).

2. We changed it to tar.gz to make things a bit more consistent as the artifact is for linux distro. I believe we will try to stick with this convention for the future.

[atarsha]

@QuChen88 Thanks, I ended up storing the client locally in our repository and deploying the tar.gz with the Elastic Beanstalk deploys.

It means manually needing to update the file in the repo from time to time, but prevents our Elastic Beanstalk deployments from suddenly breaking if/when the decision is made to implement another breaking change without warning.

Thanks!

[JohGiz]

We also are affected by the change. Also see that the new folder structure inside the archive is a bit troublesome. With a folder name that has version number and the script that @tobsn has it depends on that the version is not changed from the current AmazonElastiCacheClusterClient-2.0.1-PHP70-64bit-libmemcached-1.0.18. If it does it will not find the file.

I solved it by unpacking the new tar.gz and then searching for the filename amazon-elasticache-cluster-client.so recursively to not be dependent on any minor version change in the foldername. It would be better not to have folders with versions specified in the archive. Myself ended up with storing my own version and not fetching the client on deployment externally at all to avoid any more changes to changs in paths, archives or filename that breaks our deployment.

[malyeyev-AMZN]

Hi All,

Also see that the new folder structure inside the archive is a bit troublesome. With a folder name that has version number and the script that @tobsn has it depends on that the version is not changed from the current AmazonElastiCacheClusterClient-2.0.1-PHP70-64bit-libmemcached-1.0.18. If it does it will not find the file.

Thanks for reporting this. This is a problem indeed and was not intended. We will need to roll back to previous artifact directory name convention to avoid breaking even more users' setups in the future or having to stick to a directory name that will become very confusing starting with the next update.

IMPORTANT: To give everyone advance notice I will make the change Wednesday, Feb 22 at 18:00 UTC. Please update your automation code by then. I apologize again for the inconvenience.

Here what the final result will look like starting Feb 22:

AmazonElastiCacheClusterClient-2.0.1-PHP70.tar.gz
      |__ artifact
                 |__ amazon-elasticache-cluster-client.so

I solved it by unpacking the new tar.gz and then searching for the filename amazon-elasticache-cluster-client.so recursively to not be dependent on any minor version change in the foldername.

@JohGiz, that sounds like a much more robust approach. On some platforms tar will also support --strip=1 option I believe.

@malyeyev-AMZN So what is the expectation for users? Should we be pulling the latest and storing it internally for use with say, Elastic Beanstalk where it needs to be installed each time to avoid future issues like this? What is your expectation for how we would do that?

As far as I know we have never given any explicit or implicit guarantees about how the binary is packaged, that the URL is stable, or any other condition necessary to allow automation - after all we are not distributing it through a software repository, simply providing a link as a convenience. ElastiCache customers are free to use that in a way that suits their needs.

Now of course that we have learned that such changes cause pain for multiple users we will certainly try to avoid doing that again as much as possible.

I would still not recommend that you make any assumptions about the type of the archive file, directory structure once unpacked, name of the *.so binary or the download URL itself.

Storing the a binary locally, as many people have suggested in this thread, (or maybe putting a copy into your own S3 bucket?) seems like a sensible solution. Additionally, the later approach may be a better practice overall since it would protect you from other sorts of breaking changes. For example if a regression was introduced in the latest version of the upstream library and we didn't catch it when merging it in. As long as you keep your own copy and update it only when you actually intend to, you can first ensure that it works in your test environment first.

[sodabrew]

@malyeyev-AMZN If you're building a new object, please also pull in the changes from the 3.0.3 memcached extension release, fixed two crashes when using PHP sessions.

[malyeyev-AMZN]

New tarball is out. Please update your scripts to use the old directory structure (artifact). See long message above.

We will try to integrate 3.0.3 as soon as we can, but it's not part of the updated object just yet.

[Kspen]

I recommend renaming the file by adding a .tar.gz extension, and fix the docs at http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/Appendix.PHPAutoDiscoverySetup.Installing.html

The docs assumes wget uncompresses the archive.