search

aws-ia / taskcat

Test all the CloudFormation things! (with TaskCat)
https://aws-ia.github.io/taskcat/
Apache License 2.0
1.16k stars 213 forks source link

Pyyaml 5.4.1-6.0.0 has broken half the python eco system #802

Closed shadycuz closed 11 months ago

shadycuz commented 1 year ago

Describe the bug A recent change in https://github.com/yaml/pyyaml/issues/601 that involves cython v3.0.0 has made many packages fail to install. AWS Taskcat is affected because of how Taskcat has pyyaml pinned.

https://github.com/aws-ia/taskcat/blob/e24f9f35f0eeb83929124f4f806dcc179b05c960/requirements.txt#L9

To Reproduce Steps to reproduce the behavior:

  1. I'm unable to install taskcat when pip decides the version of pyyaml should be >=5.4.1, <=6.0.0

Expected behavior I'm not sure, I just want to install taskcat in my CICD pipeline =)

Screenshots image

Note: without pinning "6.0.1", it fails because of https://github.com/yaml/pyyaml/issues/601

The only way I can get it to work is to use pyyaml<=5.3.1 which has security vulnerabilities

**Version (Please make sure you are running the latest version of taskcat)

Additional context I think the best thing you could do is make sure you are compatible with pyyaml 6.0.1.

Sharmakartik commented 1 year ago

I'm also getting the same error while installing taskcat on python 3.10

gersonsosa commented 1 year ago

Here's a temporary workaround https://github.com/aws-ia/taskcat/issues/803#issuecomment-1665286701

shadycuz commented 1 year ago

@andrew-glenn Hey, as anyone at AWS looked at this? When we try to install taskcat in our CICD pipelines, it ends up installing like 20 version of taskcat and then 30 versions of cfn-lint just to find a combination that works with pyyaml.

Now our builders run out of disk space once a week until we delete the docker cache.

andrew-glenn commented 1 year ago

@andrew-glenn Hey, as anyone at AWS looked at this? When we try to install taskcat in our CICD pipelines, it ends up installing like 20 version of taskcat and then 30 versions of cfn-lint just to find a combination that works with pyyaml.

Now our builders run out of disk space once a week until we delete the docker cache.

I'll check this out first thing tomorrow.

andrew-glenn commented 1 year ago

@shadycuz Please install from source from my fork in https://github.com/aws-ia/taskcat/pull/810; This sorts itself out locally, but I'd rather do this once.

Apologies for the delay in responding to this.

shadycuz commented 1 year ago

@andrew-glenn will give it a go in the morning. Thanks =)

andrew-glenn commented 1 year ago

@shadycuz circling back on this; Any luck ?

shadycuz commented 1 year ago

@andrew-glenn Yes this fixes the issue where pip has to download many versions of the same package looking for an older version that has the same pyyaml requirements as taskcat. This makes it much easier to install taskcat along side other python projects that also depend on pyyaml.