storacha / w3infra

🏗️ Infra for the w3up UCAN protocol implementation
Other
17 stars 7 forks source link

Disable flaky filecoin test for now #419

Closed hannahhoward closed 1 month ago

hannahhoward commented 2 months ago

Goals

currently, seed.run PRs fail 90%+ of the time because of the Filecoin test.

until we have a more reliable way to test the filecoin flow, the PR disables it.

the inserted comment suggests a few solutions -- either spinning up custom infra for w3filecoin pipeline on each pr to get more reliable results, or mocking out the pipeline

either way, this is the temporary fix -- I don't want to put in extra retries or anything -- we just need to find a better way to test, or investigate if there are in fact underlying issues, but in the meantime, other PRs need to get merged.

seed-deploy[bot] commented 2 months ago
View stack outputs - **pr419-w3infra-BillingDbStack** Name | Value -- | -- customerTableName | pr419-w3infra-customer spaceDiffTableName | pr419-w3infra-space-diff spaceSnapshotTableName | pr419-w3infra-space-snapshot usageTable | pr419-w3infra-usage - **pr419-w3infra-BillingStack** Name | Value -- | -- ApiEndpoint | https://a4l2dy0108.execute-api.us-east-2.amazonaws.com billingCronHandlerURL | https://fv2uhnr3nht6vltfghout4n5xe0gdnew.lambda-url.us-east-2.on.aws/ CustomDomain | https://pr419.billing.web3.storage - **pr419-w3infra-CarparkStack** Name | Value -- | -- BucketName | carpark-pr419-0 Region | us-east-2 - **pr419-w3infra-RoundaboutStack** Name | Value -- | -- ApiEndpoint | https://a4xifqw4b7.execute-api.us-east-2.amazonaws.com CustomDomain | https://pr419.roundabout.web3.storage - **pr419-w3infra-UcanInvocationStack** Name | Value -- | -- invocationBucketName | invocation-store-pr419-0 taskBucketName | task-store-pr419-0 workflowBucketName | workflow-store-pr419-0 - **pr419-w3infra-UploadApiStack** Name | Value -- | -- ApiEndpoint | https://t0j8siog0f.execute-api.us-east-2.amazonaws.com CustomDomain | https://pr419.up.web3.storage - **pr419-w3infra-BusStack** - **pr419-w3infra-FilecoinStack** - **pr419-w3infra-IndexerStack** - **pr419-w3infra-ReplicatorStack** - **pr419-w3infra-UcanFirehoseStack** - **pr419-w3infra-UploadDbStack**
Peeja commented 2 months ago

I'm with @alanshaw's sentiments. I think if we have a test failure, that should be a blocker: either we confirm that the test is unnecessary, or we fix it. To do otherwise degrades our confidence in our continuous delivery, and the whole thing falls apart.

That said, it doesn't need to block everyone. We can operate with slightly less confidence for a limited time.

I recommend we merge this and escalate the card that resolves, so that one person is blocked on it while everyone else can move forward with slightly more caution.

Peeja commented 2 months ago
alanshaw commented 2 months ago

That said, it doesn't need to block everyone. We can operate with slightly less confidence for a limited time.

I recommend we merge this and escalate the card that resolves, so that one person is blocked on it while everyone else can move forward with slightly more caution.

+1 I should have cross posted here but I did suggest this same resolution in Discord.

Peeja commented 2 months ago

@alanshaw Added to top of sprint as https://github.com/storacha-network/project-tracking/issues/129. Good to merge this PR?