soniachan commented 2 years ago

Bug description

Starting from Apr 28, prisma1 deploy command is throwing 504 error randomly

[vagrant@f4df011635e4 server]$ prisma1 deploy
Warning: Your Prisma server and Prisma CLI are currently out of sync. They should be on the same minor version.

Prisma CLI version: prisma/1.34.12 (linux-x64) node-v14.19.0
Prisma server version: 1.27.3

For further information, please read: http://bit.ly/prisma-cli-server-sync

Deploying service `project` to stage `playground` to server `default` 33.3s
Service is already up to date.

post-deploy:
"Deployment finished"

Running echo "Deployment finished" ✔
project database - No changes

Running graphql get-schema -p database ✔
Running graphql prepare ✔
[vagrant@f4df011635e4 server]$ prisma1 deploy
Warning: Your Prisma server and Prisma CLI are currently out of sync. They should be on the same minor version.

Prisma CLI version: prisma/1.34.12 (linux-x64) node-v14.19.0
Prisma server version: 1.27.3

For further information, please read: http://bit.ly/prisma-cli-server-sync

Deploying service `project` to stage `playground` to server `default` !

ERROR: GraphQL Error (Code: 504)

{
  "error": "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n</body>\r\n</html>\r\n",
  "status": 504
}

Get in touch if you need help: https://slack.prisma.io
To get more detailed output, run $ export DEBUG="*"
(node:88823) [DEP0066] DeprecationWarning: OutgoingMessage.prototype._headers is deprecated
(Use `node --trace-deprecation ...` to show where the warning was created)

{"key":"error/handled","requestId":"local:cl2k6x2amobal09106ym4tnb7","payload":{"exception":"com.prisma.deploy.schema.AuthFailure: No 'Authorization' header provided.","query":"{\n listProjects {\n name\n }\n }","variables":"{}","code":"3015","stack_trace":"com.prisma.deploy.schema.SchemaBuilderImpl.verifyAuthOrThrow(SchemaBuilder.scala:317)\\n com.prisma.deploy.schema.SchemaBuilderImpl.$anonfun$listProjectsField$1(SchemaBuilder.scala:108)\\n sangria.execution.Resolver.resolveField(Resolver.scala:1024)\\n sangria.execution.Resolver.$anonfun$collectActionsPar$1(Resolver.scala:445)\\n scala.collection.TraversableOnce.$anonfun$foldLeft$1(TraversableOnce.scala:156)\\n scala.collection.TraversableOnce.$anonfun$foldLeft$1$adapted(TraversableOnce.scala:156)\\n scala.collection.Iterator.foreach(Iterator.scala:937)\\n scala.collection.Iterator.foreach$(Iterator.scala:937)\\n scala.collection.AbstractIterator.foreach(Iterator.scala:1425)\\n scala.collection.IterableLike.foreach(IterableLike.scala:70)\\n scala.collection.IterableLike.foreach$(IterableLike.scala:69)\\n scala.collection.AbstractIterable.foreach(Iterable.scala:54)\\n scala.collection.TraversableOnce.foldLeft(TraversableOnce.scala:156)\\n scala.collection.TraversableOnce.foldLeft$(TraversableOnce.scala:154)\\n scala.collection.AbstractTraversable.foldLeft(Traversable.scala:104)\\n sangria.execution.Resolver.collectActionsPar(Resolver.scala:439)\\n sangria.execution.Resolver.resolveFieldsPar(Resolver.scala:45)\\n sangria.execution.Executor.executeOperation(Executor.scala:155)\\n sangria.execution.Executor.$anonfun$execute$7(Executor.scala:97)\\n scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:303)\\n scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:37)\\n scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)\\n akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)\\n akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91)\\n scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)\\n scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81)\\n akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:91)\\n akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)\\n akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44)\\n akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)\\n akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)\\n akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)\\n akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)","message":"No 'Authorization' header provided."}}

How to reproduce

$ prisma deploy

Expected behavior

Schema to be successfully deployed

Prisma information

n/a

Environment & setup

Prisma server is hosted on AWS Fargate by following https://v1.prisma.io/tutorials/deploy-prisma-to-aws-fargate-ct14 The prisma server can be reached at Prisma1 Console https://app.prisma.sh/ and queries are executed successfully in the GraphQL Playground.

Prisma Version

1.27.3

hub-github-integration commented 2 years ago

x

soniachan commented 2 years ago

The issue is cause by horizontal scaling of prisma service where 2 service attempts to connect to management api.

2850

https://techblog.commercetools.com/prisma-horizontal-scaling-a-practical-guide-3a05833d4fc3

prisma / prisma1

Prisma1 deploy throws 504 Gateway Time-out randomly #5203