Open jonahallibone opened 3 years ago
I am not sure how Prisma client works, but there is build.postBuildCommands
which you can run arbitrary commands (like executing a script to copy files into a Lambda bundle) which you can use for this purpose.
Actually the PR you are referring, which was for image optimization, was not needed (https://github.com/serverless-nextjs/serverless-next.js/pull/841) since actually I had to modify sharp
package.json
to include the Sharp binaries. I don't think there was any issue referencing and executing the binary once it was copied the Lambda bundle.
@dphang I think the origins of this issue stem from the fact that file permissions are preserved on upload to lambda. I've tried a couple times to change them locally before upload and I always get an error. Specifically:
EROFS: read-only file system, chmod '/var/task/pages/api/query-engine-rhel-openssl-1.0.x'
at PrismaClientFetcher.request (/var/task/pages/api/drafts.js:206:52123)
at process._tickCallback (internal/process/next_tick.js:68:7) code: 'EROFS', meta: undefined }
This is why I copy to /tmp/ and modify the permissions manually.
With postBuildCommands
is it possible to restore the permissions of the that binary to something which is executable?
Yes, I believe that is correct - permissions should be same as the package you upload. You should be able to use postBuildCommands
to run an arbitrary script to update the Lambda code before it's uploaded (e.g adding additional files or changing permissions).
I did find a related issue: https://github.com/prisma/prisma/issues/5053, it seems it could be a Windows issue. Are you using Windows to deploy?
Also experiencing the same issue deploying from Mac. The root of the issue appears to be AWS not respecting permissions, as above. Running chmod
in postBuildCommands
does not work, because once the files are in AWS the permissions are reset.
Copying the files to a /tmp/ directory works, but only if you specifically set:
process.env.PRISMA_QUERY_ENGINE_BINARY = binaryPath
in code before initiating PrismaClient.
But copying a 25MB file to /tmp/ on every cold start is not the ideal solution.
What part of the deploy process resets the permissions?
@mattpocock did you visit the mentioned issue above (prisma/prisma#5392)? A prisma dev mentioned upgrading to 2.16 should rectify this issue (at least that's how i read it). I haven't tried it yet but I will tomorrow.
@jonahallibone did you have the time to test it out? if so could you share how did you solve it with postBuildCommands
. Thanks in advance
@jonahallibone That's the version I'm on.
I turns out that the binaries were not copied into the lambda. This could be solved by adding a script that triggers before upload. I have set a template up for prisma + serverless-next.js which adds a postBuildCommands
(serverless.yml
) script to do just that.
Is there any way to use something like postBuildCommands
when using the @sls-next/lambda-at-edge
Builder
through the CDK construct library.
I'm able to webpack the query-engine and schema into the bundle under /pages/api/...
but I can't get Prisma to look for those there.
Do you have a repro for me to work on? I can try to figure it out, as I am adding templates for prisma.
@millsp got something working. serverless-nextjs-js's cdk construct doesn't seem to have a way to have post command. Basically just manually move stuff after builder has finished.
import fse from "fs-extra"
import { join, resolve } from "path"
export const nextConfigDir = "."
export const outputDir = "./next-build"
new Builder(nextConfigDir, outputDir, {
args: ["build"],
env: {
NODE_ENV: "development",
NEXT_PUBLIC_API_URL: `https://${process.env.DOMAIN_NAME}`,
NEXTAUTH_URL: `https://${process.env.DOMAIN_NAME}`,
},
})
.build(true)
.then(async () => {
const apiLambdaOutput = resolve(join(outputDir, "api-lambda"))
const prismaDir = "node_modules/.prisma"
const prismaDirSrc = resolve(join(nextConfigDir, prismaDir))
const prismaDirDest = resolve(join(apiLambdaOutput, prismaDir))
await fse.copySync(prismaDirSrc, prismaDirDest)
/* Remove native query engine of workstation */
await fse.remove(resolve(join(prismaDirDest, "client/query-engine-darwin")))
})
.then(() => {
const stack = new NextJSStack(app, "NextJSStackBoilerPlate", {
env: {
region: "us-east-1",
account: process.env.CDK_DEFAULT_ACCOUNT,
},
analyticsReporting: true,
description: "NextJS Serverless CDK Construct Boilerplate",
stackName: "nextjs-boilerplate",
})
})
.catch(e => {
console.error(e)
process.exit(1)
})```
https://github.com/prisma/prisma/issues/5392 is closed now and https://github.com/prisma/prisma/issues/5392 potential soon to (same problem/same workaround).
I turns out that the binaries were not copied into the lambda. This could be solved by adding a script that triggers before upload. I have set a template up for prisma + serverless-next.js which adds a
postBuildCommands
(serverless.yml
) script to do just that.
The workaround from @millsp seems to be the best way currently. Maybe it's worth to mention it in the docs here?
Yes, feel free to add into the FAQs and make a PR, I will approve and merge it :). Perhaps we need some new examples docs...(haven't had time for that since there aren't enough resources)
Trying to transition from mikro-orm to prisma this week, I immediately ran up against one issue mentioned in #841, specifically:
After searching around, I saw many of the issues were either out of date (webpack configs don't need to be touched anymore) or linked to solutions not meant for this repository. Quickly, I want to outline the extra two steps I took to make this work. As mentioned also in #841, this seems to be a problem with
aws-lambda
not respecting file permissions on upload, and thus not allowing the Prisma binary to be executed. To get around this, we can copy to/tmp
before we run Prisma. (I wonder if there's a way we can get abuild.postDeploy
step, where we can run a copy command instead of having to do these sorts of modifications inside the app?).Step 1
I initialize Prisma once for my project in
/config/prisma.js
Inside there, I only hadTo add the copy on app initialization, I modified it to be:
Step 2
This on it's own does not work, because it is still looking in the wrong place for the binary. But as mentioned in the Prisma doc we can define custom paths for the binaries. Therefore I added
to my
.env
fileFinally, make sure you've read this guide and added this to your
prisma/schema.prisma
fileConclusion
Im not sure if a
build.postDeploy
step (I believe the Vercel platform has this now to facilitate Prisma specifically) to allow copying of files or a change/patch toaws-lambda
could mitigate these steps, but it feels sort of hacky to get Prisma working this way. If anyone has better solutions, I'd love to hear them.