Shipping Prisma binaries with Go modules

[steebchen]

Note: This might be very hard to achieve. For now, I am just writing down my thoughts.

The Go client depends on Prisma binaries, which are downloaded by the Go client after it is installed. This is also how the JS client works at this time and was found to be an acceptable solution.

Before we start, there are two types of binaries:

• Prisma CLI (to be able to use the Prisma CLI in Go, without having to install node)
• Prisma engines (query engine which is needed at runtime)

Both type of binaries are fetched on 'generate' time, meaning when you generate the Prisma client on development (or in CI); before running go build. The CLI binaries are just used for development anyway. The query engine however is needed at runtime, but it is also fetched at build time, and then included in the final go binary.

The problem with this is that installing the Go client depends on third parties, as the binaries need to be fetched from S3. Even though Prisma makes sure that this has the highest uptime and as little security issues as possible, builds are not 100% reproducible.

I keep thinking about a better solution for this. A simple way is to package these binaries into a Go file and include them in the repo. This falls apart quite quickly though, because there are different binaries for each target (linux, windows, macos, and now ARM variants for each of those as well), which would not only result in very slow download times (especially because unneeded targets would be fetched as well), but also just doesn't work with Go modules as it has a limit on size of 500 MB, which is reached quickly with these binaries for different versions (every new Prisma versions comes with a set of 10-20 binaries, which total to ~50-100 MB).

This is something I need to investigate again. Maybe it would be acceptable to add an extra dependency to just define what kind of binaries you need (e.g. linux), add these as separate dependencies (e.g. github.com/prisma-client-go/linux, and it would be low enough to run into Go mod limits.

[mateothegreat]

What about externalizing it completely and letting the user worry about getting the binaries out-of-band?

[steebchen]

Not sure I follow @mateothegreat. Feel free to join the discord https://goprisma.org/docs/community