Deployment artifact size limit

[alxgrk]

Hi there,

I tried to deploy an app that includes puppeteer to do some scraping. Expectedly, it failed saying that the disk doesn't provide enough space for downloading Chrome.

Started dbos.service - DBOS Cloud application. 
Executing command: migrate_schema 
npm detected. 
npm error code 1 
npm error path /dbos-application/application/node_modules/puppeteer 
npm error command failed 
npm error command sh -c node install.mjs 
npm error Error: ERROR: Failed to set up Chrome v126.0.6478.63! Set "PUPPETEER_SKIP_DOWNLOAD" env variable to skip download. 
npm error at file:///dbos-application/application/node_modules/puppeteer/lib/esm/puppeteer/node/install.js:58:23 
npm error at async Promise.all (index 0) 
npm error at async downloadBrowser (file:///dbos-application/application/node_modules/puppeteer/lib/esm/puppeteer/node/install.js:90:9) { 
npm error [cause]: [Error: ENOSPC: no space left on device, write] { 
npm error errno: -28, 
npm error code: 'ENOSPC', 
npm error syscall: 'write' 
npm error } 
npm error } 

Can you share what the actual resource limits are? I wanted to try the minified version of https://github.com/Sparticuz/chromium?tab=readme-ov-file, hoping that this doesn't exceed the limits.

Thanks in advance and keep on with the great work! Best regards, Alex

[kraftp]

Hello! Each DBOS Cloud executor is currently provisioned with 512 MB memory and 1G disk space. You're right that's useful information to have--we'll add it to our docs. How much space do you need?

[apoliakov]

Hi @alxgrk! In addition to what @kraftp said, I wanted to say it's great to hear from you! I actually sent you a couple of emails before. Check your inbox for alex.poliakov@dbos.dev - maybe it went to spam.

Thanks again for the issue. Let us know how else we can help - here, via email, discord or whichever is best.

[alxgrk]

@kraftp thanks for that. Is it also possible to tell what kind of execution OS is being used? Asking because I managed to load https://github.com/Sparticuz/chromium?tab=readme-ov-file, but now get an error that certain shared objects are not getting loaded. I know this is not a dbos topic - however, there is a mechanism to load those .so files in the library, which unfortunately I'm not able to trigger via the environment variables (see also https://github.com/Sparticuz/chromium/blob/master/source/index.ts#L342 & https://github.com/Sparticuz/chromium/blob/master/source/helper.ts#L43). Error is:

/tmp/chromium: error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory
TROUBLESHOOTING: https://pptr.dev/troubleshooting

@apoliakov sorry, I read your mails but forgot to answer. 🤦

[qianl15]

Hi @alxgrk we've just fixed the environment variables issue! Now env variables specified in dbos-config.yaml will be available during deploy time, so you'll be able to load .so files by specifying something like AWS_EXECUTION_ENV: 20.x. Could you try it again?

The current execution environment is a stripped down Linux based on docker-node20 bookworm-slim.

[apoliakov]

@alxgrk There is another user trying to do something very similar and we're working with them on Discord here: https://discord.com/channels/1156433345631232100/1256489733295439882 You may find that thread interesting!

[alxgrk]

Thanks for the infos, @qianl15 and @apoliakov . I will try again the latest release of dbos-sdk/dbos-cloud cli tools & also check the discord thread.

[qianl15]

Hi @alxgrk (I posted this on Discord, but also wanted to share here): We just found and fixed a Chrome dependency issue. Now I could run puppeteer + serverless chrome on my dev cluster. This fix will go live tomorrow morning and we'll keep you updated!

[qianl15]

Hi @alxgrk the fix is live! I also wrote an example puppeteer app with DBOS: https://github.com/qianl15/dbos-puppeteer It can be directly deployed to DBOS Cloud. Hope this is helpful :)