dherault / serverless-offline

Emulate AWS λ and API Gateway locally when developing your Serverless project
MIT License
5.2k stars 794 forks source link

Offline is not using neither the provided esbuild config file nor ts-node as expected #1833

Open Diboby opened 2 weeks ago

Diboby commented 2 weeks ago

Bug Report

Offline is not using neither the provided esbuild config file nor ts-node. The function works fine when deploying to aws with sls deploy or locally with sls dev. My project is based on NestJs, that means decorators et class metadata should be enabled in build configuration. When I run my code with ts-node, it works fine too. But the offline seems change the configuration and never add emission of metadata.

Sample Code

service: my-service

plugins:
  - serverless-offline

build:
  esbuild:
    configFile: ./bundlers/esbuild/esbuild.config.mjs

provider:
  runtime: nodejs20.x
  stage: dev

functions:
  hello:
    events:
      - http:
          method: get
          path: hello
    handler: handler.hello

[!NOTE]

  1. The esbuild configuration file extension is mjs
  2. serverless-offline is the only used plugin

Environment

Additional context/Screenshots

  1. The project is in typescript
  2. No file is output after the build using serverless-offline

Workaround

fridaystreet commented 2 weeks ago

+1, literally just moved over and been trying to work out what the heck was going on, before boiling it down to. umm it doesn't look like offline is actually even using this esbuild config file and where is it even putting all the build files?

Any thoughts on this anyone?

ps exact same config as above

Diboby commented 2 weeks ago

@fridaystreet No file is output after the build using serverless-offline

fridaystreet commented 2 weeks ago

@Diboby no, nothing is output and it isn't touching my esbuild config file as I've tried loggin output of that.

not sure what's going on. desparately need it fixed though. Just spent 2 days looking into migrating to localstack and that is a total nightmare by all accounts

fridaystreet commented 2 weeks ago

do you know if there is anyway to tell servereless oflline not to build and just use the existing package? Just wondering if this could be a workaround. package it with sls and esbuild and just have offline use the prebuilt package.

fridaystreet commented 2 weeks ago

Not 100% sure what I'm looking at, but aside from it basically spitting out the whole servereless class and the serverless config before these debug lines. I think this is basically where serverless actually starts and from what I can see it doesn't look like it's running any sort of build process. I even tried tricking it by running package first so the .serverless dir was there, but it makes no difference.

Hopefully someone can chime in here?

s:sls:lib:serverless: { useInProcess: true }
s:sls:lib:serverless: initializing
s:sls:lifecycle:command:register: package
s:sls:lifecycle:command:register: package:function
s:sls:lifecycle:command:register: deploy
s:sls:lifecycle:command:register: deploy:function
s:sls:lifecycle:command:register: deploy:list
s:sls:lifecycle:command:register: deploy:list:functions
s:sls:lifecycle:command:register: invoke
s:sls:lifecycle:command:register: invoke:local
s:sls:lifecycle:command:register: info
s:sls:lifecycle:command:register: dev
s:sls:lifecycle:command:register: logs
s:sls:lifecycle:command:register: metrics
s:sls:lifecycle:command:register: print
s:sls:lifecycle:command:register: remove
s:sls:lifecycle:command:register: rollback
s:sls:lifecycle:command:register: rollback:function
s:sls:lifecycle:command:register: plugin
s:sls:lifecycle:command:register: plugin
s:sls:lifecycle:command:register: plugin:list
s:sls:lifecycle:command:register: plugin
s:sls:lifecycle:command:register: plugin:search
s:sls:lifecycle:command:register: aws
s:sls:lifecycle:command:register: aws:common
s:sls:lifecycle:command:register: aws:common:validate
s:sls:lifecycle:command:register: aws:common:cleanupTempDir
s:sls:lifecycle:command:register: aws:common:moveArtifactsToPackage
s:sls:lifecycle:command:register: aws:common:moveArtifactsToTemp
s:sls:lifecycle:command:register: aws
s:sls:lifecycle:command:register: aws:package
s:sls:lifecycle:command:register: aws:package:finalize
s:sls:lifecycle:command:register: aws
s:sls:lifecycle:command:register: aws:deploy
s:sls:lifecycle:command:register: aws:deploy:deploy
s:sls:lifecycle:command:register: aws:deploy:finalize
s:sls:lifecycle:command:register: dev-build
s:sls:lifecycle:command:register: aws
s:sls:lifecycle:command:register: aws:info
s:sls:lifecycle:command:register: esbuild-package
s:sls:lifecycle:command:register: dynamodb
s:sls:lifecycle:command:register: dynamodb:migrate
s:sls:lifecycle:command:register: dynamodb:seed
s:sls:lifecycle:command:register: dynamodb:start
s:sls:lifecycle:command:register: dynamodb:noStart
s:sls:lifecycle:command:register: dynamodb:remove
s:sls:lifecycle:command:register: dynamodb:install
s:sls:lifecycle:command:register: prune
s:sls:lifecycle:command:register: offline
s:sls:lifecycle:command:register: offline:functionsUpdated
s:sls:lifecycle:command:register: offline:start
s:sls:lifecycle:command:invoke: Invoke offline:start
s:sls:lifecycle:command:invoke:hook: [1] < before:offline:start:init
sentry: Serverless Sentry is disabled from provided options.
s:sls:lifecycle:command:invoke:hook: [1] > before:offline:start:init
s:sls:lifecycle:command:invoke:hook: [1] < before:offline:start:init
Dynamodb Local Started, Visit: http://localhost:8000/shell
ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...
DynamoDB - created table aleign-api-development-Websocket-Connections
DynamoDB - created table aleign-api-development-GraphQL-Subscriptions
DynamoDB - created table aleign-api-development-GraphQL-Messages
DynamoDB - created table aleign-api-development-Events
DynamoDB - created table aleign-api-development-Notifications
DynamoDB - created table aleign-api-development-Refresh-Tokens
s:sls:lifecycle:command:invoke:hook: [1] > before:offline:start:init
s:sls:lifecycle:command:invoke:hook: [1] < before:offline:start:init
s:sls:lifecycle:command:invoke:hook: [1] > before:offline:start:init
s:sls:lifecycle:command:invoke:hook: [2] < offline:start:init
Starting Offline at stage development (ap-southeast-2)
s:sls:plugin:serverless-offline: options: {
  albPort: 3003,
  corsAllowHeaders: [ 'accept', 'content-type', 'x-api-key', 'authorization' ],
  corsAllowOrigin: [ '*' ],
  corsDisallowCredentials: true,
  corsExposedHeaders: [ 'WWW-Authenticate', 'Server-Authorization' ],
  disableCookieValidation: false,
  dockerHost: 'localhost',
  dockerHostServicePath: null,
  dockerNetwork: null,
  dockerReadOnly: true,
  enforceSecureCookies: false,
  host: 'localhost',
  httpPort: 4000,
  httpsProtocol: null,
  lambdaPort: 3002,
  layersDir: null,
  localEnvironment: false,
  noAuth: false,
  noPrependStageInUrl: true,
  noTimeout: false,
  prefix: '',
  preLoadModules: '',
  reloadHandler: false,
  resourceRoutes: false,
  terminateIdleLambdaTime: 60,
  useDocker: false,
  useInProcess: true,
  webSocketHardTimeout: 7200,
  webSocketIdleTimeout: 600,
  websocketPort: 5001,
  printOutput: false,
  corsConfig: {
    credentials: false,
    exposedHeaders: [ 'WWW-Authenticate', 'Server-Authorization' ],
    headers: [ 'accept', 'content-type', 'x-api-key', 'authorization' ],
    origin: [ '*' ]
  }
}
Diboby commented 2 weeks ago

do you know if there is anyway to tell servereless oflline not to build and just use the existing package? Just wondering if this could be a workaround. package it with sls and esbuild and just have offline use the prebuilt package.

No, unfortunately.

DorianMazur commented 2 weeks ago

Hi! serverless-offline doesn’t use any specific build configuration. Instead, it relies on tsx under the hood.

Here’s the relevant part of the code:

  // If still not loaded, try .js, .mjs, .cjs and .ts in that order.
  // Files ending with .js are loaded as ES modules when the nearest parent package.json
  // file contains a top-level field "type" with a value of "module".
  // https://nodejs.org/api/packages.html#packages_type
  const loaded =
    (pjHasModule && (await _tryAwaitImport(lambdaStylePath, ".js"))) ||
    (await _tryAwaitImport(lambdaStylePath, ".mjs")) ||
    _tryRequireFile(lambdaStylePath, ".cjs") ||
    tsxRequire(`${lambdaStylePath}.ts`, `${lambdaStylePath}.ts`)
  if (loaded) {
    return loaded
  }

https://github.com/dherault/serverless-offline/blob/master/src/lambda/handler-runner/in-process-runner/aws-lambda-ric/UserFunction.js#L185

I think we should remove tsx and switch to using esbuild for the build process and add support for the serverless v4 esbuild config

DorianMazur commented 2 weeks ago

Would you prefer using ts-node instead of esbuild? Switching from tsx to ts-node would be much easier, and I could get it done this week. Adding esbuild would be more complex.

fridaystreet commented 2 weeks ago

Hi, that's good to know. I think the issue is really that there's no control over the build process. You say you could move to tsnode easy. Our code won't build in tsnode, not without a lot of work on the config and if their is no way to pass in a custom config it's going to be a blocker for some. We just sent a long time getting it to build in esbuild to wirk with latest serverless, I think realistically the best path would be to stay aligned to serverless.

Is there any reason why it can't just allow the user to either specify pre built code or provide either a tsnode or esbuild config?

Cheers in advance and thanks for the quick response.

DorianMazur commented 2 weeks ago

We could allow users to provide a ts-node configuration, and it should be straightforward to implement. As for esbuild, I'm not very familiar with how it works in a serverless v4. I'll need to look into whether it's possible to integrate it. If we're talking about pre-built code, I believe you can just reference those pre-built files directly in the serverless.yaml, correct?

Or how do you think that would work?

fridaystreet commented 2 weeks ago

Yeah sure. I mean esbuild is pretty much just out of the box on v4 no config required or you can just provide your own config in to the yaml or a file.

The pre built code thing in serverless as far as I've tested doesn't work. Well at least as far ad I've tried I can't work out how it's supposed to work. It just seems to tred on itself and the doco is pretty shocking.

Just go whatever way you think is best. Apologies I haven't really looked at offline under the hood. I actually just assumed it was just letting serverless do the build. If tsnode is an easy first step, just do that.

fridaystreet commented 1 week ago

bit of an update on this. Seems after some playing around, worked out how to use the pre packaging stage, although it still seems to be a bit flakey and causes a missing package.json error when trying to run with a custom config file.

but the command is

sls package --package ./packaged-path sls deploy --package ./packaged-path/build

the trick is you need to expand the deploy to the build subfolder of the path.

I think this just need to use the esbuild config provided by sls. I could be wrong, It looks like sls either builds its own config from your esbuild settings in the build.esbuild.... section of the yaml or it takes your config build.esbuild.configFile and merges it with some extra stuff from it's own config (ie exclude aws packegs) to produce a final esbuild config.

either way, it just need sto get some sort of final esbuild config via sls config and use that

fridaystreet commented 3 days ago

Thought just drop another quick update. While it would be great to have this just working with slsv4 esbuild, we've just moved it to serverless-esbuild plugin. There's a few quirks with it whcih is why ootb slsv4 support would be good, but it's working