Serverless CLI hanging

[zakisaadjw]

Are you certain it's a bug?

• [X] Yes, it looks like a bug

Is the issue caused by a plugin?

• [X] It is not a plugin issue

Are you using the latest v3 release?

• [X] Yes, I'm using the latest v3 release

Is there an existing issue for this?

• [X] I have searched existing issues, it hasn't been reported yet

Issue description

Serverless commands hang when run from the commandline. Maybe an auth issue? Scanned codebase for potential calls to serverless endpoints which may be down, to no avail.

Service configuration (serverless.yml) content

N/A - multiple previously deployed applications all hanging

Command name and used flags

serverless deploy --stage dev | serverless login | serverless info --stage dev | etc...

Command output

N/A - console hangs

Environment information

Framework Core: 3.22.0
Plugin: 6.2.2
SDK: 4.3.2

[medikoo]

@zakisaadjw please share full command name, service configuration and command output as requested in template. We're unable to help having zero information

[zakisaadjw]

There is no output. Terminal hangs. I have specified multiple command names in the initial issue... even serverless login does not work.

For service configuration, none are working, but see an example here:

app: <redacted>
service: <redacted>
org: <redacted>
frameworkVersion: "3"

provider:
  profile: <redacted>
  name: aws
  runtime: nodejs16.x
  region: ap-southeast-2
  timeout: 900
  memorySize: 256
  deploymentPrefix: ${opt:stage,self:provider.stage,'dev'}-<redacted>
  # static IP
  vpc:
    securityGroupIds:
      - <redacted>
    subnetIds:
      - <redacted>
      - <redacted>
      - <redacted>
  iam:
    role:
      statements:
        - Effect: "Allow"
          Action:
            - "s3:*"
          Resource:
            Fn::Join:
              - ''
              - - 'arn:aws:s3:::'
                - Ref: CronTimestampStorage
                - '/*'
  environment:
    CRON_TIMESTAMP_BUCKET: ${self:provider.deploymentPrefix}-<redacted>

functions:
  <redacted>:
    environment:
      JOB_NAME: <redacted>
      QUEUE_URL: ${file(./serverless.${opt:stage, self:custom.default_stage}.json):<redacted>}
    handler: src/<redacted>.run
    events:
      - schedule: rate(1 day)

resources:
  Resources:
    CronTimestampStorage:
      Type: AWS::S3::Bucket
      Properties:
        BucketName: ${self:provider.deploymentPrefix}-<redacted>
        AccessControl: Private

[medikoo]

What output do you get with serverless print --debug=* ?

[zakisaadjw]

Hi @medikoo - please see below for output (it has stalled, nothing after what I have attached below):

zaki@<redacted> ~/Projects/<redacted> $ serverless print --debug=* --stage dev
process: resolve CLI input (no service schema)
process: resolve CLI input (service schema)
process: resolve eventual service configuration
process: service configuration found at /home/zaki/Projects/<redacted>/serverless.yml
process: service configuration file successfully parsed
process: resolve variables meta
process: resolve variables in core properties
process: resolve variables in all properties
process: resolve CLI input (AWS service schema)
process: construct Serverless instance
process: initialize Serverless instance
lifecycle:command:register: config
lifecycle:command:register: config:credentials
lifecycle:command:register: create
lifecycle:command:register: install
lifecycle:command:register: package
lifecycle:command:register: package:function
lifecycle:command:register: deploy
lifecycle:command:register: deploy:function
lifecycle:command:register: deploy:list
lifecycle:command:register: deploy:list:functions
lifecycle:command:register: invoke
lifecycle:command:register: invoke:local
lifecycle:command:register: info
lifecycle:command:register: logs
lifecycle:command:register: metrics
lifecycle:command:register: print
lifecycle:command:register: remove
lifecycle:command:register: rollback
lifecycle:command:register: rollback:function
lifecycle:command:register: slstats
lifecycle:command:register: plugin
lifecycle:command:register: plugin
lifecycle:command:register: plugin:list
lifecycle:command:register: plugin
lifecycle:command:register: plugin:search
lifecycle:command:register: config
lifecycle:command:register: config:credentials
lifecycle:command:register: aws
lifecycle:command:register: aws:common
lifecycle:command:register: aws:common:validate
lifecycle:command:register: aws:common:cleanupTempDir
lifecycle:command:register: aws:common:moveArtifactsToPackage
lifecycle:command:register: aws:common:moveArtifactsToTemp
lifecycle:command:register: aws
lifecycle:command:register: aws:package
lifecycle:command:register: aws:package:finalize
lifecycle:command:register: aws
lifecycle:command:register: aws:deploy
lifecycle:command:register: aws:deploy:deploy
lifecycle:command:register: aws:deploy:finalize
lifecycle:command:register: aws
lifecycle:command:register: aws:info
lifecycle:command:register: upgrade
lifecycle:command:register: uninstall
lifecycle:command:register: login
lifecycle:command:register: logout
lifecycle:command:register: generate-event
lifecycle:command:register: test
lifecycle:command:register: dashboard
lifecycle:command:register: output
lifecycle:command:register: output:get
lifecycle:command:register: output:list
lifecycle:command:register: param
lifecycle:command:register: param:get
lifecycle:command:register: param:list

[medikoo]

Is it same when you remove app and org ?

[zakisaadjw]

Looks like everything is now working as intended if I remove these references. Why would this be the case, all of a sudden? How would I go about specifying these parameters in our projects moving forward?

I'm under the impression that these parameters ensure the application is created in the correct org and app within our serverless console, so we'd need a workaround of some sort.

Appreciate the help so far!

[zakisaadjw]

Looks like one of our serverless applications "deployed" successfully after removing these two params - but I cannot find the Cloudformation stack anywhere in our serverless account - so I'm not sure where it got deployed...

Edit: turns out the Cloudformation stack did get updated, only the serverless console wasn't updated (makes sense given the missing params).

[zakisaadjw]

FYI: I added the paramaters back to the YAML today, and have been deploying all day. Just now, running into the same issues. CLI hangs when the paramaters exist in the YAML.

Happy to help debug, how would I be able to assist?

[medikoo]

@zakisaadjw on my side Dashboard deployments work ok.

Are you using deployment profiles? This hang may signal that AWS hangs on authenticaiton, maybe there's some issue with profile you use?

[zakisaadjw]

Everything was working all day again, however it just died - looks like it happens around the same time every day. It's 6:30pm Sydney time currently, and it always seems to die around this time. No logging, no error messages, just straight hanging as I'm trying to close out the day... 😅

We are specifying a profile in the provider block of the YAML, if this is what you're referring to. This profile has an active, valid access key/secret and the AWS CLI is entirely unaffected when using this profile.

[medikoo]

We are specifying a profile in the provider block of the YAML, if this is what you're referring to. This profile has an active, valid access key/secret and the AWS CLI is entirely unaffected when using this profile.

I was referring to deployment profile you may configure directly in the Dashboard.

Still if you're not relying on those, can you please report issue to Dashboard support (in UI you should have support chat)