search

kdybicz / dispose-me

Dispose Me is a simple AWS-hosted disposable email service.
MIT License
5 stars 0 forks source link

Question: Domain hosted outside of AWS #6

Closed shokinn closed 3 years ago

shokinn commented 3 years ago

Hey, I would like to ask if it is possible to set this up with a domain hosted on a different DNS provider/server than Route 53?

See here for my Solution

kdybicz commented 3 years ago

Hey @shokinn,

I'm not an AWS expert yet, so I could be wrong, but I believe that as long as you're able to verify in SES that domain is yours (by adding provided DNS record) and after setting MX DNS record pointing to AWS SES service (in my case it's 10 inbound-smtp.eu-west-1.amazonaws.com) it could be possible... in theory... :)

I would be really curious to see if you managed to set it up! Please keep me posted.

shokinn commented 3 years ago

Hey,

For creating the certificates it was quiet easy. Just create the required CNAME records. https://www.screencast.com/t/AIZPQjnX84

Fore the AWS SES part it was simple as well, just create the correct TXT record for validation and the CNAME records for DKIM, if you want to setup it.

I run into this issue when i try to deploy the app. When I ran it a second time the script successfully completed. https://www.screencast.com/t/9Hmnk3BY

Serverless Domain Manager: Info: Creating domain name before deploy.

  Error --------------------------------------------------

  Error: Unable to create domain mail.xxx.dev
      at ServerlessCustomDomain.<anonymous> (/Users/phg/Workspace/Privat/dispose-me/node_modules/serverless-domain-manager/dist/src/index.js:182:23)
      at Generator.throw (<anonymous>)
      at rejected (/Users/phg/Workspace/Privat/dispose-me/node_modules/serverless-domain-manager/dist/src/index.js:6:65)
      at processTicksAndRejections (internal/process/task_queues.js:93:5)

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information ---------------------------
     Operating System:          darwin
     Node Version:              14.5.0
     Framework Version:         2.11.1 (local)
     Plugin Version:            4.1.2
     SDK Version:               2.3.2
     Components Version:        3.3.0

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

2nd run with debugging enabled.: 

❯ export SLS_DEBUG=*
❯ yarn deploy production
yarn run v1.22.4
$ yarn && yarn lint && jest --all
[1/5] 🔍  Validating package.json...
[2/5] 🔍  Resolving packages...
success Already up-to-date.
$ eslint src -c .eslintrc.json --ext ts
 PASS  src/__tests__/tools/utils.test.ts
 PASS  src/__tests__/email.processor.test.ts
 PASS  src/__tests__/tools/EmailParser.test.ts

Test Suites: 3 passed, 3 total
Tests:       12 passed, 12 total
Snapshots:   0 total
Time:        1.325 s, estimated 2 s
Ran all test suites.
$ sls deploy --stage production
Serverless: Load command interactiveCli
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command config:tabcompletion
Serverless: Load command config:tabcompletion:install
Serverless: Load command config:tabcompletion:uninstall
Serverless: Load command create
Serverless: Load command install
Serverless: Load command package
Serverless: Load command deploy
Serverless: Load command deploy:function
Serverless: Load command deploy:list
Serverless: Load command deploy:list:functions
Serverless: Load command invoke
Serverless: Load command invoke:local
Serverless: Load command info
Serverless: Load command logs
Serverless: Load command metrics
Serverless: Load command print
Serverless: Load command remove
Serverless: Load command rollback
Serverless: Load command rollback:function
Serverless: Load command slstats
Serverless: Load command plugin
Serverless: Load command plugin
Serverless: Load command plugin:install
Serverless: Load command plugin
Serverless: Load command plugin:uninstall
Serverless: Load command plugin
Serverless: Load command plugin:list
Serverless: Load command plugin
Serverless: Load command plugin:search
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command rollback
Serverless: Load command rollback:function
Serverless: Load command upgrade
Serverless: Load command uninstall
Serverless: Load command create_domain
Serverless: Load command delete_domain
Serverless: Load command webpack
Serverless: Load command offline
Serverless: Load command offline:start
Serverless: Load command login
Serverless: Load command logout
Serverless: Load command generate-event
Serverless: Load command test
Serverless: Load command dashboard
Serverless: Load command output
Serverless: Load command output:get
Serverless: Load command output:list
Serverless: Load command param
Serverless: Load command param:get
Serverless: Load command param:list
Serverless: Load command studio
Serverless: Invoke deploy
Serverless: Invoke package
Serverless: Invoke aws:common:validate
Serverless: Invoke aws:common:cleanupTempDir
Serverless: Invoke webpack:validate
Serverless: Invoke webpack:compile
Serverless: Bundling with Webpack...
assets by path views/pages/*.ejs 6.97 KiB
  asset views/pages/index.ejs 1.74 KiB [emitted] [from: views/pages/index.ejs] [copied]
  asset views/pages/inbox.ejs 1.53 KiB [emitted] [from: views/pages/inbox.ejs] [copied]
  asset views/pages/email.ejs 1.38 KiB [emitted] [from: views/pages/email.ejs] [copied]
  asset views/pages/404.ejs 1.2 KiB [emitted] [from: views/pages/404.ejs] [copied]
  asset views/pages/error.ejs 1.11 KiB [emitted] [from: views/pages/error.ejs] [copied]
assets by path views/partials/ 3.69 KiB
  assets by path views/partials/*.ejs 2.44 KiB
    asset views/partials/head.ejs 1.27 KiB [emitted] [from: views/partials/head.ejs] [copied]
    asset views/partials/header.ejs 940 bytes [emitted] [from: views/partials/header.ejs] [copied]
    asset views/partials/footer.ejs 257 bytes [emitted] [from: views/partials/footer.ejs] [copied]
  assets by path views/partials/email/*.ejs 1.25 KiB
    asset views/partials/email/details.ejs 661 bytes [emitted] [from: views/partials/email/details.ejs] [copied]
    asset views/partials/email/list.ejs 623 bytes [emitted] [from: views/partials/email/list.ejs] [copied]
asset src/api.js 4.62 KiB [emitted] [minimized] (name: src/api) 1 related asset
orphan modules 6.98 KiB [orphan] 11 modules
runtime modules 931 bytes 4 modules
./src/api.ts + 11 modules 8.22 KiB [not cacheable] [built] [code generated]
webpack compiled successfully in 1143 ms
assets by path views/pages/*.ejs 6.97 KiB
  asset views/pages/index.ejs 1.74 KiB [emitted] [from: views/pages/index.ejs] [copied]
  asset views/pages/inbox.ejs 1.53 KiB [emitted] [from: views/pages/inbox.ejs] [copied]
  asset views/pages/email.ejs 1.38 KiB [emitted] [from: views/pages/email.ejs] [copied]
  asset views/pages/404.ejs 1.2 KiB [emitted] [from: views/pages/404.ejs] [copied]
  asset views/pages/error.ejs 1.11 KiB [emitted] [from: views/pages/error.ejs] [copied]
assets by path views/partials/ 3.69 KiB
  assets by path views/partials/*.ejs 2.44 KiB
    asset views/partials/head.ejs 1.27 KiB [emitted] [from: views/partials/head.ejs] [copied]
    asset views/partials/header.ejs 940 bytes [emitted] [from: views/partials/header.ejs] [copied]
    asset views/partials/footer.ejs 257 bytes [emitted] [from: views/partials/footer.ejs] [copied]
  assets by path views/partials/email/*.ejs 1.25 KiB
    asset views/partials/email/details.ejs 661 bytes [emitted] [from: views/partials/email/details.ejs] [copied]
    asset views/partials/email/list.ejs 623 bytes [emitted] [from: views/partials/email/list.ejs] [copied]
asset src/email-processor.js 3.01 KiB [emitted] [minimized] (name: src/email-processor) 1 related asset
orphan modules 3.96 KiB [orphan] 8 modules
runtime modules 668 bytes 3 modules
./src/email-processor.ts + 8 modules 5.26 KiB [not cacheable] [built] [code generated]
webpack compiled successfully in 1018 ms
assets by path views/pages/*.ejs 6.97 KiB
  asset views/pages/index.ejs 1.74 KiB [emitted] [from: views/pages/index.ejs] [copied]
  asset views/pages/inbox.ejs 1.53 KiB [emitted] [from: views/pages/inbox.ejs] [copied]
  asset views/pages/email.ejs 1.38 KiB [emitted] [from: views/pages/email.ejs] [copied]
  asset views/pages/404.ejs 1.2 KiB [emitted] [from: views/pages/404.ejs] [copied]
  asset views/pages/error.ejs 1.11 KiB [emitted] [from: views/pages/error.ejs] [copied]
assets by path views/partials/ 3.69 KiB
  assets by path views/partials/*.ejs 2.44 KiB
    asset views/partials/head.ejs 1.27 KiB [emitted] [from: views/partials/head.ejs] [copied]
    asset views/partials/header.ejs 940 bytes [emitted] [from: views/partials/header.ejs] [copied]
    asset views/partials/footer.ejs 257 bytes [emitted] [from: views/partials/footer.ejs] [copied]
  assets by path views/partials/email/*.ejs 1.25 KiB
    asset views/partials/email/details.ejs 661 bytes [emitted] [from: views/partials/email/details.ejs] [copied]
    asset views/partials/email/list.ejs 623 bytes [emitted] [from: views/partials/email/list.ejs] [copied]
asset src/authorizer.js 1.55 KiB [emitted] [minimized] (name: src/authorizer) 1 related asset
runtime modules 668 bytes 3 modules
orphan modules 937 bytes [orphan] 1 module
./src/authorizer.ts + 1 modules 1.8 KiB [built] [code generated]
webpack compiled successfully in 995 ms
Serverless: Invoke webpack:package
(node:16427) [DEP_WEBPACK_CHUNK_MODULES_ITERABLE] DeprecationWarning: Chunk.modulesIterable: Use new ChunkGraph API
(Use `node --trace-deprecation ...` to show where the warning was created)
Serverless: Package lock found - Using locked versions
Serverless: Packing external modules: ejs@^3.1.5
Serverless: Packaging service...
Serverless: Invoke aws:package:finalize
Serverless: Invoke aws:common:moveArtifactsToPackage
Serverless: Invoke aws:common:validate
Serverless Domain Manager: Info: Creating domain name before deploy.
Serverless Domain Manager: Info: Custom domain mail.xxx.dev already exists.
Serverless: Invoke aws:deploy:deploy
Serverless: [AWS cloudformation 400 0.573s 0 retries] describeStacks({ StackName: 'dispose-me-production' })
Serverless: Creating Stack...
Serverless: [AWS cloudformation 200 0.385s 0 retries] createStack({
  StackName: 'dispose-me-production',
  OnFailure: 'DELETE',
  Capabilities: [ 'CAPABILITY_IAM', 'CAPABILITY_NAMED_IAM', [length]: 2 ],
  Parameters: [ [length]: 0 ],
  TemplateBody: '{"AWSTemplateFormatVersion":"2010-09-09","Description":"The AWS CloudFormation template for this Serverless application","Resources":{"ServerlessDeploymentBucket":{"Type":"AWS::S3::Bucket","Properties":{"BucketEncryption":{"ServerSideEncryptionConfiguration":[{"ServerSideEncryptionByDefault":{"SSEAlgorithm":"AES256"}}]}}},"ServerlessDeploymentBucketPolicy":{"Type":"AWS::S3::BucketPolicy","Properties":{"Bucket":{"Ref":"ServerlessDeploymentBucket"},"PolicyDocument":{"Statement":[{"Action":"s3:*","Effect":"Deny","Principal":"*","Resource":[{"Fn::Join":["",["arn:",{"Ref":"AWS::Partition"},":s3:::",{"Ref":"ServerlessDeploymentBucket"},"/*"]]}],"Condition":{"Bool":{"aws:SecureTransport":false}}}]}}}},"Outputs":{"ServerlessDeploymentBucketName":{"Value":{"Ref":"ServerlessDeploymentBucket"}}}}',
  Tags: [ { Key: 'STAGE', Value: 'production' }, [length]: 1 ]
})
Serverless: Checking Stack create progress...
Serverless: [AWS cloudformation 200 0.226s 0 retries] describeStackEvents({
  StackName: 'arn:aws:cloudformation:eu-west-1:***:stack/dispose-me-production/6b6cc6d0-3424-11eb-b530-0605bf425171'
})
...Serverless: [AWS cloudformation 200 0.232s 0 retries] describeStackEvents({
  StackName: 'arn:aws:cloudformation:eu-west-1:***:stack/dispose-me-production/6b6cc6d0-3424-11eb-b530-0605bf425171'
})
Serverless: [AWS cloudformation 200 0.227s 0 retries] describeStackEvents({
  StackName: 'arn:aws:cloudformation:eu-west-1:***:stack/dispose-me-production/6b6cc6d0-3424-11eb-b530-0605bf425171'
})
Serverless: [AWS cloudformation 200 0.241s 0 retries] describeStackEvents({
  StackName: 'arn:aws:cloudformation:eu-west-1:***:stack/dispose-me-production/6b6cc6d0-3424-11eb-b530-0605bf425171'
})
Serverless: [AWS cloudformation 200 0.223s 0 retries] describeStackEvents({
  StackName: 'arn:aws:cloudformation:eu-west-1:***:stack/dispose-me-production/6b6cc6d0-3424-11eb-b530-0605bf425171'
})
.Serverless: [AWS cloudformation 200 0.22s 0 retries] describeStackEvents({
  StackName: 'arn:aws:cloudformation:eu-west-1:***:stack/dispose-me-production/6b6cc6d0-3424-11eb-b530-0605bf425171'
})
....
Serverless: Stack create finished...
Serverless: [AWS cloudformation 200 0.225s 0 retries] describeStackResource({
  StackName: 'dispose-me-production',
  LogicalResourceId: 'ServerlessDeploymentBucket'
})
Serverless: [AWS s3 200 0.312s 0 retries] listObjectsV2({
  Bucket: 'dispose-me-production-serverless***',
  Prefix: 'serverless/dispose-me/production'
})
Serverless: [AWS lambda 404 0.301s 0 retries] getFunction({ FunctionName: 'dispose-me-production-api' })
Serverless: [AWS lambda 404 0.3s 0 retries] getFunction({ FunctionName: 'dispose-me-production-processor' })
Serverless: [AWS lambda 404 0.184s 0 retries] getFunction({ FunctionName: 'dispose-me-production-authorizer' })
Serverless: [AWS sts 200 0.523s 0 retries] getCallerIdentity({})
Serverless: Uploading CloudFormation file to S3...
Serverless: [AWS s3 200 0.45s 0 retries] putObject({
  Body: <Buffer 7b 22 41 57 53 54 65 6d 70 6c 61 74 65 46 6f 72 6d 61 74 56 65 72 73 69 6f 6e 22 3a 22 32 30 31 30 2d 30 39 2d 30 39 22 2c 22 44 65 73 63 72 69 70 74 ... 13461 more bytes>,
  Bucket: 'dispose-me-production-serverless***',
  Key: 'serverless/dispose-me/production/1606861657454-2020-12-01T22:27:37.454Z/compiled-cloudformation-template.json',
  ContentType: 'application/json',
  Metadata: { filesha256: 'TSHpYlVr+FAZz/cR8+n9V/yxldi1yteXMAZKjpyeykI=' }
})
Serverless: Uploading artifacts...
Serverless: Uploading service api.zip file to S3 (403.64 KB)...
Serverless: Uploading service processor.zip file to S3 (401.1 KB)...
Serverless: Uploading service authorizer.zip file to S3 (398.15 KB)...
Serverless: [AWS s3 200 3.444s 0 retries] putObject({
  Body: <Buffer 50 4b 03 04 14 00 08 00 08 00 00 00 21 00 00 00 00 00 00 00 00 00 00 00 00 00 15 00 00 00 6e 6f 64 65 5f 6d 6f 64 75 6c 65 73 2f 2e 62 69 6e 2f 65 6a ... 410678 more bytes>,
  Bucket: 'dispose-me-production-serverless***',
  Key: 'serverless/dispose-me/production/1606861657454-2020-12-01T22:27:37.454Z/processor.zip',
  ContentType: 'application/zip',
  Metadata: { filesha256: 'DhoDDMa+jtl93x5bC23gCFvgi4cLsK0RcX+iWNrUZPg=' }
})
Serverless: [AWS s3 200 3.598s 0 retries] putObject({
  Body: <Buffer 50 4b 03 04 14 00 08 00 08 00 00 00 21 00 00 00 00 00 00 00 00 00 00 00 00 00 15 00 00 00 6e 6f 64 65 5f 6d 6f 64 75 6c 65 73 2f 2e 62 69 6e 2f 65 6a ... 413278 more bytes>,
  Bucket: 'dispose-me-production-serverless***',
  Key: 'serverless/dispose-me/production/1606861657454-2020-12-01T22:27:37.454Z/api.zip',
  ContentType: 'application/zip',
  Metadata: { filesha256: 'AKrm7Vz0b85FbAFJBfYcyv8MNTOUFYraDEodTi2hDZI=' }
})
Serverless: [AWS s3 200 3.472s 0 retries] putObject({
  Body: <Buffer 50 4b 03 04 14 00 08 00 08 00 00 00 21 00 00 00 00 00 00 00 00 00 00 00 00 00 15 00 00 00 6e 6f 64 65 5f 6d 6f 64 75 6c 65 73 2f 2e 62 69 6e 2f 65 6a ... 407654 more bytes>,
  Bucket: 'dispose-me-production-serverless***',
  Key: 'serverless/dispose-me/production/1606861657454-2020-12-01T22:27:37.454Z/authorizer.zip',
  ContentType: 'application/zip',
  Metadata: { filesha256: 'zGXyE/OJivEiCKpRrMU5tYPVDnYwbwFCT/8sZGeHAxA=' }
})
Serverless: Validating template...
Serverless: [AWS cloudformation 200 0.849s 0 retries] validateTemplate({
  TemplateURL: 'https://s3.amazonaws.com/dispose-me-production-serverless***/serverless/dispose-me/production/1606861657454-2020-12-01T22:27:37.454Z/compiled-cloudformation-template.json'
})
Serverless: Updating Stack...
Serverless: [AWS cloudformation 200 1.261s 0 retries] updateStack({
  StackName: 'dispose-me-production',
  Capabilities: [ 'CAPABILITY_IAM', 'CAPABILITY_NAMED_IAM', [length]: 2 ],
  Parameters: [ [length]: 0 ],
  TemplateURL: 'https://s3.amazonaws.com/dispose-me-production-serverless***/serverless/dispose-me/production/1606861657454-2020-12-01T22:27:37.454Z/compiled-cloudformation-template.json',
  Tags: [ { Key: 'STAGE', Value: 'production' }, [length]: 1 ]
})
Serverless: Checking Stack update progress...
Serverless: [AWS cloudformation 200 0.242s 0 retries] describeStackEvents({
  StackName: 'arn:aws:cloudformation:eu-west-1:***:stack/dispose-me-production/6b6cc6d0-3424-11eb-b530-0605bf425171'
})
..........Serverless: [AWS cloudformation 200 0.326s 0 retries] describeStackEvents({
  StackName: 'arn:aws:cloudformation:eu-west-1:***:stack/dispose-me-production/6b6cc6d0-3424-11eb-b530-0605bf425171'
})
..............Serverless: [AWS cloudformation 200 0.304s 0 retries] describeStackEvents({
  StackName: 'arn:aws:cloudformation:eu-west-1:***:stack/dispose-me-production/6b6cc6d0-3424-11eb-b530-0605bf425171'
})
Serverless: [AWS cloudformation 200 0.31s 0 retries] describeStackEvents({
  StackName: 'arn:aws:cloudformation:eu-west-1:***:stack/dispose-me-production/6b6cc6d0-3424-11eb-b530-0605bf425171'
})
Serverless: [AWS cloudformation 200 0.99s 0 retries] describeStackEvents({
  StackName: 'arn:aws:cloudformation:eu-west-1:***:stack/dispose-me-production/6b6cc6d0-3424-11eb-b530-0605bf425171'
})
.Serverless: [AWS cloudformation 200 0.293s 0 retries] describeStackEvents({
  StackName: 'arn:aws:cloudformation:eu-west-1:***:stack/dispose-me-production/6b6cc6d0-3424-11eb-b530-0605bf425171'
})
..Serverless: [AWS cloudformation 200 0.409s 0 retries] describeStackEvents({
  StackName: 'arn:aws:cloudformation:eu-west-1:***:stack/dispose-me-production/6b6cc6d0-3424-11eb-b530-0605bf425171'
})
Serverless: [AWS cloudformation 200 0.434s 0 retries] describeStackEvents({
  StackName: 'arn:aws:cloudformation:eu-west-1:***:stack/dispose-me-production/6b6cc6d0-3424-11eb-b530-0605bf425171'
})
...Serverless: [AWS cloudformation 200 0.427s 0 retries] describeStackEvents({
  StackName: 'arn:aws:cloudformation:eu-west-1:***:stack/dispose-me-production/6b6cc6d0-3424-11eb-b530-0605bf425171'
})
...Serverless: [AWS cloudformation 200 0.583s 0 retries] describeStackEvents({
  StackName: 'arn:aws:cloudformation:eu-west-1:***:stack/dispose-me-production/6b6cc6d0-3424-11eb-b530-0605bf425171'
})
............................Serverless: [AWS cloudformation 200 0.64s 0 retries] describeStackEvents({
  StackName: 'arn:aws:cloudformation:eu-west-1:***:stack/dispose-me-production/6b6cc6d0-3424-11eb-b530-0605bf425171'
})
.........Serverless: [AWS cloudformation 200 0.628s 0 retries] describeStackEvents({
  StackName: 'arn:aws:cloudformation:eu-west-1:***:stack/dispose-me-production/6b6cc6d0-3424-11eb-b530-0605bf425171'
})
.................
Serverless: Stack update finished...
Serverless: Invoke aws:info
Serverless: [AWS cloudformation 200 0.227s 0 retries] describeStacks({ StackName: 'dispose-me-production' })
Serverless: [AWS cloudformation 200 0.3s 0 retries] listStackResources({ StackName: 'dispose-me-production', NextToken: undefined })
Serverless: [AWS cloudformation 200 0.366s 0 retries] describeStackResources({ StackName: 'dispose-me-production' })
Serverless: [AWS apigateway 200 0.283s 0 retries] getApiKey({ apiKey: '***', includeValue: true })
Serverless: [AWS apigateway 200 0.289s 0 retries] getApiKey({ apiKey: '***', includeValue: true })
Service Information
service: dispose-me
stage: production
region: eu-west-1
stack: dispose-me-production
resources: 30
api keys:
  Postman: ***
  Appium tests: ***
endpoints:
  GET - https://xxx.execute-api.eu-west-1.amazonaws.com/production/
  GET - https://xxx.execute-api.eu-west-1.amazonaws.com/production/{proxy+}
functions:
  api: dispose-me-production-api
  processor: dispose-me-production-processor
  authorizer: dispose-me-production-authorizer
layers:
  None
Serverless Domain Manager: Info: Found apiId: *** for mail.xxx.dev
Serverless Domain Manager: Info: Created API mapping '(none)' for mail.xxx.dev
Serverless Domain Manager: Summary: Distribution Domain Name
Serverless Domain Manager:    Domain Name: mail.xxx.dev
Serverless Domain Manager:    Target Domain: xxx.execute-api.eu-west-1.amazonaws.com
Serverless Domain Manager:    Hosted Zone Id: ***
Serverless: Invoke aws:deploy:finalize
Serverless: [AWS s3 200 0.278s 0 retries] listObjectsV2({
  Bucket: 'dispose-me-production-serverless***',
  Prefix: 'serverless/dispose-me/production'
})

**************************************************************************************************************************************
Serverless: Announcing Metrics, CI/CD, Secrets and more built into Serverless Framework. Run "serverless login" to activate for free..
**************************************************************************************************************************************

✨  Done in 131.55s.

So the app is successfull deployed. But I still can't access it. I setup the following CNAME record for mail.xxx.dev, is that correct? 

mail    300 IN  CNAME   xxx.execute-api.eu-west-1.amazonaws.com
shokinn commented 3 years ago

But I still can't access it. I've setup the following CNAME record for mail.xxx.dev, is that correct?

Fixed. It was a typo. I forgot the full stop at the end. The correct record would be: 

mail    300 IN  CNAME   xxx.execute-api.eu-west-1.amazonaws.com.

Next issue 😅 When I try to access the website (https://mail.xxx.dev/?x-api-key=xxx) I get the following error:

{"message": "Internal server error"}

When I try it without key or with a random string I get:

{"message":"Forbidden"}

So the API key should be correct. I tried both API keys which were displayed at the end.

kdybicz commented 3 years ago

I'm seeing it's giving you a hell of a fight 😅 That's probably it's easier just to pay 9 USD for an in-house domain :D As for the next issue:

{"message": "Internal server error"}

This could mean many things, so I would suggest looking into Cloud Watch /aws/lambda/dispose-me-staging-api Log Group for any errors.

shokinn commented 3 years ago

Ah, thanks! I was just looking at a wrong direction. I'm not so familiar with AWS 😅 .

It looks like there is a module called express missing. Here are the logs: 

2020-12-03T09:26:01.138Z    undefined   ERROR   Uncaught Exception  
{
    "errorType": "Runtime.ImportModuleError",
    "errorMessage": "Error: Cannot find module 'express'\nRequire stack:\n- /var/task/src/api.js\n- /var/runtime/UserFunction.js\n- /var/runtime/index.js",
    "stack": [
        "Runtime.ImportModuleError: Error: Cannot find module 'express'",
        "Require stack:",
        "- /var/task/src/api.js",
        "- /var/runtime/UserFunction.js",
        "- /var/runtime/index.js",
        "    at _loadUserApp (/var/runtime/UserFunction.js:100:13)",
        "    at Object.module.exports.load (/var/runtime/UserFunction.js:140:17)",
        "    at Object.<anonymous> (/var/runtime/index.js:43:30)",
        "    at Module._compile (internal/modules/cjs/loader.js:1015:30)",
        "    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1035:10)",
        "    at Module.load (internal/modules/cjs/loader.js:879:32)",
        "    at Function.Module._load (internal/modules/cjs/loader.js:724:14)",
        "    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12)",
        "    at internal/main/run_main_module.js:17:47"
    ]
}

START RequestId: 388474a4-db48-492a-ba2c-2f93669cddae Version: $LATEST
2020-12-03T09:26:01.529Z    undefined   ERROR   Uncaught Exception  
{
    "errorType": "Runtime.ImportModuleError",
    "errorMessage": "Error: Cannot find module 'express'\nRequire stack:\n- /var/task/src/api.js\n- /var/runtime/UserFunction.js\n- /var/runtime/index.js",
    "stack": [
        "Runtime.ImportModuleError: Error: Cannot find module 'express'",
        "Require stack:",
        "- /var/task/src/api.js",
        "- /var/runtime/UserFunction.js",
        "- /var/runtime/index.js",
        "    at _loadUserApp (/var/runtime/UserFunction.js:100:13)",
        "    at Object.module.exports.load (/var/runtime/UserFunction.js:140:17)",
        "    at Object.<anonymous> (/var/runtime/index.js:43:30)",
        "    at Module._compile (internal/modules/cjs/loader.js:1015:30)",
        "    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1035:10)",
        "    at Module.load (internal/modules/cjs/loader.js:879:32)",
        "    at Function.Module._load (internal/modules/cjs/loader.js:724:14)",
        "    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12)",
        "    at internal/main/run_main_module.js:17:47"
    ]
}

END RequestId: 388474a4-db48-492a-ba2c-2f93669cddae
REPORT RequestId: 388474a4-db48-492a-ba2c-2f93669cddae  Duration: 353.31 ms Billed Duration: 354 ms Memory Size: 512 MB Max Memory Used: 14 MB  
Unknown application error occurred
Runtime.ImportModuleError
2020-12-03T09:26:02.434Z    undefined   ERROR   Uncaught Exception  
{
    "errorType": "Runtime.ImportModuleError",
    "errorMessage": "Error: Cannot find module 'express'\nRequire stack:\n- /var/task/src/api.js\n- /var/runtime/UserFunction.js\n- /var/runtime/index.js",
    "stack": [
        "Runtime.ImportModuleError: Error: Cannot find module 'express'",
        "Require stack:",
        "- /var/task/src/api.js",
        "- /var/runtime/UserFunction.js",
        "- /var/runtime/index.js",
        "    at _loadUserApp (/var/runtime/UserFunction.js:100:13)",
        "    at Object.module.exports.load (/var/runtime/UserFunction.js:140:17)",
        "    at Object.<anonymous> (/var/runtime/index.js:43:30)",
        "    at Module._compile (internal/modules/cjs/loader.js:1015:30)",
        "    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1035:10)",
        "    at Module.load (internal/modules/cjs/loader.js:879:32)",
        "    at Function.Module._load (internal/modules/cjs/loader.js:724:14)",
        "    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12)",
        "    at internal/main/run_main_module.js:17:47"
    ]
}
kdybicz commented 3 years ago

woow that's strange... 🤔 All dependencies should be in place after running yarn deploy production. I assume all the tests are passing before deployment?

As a potential workaround you can try to edit serverless.yml file and change:

  webpack:
    includeModules:
      forceExclude:
        - aws-sdk
      forceInclude:
        - ejs

to

  webpack:
    includeModules:
      forceExclude:
        - aws-sdk
      forceInclude:
        - ejs
        - express

Though I have no idea why webpack would want to exclude express framework from the api function 🤯

shokinn commented 3 years ago

Hey, jupp test succeeded for all deployments.

I added express, it failed again in this case for serverless-http. After adding serverless-http it failed on address-rfc2822. After adding these it's now failing on mailparser.

Is there an easy way to completely start over and destroy the stack or do I have to delete every service by hand? #7

shokinn commented 3 years ago

Mhh destroying "works" with: npx sls remove --stage production Like deploying during the first run it breaks at some route53 domain tasks.

Cloud these be made optional?

shokinn commented 3 years ago

I fixed the route53/domain problem. I just created a Hosted Zone on route53 for the zone mail.xxx.dev and delegated it from my primary nameserver.

I also tried to build and deploy the app from a linux machine because I thought that maybe the macOS creating the issues regarding the missing dependencies. Turns out nope does not halt at all -.-

So basically I'm back at that all(?) dependencies are missing.

Tested with: macOS, node 14.5.0, yarn 1.22.4 ubuntu, node 15.3.0, yarn 1.22.5

shokinn commented 3 years ago

Yay 🎉 Finally I was able to deploy it. Basically I was adding the dependencies to the "force include" section until it successfully worked.

serverless.yml: 

[...]
  webpack:
    includeModules:
      forceExclude:
        - aws-sdk
      forceInclude:
        - ejs
        - express
        - serverless-http
        - address-rfc2822
        - mailparser
        - source-map-support
[...]
kdybicz commented 3 years ago

That's strange. I'm running it all on macOS with yarn 1.22.10. Though I have different node version and I would suspect this could be the cause of your problems. As AWS Lambda is running with node 12.x I've installed same version of my machine. I've noticed that sometimes in some of my projects when trying to run them on latest Node version some of the dependencies weren't resolved properly, though in such case Serverless was failing to deploy the stack at all...

I'm glad you finally managed to resolve all of the problems! :)

shokinn commented 3 years ago

For everyone who is looking for a tl;dr, here you go.

  1. Clone the repo
  2. Edit the serverless.yml as shown below (This issue could be based on that I used node v15 instead of v12 as lambda is using it. Thanks @kdybicz for pointing this out!).
  3. Run yarn to install all dependencies
  4. Run yarn test to check if everything is fine.
  5. Setup the aws user
  6. Create a hosted zone on route53 (see below for further instructions)
  7. Choose the Region
  8. Setup the SSL certificate
  9. Setup the .env configuration file. See .env.example for reference.
  10. Deploy the app
  11. Setup SES
  12. Done 🎉 you can now access Dispose Me

Edit serverless.yml

Change the serverless.yml from this: 

  webpack:
    includeModules:
      forceExclude:
        - aws-sdk
      forceInclude:
        - ejs

to that: 

  webpack:
    includeModules:
      forceExclude:
        - aws-sdk
      forceInclude:
        - ejs
        - express
        - serverless-http
        - address-rfc2822
        - mailparser
        - source-map-support

Setup nameserver / route53

E.g. you want to use abc.xyz for receiving emails and mail.abc.xyz for accessing the web interface you have to do the following:

Setup your subdomain (in our example it would be mail.abc.xyz) on aws route53 in the "Hosted Zones section".

Afterwards setup the NS records on your primary nameserver to delegate the subdomain mail.abc.xyz to aws.