Monorepo compatibility - Githubissues

mrlubos commented 2 years ago

Hey team mailing, great strides over the last few weeks towards v1!

Based on v0.6.9, I think the last missing piece to unlock usage in monorepos is adding support for path mappings. I have multiple packages in my project that import other packages, so my template could look like this.

import { veryUsefulFunction } from '@repo/package';

export function MailingTemplate() {
  return (
    <Layout>
      <Button onClick={veryUsefulFunction}>
        I want to be useful!
      </Button>
    </Layout>
  );
}

This currently won't work with either dev or build commands. It will fail with the module not found error message. I tried to poke in mailing to see what needs to change and it seems it's the registerRequireHooks() method in registerRequireHooks.ts.

I was able to manually update it to fix the module not found error, but then I ran into other issues such as:

SyntaxError: Unexpected token 'export'
SyntaxError: Cannot use import statement outside a module

Resolving these would require a more involved approach and it's possible this has something to do with mailing itself, so I didn't continue. Any pointers?

psugihara commented 2 years ago

thanks @mrlubos.

a few thoughts:

i'm sure this is just an example but don't put onClicks in your email templates, client-side js won't work in email clients :)
maybe this is related to #93
if you think it may be different from #93, a minimal repro example we could test fixes on would be super useful

mrlubos commented 2 years ago

Hahah yes @psugihara, it's a href in my code. The specific use case is shared path generators between client and emails as emails deep link the application state. I will have a look at the issue and see if I can reproduce anything!

mrlubos commented 2 years ago

@psugihara looks like it's related to #93. I'll put together a demo later

mrlubos commented 2 years ago

@psugihara You should've received an invitation to a CodeAnywhere repository. I've left the instructions to reproduce in README, it's very straightforward. Please let me know if you need any assistance with reproduction or anything else

mrlubos commented 2 years ago

@psugihara Just checking in if you've received any emails/had time to look at this issue?

psugihara commented 2 years ago

Hey, sorry for the delay. I got the email but haven't had time to look into the issue. When I looked at #93 I was able to repro but was not able to figure out a fix (yet).

mrlubos commented 2 years ago

No worries @psugihara, just wanted to make sure my link worked. Let me know if I can help in any way. Would love to help with this project but it's difficult as it's not obvious what's a priority and who's working on what

psugihara commented 2 years ago

Sorry that's unclear. @alexfarrill, @monicakogler, and i sync up periodically live in LA. We can do a better job tracking work in public! Just added #133

Going forward, I'll make sure we're assigned on issues. In general, issues assigned are being worked on, unassigned issues are up for grabs if you want to pull one (though of course best to discuss if it's a huge change).

alex and I are working on a simple way to deploy the preview server in a production mode with the render HTML API a few people have been asking for (in #131 which we're trying to merge today)
monica is working on design for the email client view, dark mode, analytics features and a lot of good issues will flow out of here

@mrlubos would you want to investigate this issue / #93?

monicakogler commented 2 years ago

https://github.com/sofn-xyz/mailing/issues/119 this would also be a good one to close out – i think the remaining work is short and sweet? outside of my wheelhouse 😬

izakfilmalter commented 2 years ago

Mailing works just fine within a monorepo. I have done the following to get it configured correctly.

Make a new package for your emails.
1. ```
cd packages
mkdir email
cd email
yarn init
```
2. Open the newly created package.json, and adjust the name to match your monorepo structure, eg:
```
{
  "name": "@mymonorepo/email"
}
```
3. Add needed dependancies:
```
yarn add mailing mailing-core mjml mjml-react nodemailer react react-dom
yarn add -D @types/mjml @types/mjml-react @types/nodemailer @types/react @zerollup/ts-transform-paths ttypescript typescript
```
  You will see that we have added both @zerollup/ts-transform-paths and ttypescript. We are gonna use both of these to rewrite our imports within our new @mymonorepor/email package to relative imports when we build the package.

Add tsconfig.json:

{
  "compilerOptions": {
    "composite": true,
    "noImplicitReturns": true,
    "noUnusedLocals": true,
    "jsx": "react-jsx",
    "outDir": "lib",
    "sourceMap": false,
    "strict": true,
    "target": "ES2018",
    "baseUrl": ".",
    "typeRoots": ["node_modules/@types", "./@types"],
    "rootDir": ".",
    "module": "commonjs",
    "esModuleInterop": true,
    "noEmit": false,
    "moduleResolution": "node",
    "declaration": true,
    "stripInternal": true,
    "forceConsistentCasingInFileNames": true,
    "skipLibCheck": true,
    "plugins": [
      {
        "transform": "@zerollup/ts-transform-paths",
        "exclude": ["*"]
      }
    ],
    "paths": {
      "@mymonorepo/email/*": ["*"]
    }
  },
  "compileOnSave": true,
  "include": ["."],
  "exclude": [
    "./lib",
    "node_modules",
    "node_modules/*",
    "./node_modules",
    "./node_modules/*",
    "../../node_modules",
    "../../node_modules/*"
  ]
}

You will need to edit the paths array to match your monorepo schema.

Time to finally init mailing.
```
npx mailing
```
This is gonna make a new directory for you within packages/email called emails. I find this to be redundant. I have copied all of these files out of emails into the root of packages/email. The above tsconfig.json is setup for this. If you want to keep everything in emails, change "include": ["."] to "include": ["./emails"].
(Optional) Rewire imports. This is totally optional but I hate relative imports. You can now safely redo all the imports in all the files that mailing has created for you to be absolute: @mymonorepo/email/.....

More package.json changes:

Add main, typings, and scripts

{
  "main": "lib/index.js",
  "typings": "lib/index.d.ts",
  "scripts": {
    "build": "rm -rf ./lib && yarn run ttsc",
    "start": "npx mailing preview"
  },
}

Add template exports to index.ts

export * from '@mymonorepo/email/TextEmail'
export * from '@mymonorepo/email/Welcome'

Add @mymonorepo/email as a dependency to your backend packages.
Build @mymonorepo/email
```
yarn build
```

You now have a package in your monorepo for mailing. It will build on save into packages/email/dist, and work as a package for any other package.

Notes:

You will want add "jsx": "react-jsx" to the tsconfig.json of any package that depends on @mymonorepo/email. You will also need to change the extension of any file that calls sendMail to .tsx. If you do not want to do this, you can call your templates as functions instead of jsx, eg Welcome({ name: 'Foo Bar' }).
The above process works for basically anything you want as a package. Want prisma as it's own package? Do the above. Want a next.js site, do the above.

mrlubos commented 2 years ago

@izakfilmalter what's your email? I can invite you to the repository where I reproduced the issue with monorepo. My example already has tsconfig.json in its folder but it doesn't work

izakfilmalter commented 2 years ago

@mrlubos I would compare your tsconfig to the one above. Just cause you have one doesn't mean it will "work". I've fought a bunch with em before.

psugihara commented 2 years ago

@izakfilmalter that's an awesome setup, so cool. One note...

I have copied all of these files out of emails into the root of packages/email.

This is the only piece I'd steer clear of. It may break if you have anything else in packages/email (config files, etc) since it's not something we explicitly test for. I manually tested this in #131 with npx mailing --emails-dir . and encountered some errors. Instead to avoid the redundancy, I'd recommend putting the templates in packages/emails/src with a mailing.config.json mapping emailsDir to that file.

izakfilmalter commented 2 years ago

@psugihara You are probably right that something could break, it's been find for me so far. Happy for you to take the above comment and work it into your docs / change it.

psugihara commented 2 years ago

That would be awesome! I think the README is getting a bit unwieldy. Want to start a docs directory in the root and add docs/monorepos.md ?

izakfilmalter commented 2 years ago

Ya, I'll open a pr with this some time this week.

pstoica commented 2 years ago

I think there are two separate use cases being discussed here. The last solution describes how to import the mailing package from another monorepo package. The original post (and what I'm looking for) is about using unbuilt monorepo packages within Mailing for things like formatters, theme tokens, etc. It doesn't seem to attempt Babel or Typescript for these packages.

izakfilmalter commented 2 years ago

Good point @pstoica. I am successfully importing my other packages into my mailing package, but not that they are be compiled into commonjs.

mrlubos commented 2 years ago

@psugihara some progress with the latest v0.7.1 release, hoping you'll be able to provide further pointers. My original theory was that mailing simply uses the wrong tsconfig.json config, since it can't possibly know about my paths. To test that, I modify the newly generated .mailing folder. Given a project like this

|- project/
|-- .mailing/
|--- tsconfig.json
|-- tsconfig.json # my file

I can modify .mailing/tsconfig.json by adding to it this line:

"extends": "../tsconfig.json"

This resolves path import problems! 🎉

But it has some issues. First, this cannot be expected of users as I'd consider it an advanced configuration. Second, .mailing is an artefact of the build process, so I can't rely on this file not changing. The solution here I believe is automatically picking up my tsconfig.json file and either using those settings or extending mailing config with the custom settings. There's an existing looksLikeTypescriptProject() method which I believe could be used to detect this.

It's a good start though. This change allows me to import any methods across the monorepo. However, since I'm using TypeScript, I am likely to have type definitions in my files. Specifically, I have this in the same file that exports my veryUsefulFunction() from the original example.

export enum Foo {
  Bar = 'bar'
}

Without this enum, everything works. When I add it, I get this error.

error - ../../../libs/project/src/lib/index.ts
Module parse failed: Unexpected token (4:7)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
| // import { stuff } from 'react-router-dom';
| 
> export enum Foo {
|   Bar = 'bar'

I spent some time digging around but wasn't able to figure out how to get rid of this error. I tried reverting from SWC to Babel config and using some presets, but ultimately I arrived at the same error. I also tried looking at the next.config.js file in the .mailing folder, but wasn't sure how to modify the Webpack config as I'm not sure what's needed. I am not sure where exactly should this loader be configured (if it's indeed a Webpack issue).

Moreover, I read around that this might be related to importing files from outside the project source folder. Even if it was, I've never had this issue with other apps, so I believe at this point it's just a matter of some configuration.

Are you able to offer any further insights that might help?

P.S. I see there's another tsconfig.json file in .mailing/src/tsconfig.json. I wasn't sure what that one does and modifying it didn't seem to yield any results.

alexfarrill commented 2 years ago

@mrlubos will you share your CodeAnywhere repo with me as well? I'm wondering if this could be resolved by changing some of the options passed to esbuild (like "external") here: https://github.com/sofn-xyz/mailing/blob/c3e24e0ad72e8bc6ec536c67c10ed7d8c6e7f006/packages/cli/src/commands/preview/server/setup.ts#L203

mrlubos commented 2 years ago

@mrlubos will you share your CodeAnywhere repo with me as well? I'm wondering if this could be resolved by changing some of the options passed to esbuild (like "external") here:

https://github.com/sofn-xyz/mailing/blob/c3e24e0ad72e8bc6ec536c67c10ed7d8c6e7f006/packages/cli/src/commands/preview/server/setup.ts#L203

Ugh it already got deleted 😔 I can try to create a new project later. Do you know any service where I can create monorepos and it won't delete my project after a few days?

sofn-xyz / mailing

Monorepo compatibility #129