Improve Docs for monorepo setup

[givethemheller]

In the process of using this package in a Lerna MonoRepo I believe I have a good use case for an external dependency mode.

Note - I did read the MD file on why you made this choice. In the detailed description, I present a proposal to justify this use case.

Side bar: Happy to do the work and contribute this. Love your package. Would like to treat this as a discussion.

Sorting

• I'm submitting a ...

  • [ ] bug report
  • [X] feature request
  • [ ] support request

• I confirm that I

  • [x] used the search to make sure that a similar issue hasn't already been submit

Expected Behavior

When I set an "externalDependencies" CLI flag, TSOA crawls my node modules to build a complete type file for the composition that is generated through the dependency tree. As the developer/consumer, I manage the compile time cost of this independently.

Current Behavior

I have to duplicate typings, by hand, through all levels of the types dependency tree. Further, I have to maintain those during upgrades.

Possible Solution

Started looking at the TS API - but its new.

Detailed Description

Use Case

I have recently started building a monorepo out to host a project that has grown to deserve it. I'm also breaking up my express servers into multiple smaller services. They still share the same data structure and to support that, I've created a shared resources library. It includes my models and a few things for authentication and various other tasks.

Discussion

You make two arguments against external dependencies.

1. You don't want to slow down the compile time.
2. You don't want API interfaces changing on you unexpectedly.

I propose that creating a flag and a corresponding spelunking method to fetch those types would make this a much more useful package. Creating Common libraries shared across packages is frequently done and in large enterprise applications.

1: If I have control over it, I can be expeditious with how I use typings. I can actively look at the types I am importing and control this outcome manually.

2. I think this runs somewhat counter intuitively to semantic versioning. These are all typescript projects being compiled, so if my dependency changes its interfaces without a breaking change version bump, someone has dropped the ball. More over, even if they fail to document a breaking change, anywhere that I am using that imported type, I am going to get compile time errors.

I actually want all of my dependent packages to dictate my dependent interfaces and I want my builds to break when interfaces change on me unexpectedly.

A Corner Case: There is a use case where this is still problematic. If I pull an interface into a server and never operate on that interface programmatically, I wouldn't see a improperly versioned breaking change. What I would see though, is my swagger spec changing on generation before committing it to master.

I would rather be responsible for improperly versioned breaking changes in dependencies than have to manage multiple copies of my model interfaces in multiple servers. From a time and error perspective, this could be much more costly than the aforementioned issue.

For the moment, I'll work around this by keeping the latest versions all of my servers and duplicating my models into the consumer servers.... but I really really want to be able to maintain independent version control. If I make a model change that only affects one api-server consuming it, I want to be able to version bump it on its own instead of having to bump all of my api-servers.

[dgreene1]

Sound alike a good proposal. I’d be interested in seeing a PR to see what’s possible. But I want to defer to my peer.

@WoH what are your thoughts?

[WoH]

I'm kinda out of the loop here I think. Any nodes that typescript considers within the program should be usable. Can someone provide a minimal repo that reproduces the issue here? How are you importing your Controllers?

We are sharing interfaces and types across services using our npm registry and I have not had issues. However, I'm pretty sure all of our services are running against 3.x, and our entrypoint is a factory that tsoa can scan for Controllers, so that may be the difference here?

[givethemheller]

Gonna get you all a demo branch today. Just cut a PR for you on a little issue I ran into yesterday with multiple servers in the 3.0 spec. Wanted to bump to that before moving forward.

@WoH How are you sharing your types and interfaces? Are they a separate package that you build out with just types to an npm package? In my case, I have my interfaces with my mongoose schemas and some other functional code I want to share

I think the problem here is that TS treats the packages in a mono repo as separate and outside of the program root. e.g. in the example below, you use a import {aModel} from "@project/package2" in package1 and tsoa is seeing that as being an external import.

package.json -- packages ---- package 1 ------ package.json ------ tsconfig.json ---- package 2 ------ models -------- aModelSchema.ts ------ authentication -------- sharedAuthMode1.ts -------- sharedAuthMode2.ts ------ package.json ------ tsconfig.json

Once I get the mono repo demo up, it should be easier to discuss what I may be doing wrong or how it "could" be done in the mono repo setup

[givethemheller]

also - for sake of noting it, I believe this is where the decision is being made to reject an imported type... though im not completely certain of that. First time looking at code that uses the TS compiler.

    while (leftmost.parent && leftmost.parent.kind === ts.SyntaxKind.QualifiedName) {
      const leftmostName = leftmost.kind === ts.SyntaxKind.Identifier ? (leftmost as ts.Identifier).text : (leftmost as ts.QualifiedName).right.text;
      const moduleDeclarations = statements.filter(node => {
        if (node.kind !== ts.SyntaxKind.ModuleDeclaration || !this.current.IsExportedNode(node)) {
          return false;
        }
        const moduleDeclaration = node as ts.ModuleDeclaration;
        return (moduleDeclaration.name as ts.Identifier).text.toLowerCase() === leftmostName.toLowerCase();
      }) as ts.ModuleDeclaration[];
      if (!moduleDeclarations.length) {
        throw new GenerateMetadataError("No matching module declarations found for ${leftmostName}.");
      }
      if (moduleDeclarations.length > 1) {
        throw new GenerateMetadataError("Multiple matching module declarations found for ${leftmostName}; please make module declarations unique.");
      }
      const moduleBlock = moduleDeclarations[0].body as ts.ModuleBlock;
      if (moduleBlock === null || moduleBlock.kind !== ts.SyntaxKind.ModuleBlock) {
        throw new GenerateMetadataError("Module declaration found for ${leftmostName} has no body.");
      }
      statements = moduleBlock.statements;
      leftmost = leftmost.parent as ts.EntityName;
    }
    return statements;
  } ```

If this line of code was removed, would that cause the resolver to accept all types? 

`
         if (node.kind !== ts.SyntaxKind.ModuleDeclaration || !this.current.IsExportedNode(node)) {
          return false;
        }
`

It looks to me like TS has already loaded the typing in full when it parsed the files, but again.. new to the ts compiler?

[WoH]

@WoH How are you sharing your types and interfaces? Are they a separate package that you build out with just types to an npm package? In my case, I have my interfaces with my mongoose schemas and some other functional code I want to share

For the sake of the argument, it's just like a typings package from npm that exports types.

I believe your issue might caused here:

package.json
-- packages
---- package 1
------ package.json
------ tsconfig.json
---- package 2
------ models
-------- aModelSchema.ts
------ authentication
-------- sharedAuthMode1.ts
-------- sharedAuthMode2.ts
------ package.json
------ tsconfig.json

If your package 2's tsconfig only includes files in package2/*, tsoa will use that config and not find definitions in package1/*. If you fix that any exports from package1 should be available in package2.

[givethemheller]

Ah, makes sense. The only thing it really runs afoul of at this point is the independent versioning. I suspect that I could just include the particular package I am interested, that's located in my node modules and have access to it just the same. I'll give that a spin and add some details to the docs if it works.

[dgreene1]

We would not accept a tsoa configuration that allows you to specify the version number of a package. Tsoa would need to respect the version in the lock file and if that isn’t present then I’m not really sure how tsoa should decide....?

How does TypeScript resolve which folder? I’m assuming it takes the root folders only...?

[givethemheller]

Following up from WoH's recommendations. Adding access to the particular file of interest in my tsconf.json worked. "include": [ "./src/**/*", "./src/tests", "./node_modules/@cannabinder/api-shared/src/index.ts" ],

With reference to the local node_modules for the package in the monorepo, you get the version the package points to.

If you reference the package through the root of the project you will get the current version in the mono repo and not the specified version. That could be fine. "packages/api-shared/src/index.ts"

I'd like to leave this ticket open until I can get an update to the Docs open. Thanks @WoH

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[fantapop]

This isn't working for me. No files are found in ./node_modules/ for my subpackage because they're all hoisted to the root by yarn workspaces. If i try to reference them at the monorepo root node_modules, it finds the files but complains that its out of the rootDir. Any tips here? My use case is similar to @givethemheller . I was planning to share a data access layer between 2 of my subpackages so I was trying to migrate those files into the shared subpackage.

[givethemheller]

@fantapop Happy to lend a hand.

Setup in yarn workspaces.

at Root of package lerna.json

{
  "lerna": "3.2.0",
  "npmClient": "yarn",
  "useWorkspaces": true,
  "version": "independent"
}

package.json

  "scripts": {
    "bootstrap": "lerna bootstrap",
    "build:watch": "lerna run build:watch --parallel",
    "build": "lerna run build --stream ",
    "test": "lerna run test",
    "clean": "rimraf packages/**/lib",
    "pub": "lerna publish"
  },
  "devDependencies": {
    "@types/node": "^13.9.1",
    "ava": "^3.5.0",
    "lerna": "^3.20.2",
    "rimraf": "^3.0.2",
    "typescript": "3.8.3"
  },
  "workspaces": {
    "packages": [
      "apis/*",
      "apps/*",
      "sdks/*"
    ],
    "nohoist": [
      "typescript",
      "**/typescript/**",
      "**/typescript"
    ]
  }

If you wanna drop a link to a repo, I can take a look at do some comparing. If your on slack or gchats, could talk there too. tom@cannabinder.com

[givethemheller]

oh and oh jeeeee golly is everything working, including my SDK generation.

[fantapop]

I was able to get this working. I didn't need to add the node_module reference in my tsconfig.json file however. @givethemheller are you certain this is necessary? The issue I was having is that the path-mapping hadn't been setup yet in tsoa.json. https://tsoa-community.github.io/docs/path-mapping.html

[juiceo]

@givethemheller @fantapop I'm still struggling to get tsoa to work in a lerna monorepo. Do you maybe have a repo to share with your setup?

Currently running into a brick wall with yarn run tsoa routes, gives the following TypeError:

Generate routes error.
 TypeError: Cannot read property 'text' of undefined

[fantapop]

@juiceo I don't have a repo I can share at this time. Is your project available? I could take a peak. Are you able to get a non-lerna tsoa repo working? Are you specifically having issues with types that are stored outside of tsoa directory?

[givethemheller]

I can get something up this weekend. Just need to do a once over and make sure nothing bleed over from the project that spurred it.

If your lucky it will also have a react native app working in it too

On Fri, Jun 19, 2020 at 7:57 PM Christopher Fitzner < notifications@github.com> wrote:

@juiceo https://github.com/juiceo I don't have a repo I can share at this time. Is your project available? I could take a peak. Are you able to get a non-lerna tsoa repo working? Are you specifically having issues with types that are stored outside of tsoa directory?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lukeautry/tsoa/issues/592#issuecomment-646926820, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACH5LTWFNPQXBAMNDDVCTS3RXQQSFANCNFSM4KYTPOTA .

[givethemheller]

@fantapop @juiceo Here's what I have been working on. Please use what ever helps you from here. https://github.com/givethemheller/helium

[dasdachs]

@fantapop having same issues here, types are in the packages/shared folder, the server is in packages/server and I get the ...No matching model found for referenced error.

Do we have any example implementation for a yarn/lerna monorepo setup?

[sethmbrown]

Any updates here? Struggling to get a yarn monorepo set up with a shared type library