Clarify Settings for Resolving Packages w/non-matching Folder Names

[zfalen]

Great package. But, ran into this while playing with firelink for the first time.

I have a mono-repo where @zfalen/some-package refers to an npm workspace at packages/some-totally-different-folder-name.

The root package.json looks like this

{
  "name": "@zfalen/my-repo-root-package",
  "author": "Zach Falen",
  "version": "1.0.0",
  "license": "ISC",
  "private": true,
  "devDependencies": { ...... },
  "workspaces": [
    "packages/*",
    "firebase-app/functions"
  ]
}

The packages/some-totally-different-folder-name/packages.json looks like

{
  "name": "@zfalen/some-package",
  "author": "Zach Falen",
  "version": "1.0.1",
   // .... rest of it 
}

Then, in firebase-app/functions/package.json we have

{
  "name": "functions",
  "dependencies": {
    "@zfalen/some-package": "^1.0.0",
    // .......
  },
  "fireDependencies": {
    "@zfalen/some-package": "../../packages/some-totally-different-folder-name"
  },
  // .... rest of it 
}

firelink deploy grabs my module and dumps it into firebase-app/functions/.packages/some-totally-different-folder-name - HOWEVER it patches the dependency to read "@zfalen/some-package": "file:./.packages/some-package".

While this weirdly worked out locally testing against the emulator, it resulted in a failure on deploying to Firebase. That error was predictably: Detailed stack trace: Error: Cannot find module '@zfalen/some-package'

I updated the name of the folder to match the name of some-package, and updated the path under fireDependencies, which made it work.

TLDR; Readme should reflect the requirement that folder names match package names, or script should use the correct folder name when mapping the file:./.packages/whatever in package.json :)

[Stradivario]

Actually folder name shouldn't match the package name.... it shouldn't be required.

If it happens than it sounds to me like a bug.

I will give it a shot to see if what you are seeing is correct or it happens due to some race condition...

[Stradivario]

You are right!

I just tested it out and it works like you say i think this is not correct and should be fixed!

I am on it just now.

Cheers !

[zfalen]

🍻 just confirmed with firelink --leave-changes again as well :)

This saved me colossal amounts of headache setting up a project today. Big fan of this simplistic approach until firebase finally supports npm workspaces / node_modules outside of the functions folder

[Stradivario]

Based on your problem i have found a way to remove the whole rsync logic and everything related to copying monorepo packages to a specific folder .packages.

I asked myself "Why do we need to copy packages folder to .packages folder?" since we could just install the local packages with npm install and after install we could execute deploy function which will directly deploy already installed packages. So it appears that if we leave the resolution of the packages to npm with the new mapped ones actually there is no need to do additional copy since the mono packages are already inside node_modules installed.

This means that i would need to deploy some MAJOR release of the library without many of the unnecessery logic related to copying packages.

I will think about these days and probably i will refactor everything since i am planning it from a long time.

So please for now use the correct namespacing like you already found out and in couple of days there will be a major release.

Thanks for the ticket it reminds me of some things that i needed to change!

[Stradivario]

These 2 PR's represent some of the changes

https://github.com/rxdi/firelink/pull/68 https://github.com/rxdi/firelink/pull/69

[Stradivario]

@zfalen is it possible to prepare some public repository with your changes to test it out realtime and not bother you to test it after i deploy the release ?

[zfalen]

Sure. Let me whip one up

[zfalen]

@Stradivario - try this https://github.com/zfalen/npm-ts-workspaces-firebase

I havent actually deployed the functions folder but i did get it working with npm install for workspace configuration and the tsconfig.json resolves the typings. I ran firelink --leave-changes and verified it worked.

I did not name the folder a different name than the package, although it is missing the @quramy scope which seems to be handled just fine by default.

[Stradivario]

Thank you very much for the repository.

I have made like 90% of the refactoring and deployed version 1.0.0-beta-1 in order to test the things out

Can you please try to install it using this command:

npm i -g @rxdi/firelink@1.0.0-beta-1

There is additional --help method to track all available arguments and also the library is using now commander as a main task runner.

Removed the whole rsync and copy logic which leaves the library with less and less code to break.

Also many of the issues related with rsync problems will be resolved since there is no more rsync :D :D

I have test it out against provided repository and it appears that it works flowless.

Sample command to use to prepare the environment it will execute npm install and will do the magic of mapping the fireDependencies and then will revert back the changes after the install

firelink --bootstrap --skip-runner

After the bootstrap step i think even firebase deploy or firelink deploy will do the job since we have already installed the correct packages!

There is a oneliner which will bootstrap(do a npm install with the local packages mapped) and execute deploy

firelink deploy --boostrap 

Will do a quick deployment in firebase using your provided repository to see if i will find some bugs that may occur.

Cheers!

[Stradivario]

Don't bother testing it since it will not work as expected :D

I forgot how stupid is the deployment of firebase function. I now remembered why i needed in the first place library like firelink

So in order for the monorepo packages to be deployed they should be in the folder where the lambda is else it will not be deployed and will lead to broken packages.

So basically i will need to add additional copy logic again to accomplish the purpose of the library... unfortunate.

[zfalen-deloitte]

:) yes it is frustrating. I am surprised they have not addressed this considering the adoption of monorepo structures.

[Stradivario]

@zfalen-deloitte

I can suggest you to do something which i am practicing a lot.

I am building a lambda infrastructure using Kuberenetes and will release soon to production it is part of a bigger features inside https://graphql-server.com website.

So what i am doing there is the following:

1. I am installing all packages inside the build step using firelink
2. Building the modules as single js file
3. Deploying lambdas to the platform as a single js file

I am doing this using esbuild with the following code

build.mjs file

import esbuild from 'esbuild';

esbuild
  .build({
    entryPoints: ['./src/index.ts'],
    bundle: true,
    treeShaking: true,
    platform: 'node',
    target: 'node16',
    outfile: './lib/index.js',
  })
  .then((data) => console.log('SUCCESS', data))
  .catch((e) => {
    console.error(e);
    process.exit(1);
  });

My build command looks as follow:

node build.mjs

The esbuild will grab the file and tree shake everything and will bundle it to single executable .js which is then deployed to the lambda.

This is how i deploy lambdas for less than 10 seconds without any downtime for bootstrapping.

This is one lambda being deployed and the npm install takes 17 seconds more than the build and deploy combined :D

Cheers and i hope you find this useful i will work more on the beta 1 firelink now :D

Cheers !

EDIT: This approach has a lot of benefits since the bootstrap of the lambda is faster, the response is faster due to smaller code to execute and you don't need to deal with the stupid firebase deploy :)

This approach is also applicable to firebase, serverless, etc.

[zfalen]

@Stradivario I like that quite a bit. Most of what I do daily is actually Kubernetes. Mostly AWS. Playing with firebase for a side project to get the EZPZ user auth and such. Half the point is just checking back in on where firebase is as a product from a usability standpoint.

Ironically enough for a different product at work, which is built from something like 18 microservices on K8s and deployed exclusively with helm - we are looking at what you just described to make lambdas easier using something like https://fission.io/

[Stradivario]

I really don't beleive it but you actually write down what i currently adopted before 2 years :D

And all of my lambdas are running on fission.

Can we talk about such a technologies in private ? Do you have some email or facebook account ?

https://www.facebook.com/ktachev/

This is me if you want to talk about some different stuff :)))

[zfalen]

@Stradivario love to. I don't have any social though because I am an internet hermit. I have a reddit? Discord? 🤣