Support running an app from a subdirectory

[jmorrell]

There is currently no supported way to run a node app from a directory within your git repo. You can work around it most of the time by using git subtree push but that wouldn't work if you have shared code outside of your directory.

A super common situation is to have the following repo structure:

├── shared-code
├── server
└── frontend-app

Being able to direct the buildpack to look in /server instead of / would be helpful for cases like this.

[Arrow7000]

Exactly what I need. Thanks @jmorrell!

[jmike]

@jmorrell here's a workaround. It ain't perfect but works...

1. Make sure to add package.json under server and frontend-app. List dependencies (as you normally do) and specify a start script for the server.

2. Create a root package.json file as follows:

{
  "name": "foobar-app",
  "version": "0.1.0",
  "scripts": {
    "postinstall": "npm install --prefix server && npm install --prefix frontend-app"
  }
}

Notice the --prefix argument which tells npm which subfolder to work on.

3. Add the following to your procfile:

web: npm start --prefix server

Hope that helps,

[jmike]

OK, I just realized you are working for Heroku, so this is probably not a real question, but rather a feature request. Sorry for the misunderstanding.

In any case, I will leave my answer above in case someone finds it useful.

[Arrow7000]

@jmike thanks for weighing in. Is this something you've tried yourself and has worked? Because in the absence of an official solution I'll give that a try.

[jmike]

@Arrow7000 yeap, works like a charm. The only problem is caching - see issue #387 even though it's somewhat unrelated to the application structure. See, I am using local files as npm dependencies.

In any case if you face a similar issue you can always disable caching according to heroku instructions https://devcenter.heroku.com/articles/nodejs-support#cache-behavior.

[Arrow7000]

Thanks @jmike that worked like a treat! It's not the cleanest solution, because it requires an extra package.json with important options duplicated - eg engines.node - but until we get an official solution this seems like the best and simplest way to get it done!

[gablg1]

We had been using git subtree push for a while but started facing the exact issue described above, so we implemented the following custom buildpack https://github.com/Pagedraw/heroku-buildpack-select-subdir

which allows us to deploy multiple apps from the same Heroku repo. Then we just make each of the frontend-app and server require the shared-code as an npm local dependency.

One caveat is that we have to explicitly add the node_modules folder installed to the NODE_PATH so npm knows where to look for requires within shared-code.

It works for us. Let me know if it also works for you!

[Jan0707]

@jmike I tried your approach and it sadly does not work for me. Here is my package.json

{
  "name": "App",
  "engines": {
    "node": "8.1.x"
  },
  "scripts": {
    "postinstall": "npm install --prefix app/Resources && app/Resources/node_modules/.bin/gulp --gulpfile app/Resources/gulpfile.js"
  }
}

It fails though when trying to run gulp:

remote: -----> Building dependencies
remote:        Installing node modules (package.json)
remote:
remote:        > App@ postinstall /tmp/build_98b4546541f7050670cac9ef04e8ace1
remote:        > npm install --prefix app/Resources && app/Resources/node_modules/.bin/gulp --gulpfile app/Resources/gulpfile.js
remote:
remote:        added 14 packages in 2.164s
remote:        sh: 1: app/Resources/node_modules/.bin/gulp: not found
remote:        npm ERR! file sh
remote:        npm ERR! code ELIFECYCLE
remote:        npm ERR! errno ENOENT
remote:        npm ERR! syscall spawn
remote:        npm ERR! App@ postinstall: `npm install --prefix app/Resources && app/Resources/node_modules/.bin/gulp --gulpfile app/Resources/gulpfile.js`
remote:        npm ERR! spawn ENOENT
remote:        npm ERR!
remote:        npm ERR! Failed at the App@ postinstall script.
remote:        npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
remote:
remote:        npm ERR! A complete log of this run can be found in:
remote:        npm ERR!     /app/.npm/_logs/2017-07-14T10_49_30_508Z-debug.log

Any idea on what I am doing wrong here would be highly appreciated :)

[atomkirk]

It appears that if you run the buildpack from within a subdirectory, the commands it's suppose to install are not available after release. My buildpack runs the heroku/php buildpack in a sub directory, but when I heroku run bash, which php returns nothing. Same goes if I cd into the directory.

[jmorrell]

@atomkirk The buildpacks install their dependencies globally, and I'm not sure what you mean by running the "php buildpack in a sub directory". Could you open a support ticket at https://help.heroku.com/ so we can sort out what's going wrong?

[atomkirk]

So it looks like they don't install them globally. When I deploy a root directory php app with heroku/php buildpack, then heroku run bash, and which php, its found in /app/.heroku/php/bin/php and echo $PATH is /app/.heroku/php/bin:/app/.heroku/php/sbin:/app/.heroku/php/bin:/app/.heroku/php/sbin:/app/.heroku/php/bin:/app/.heroku/php/sbin:/app/.heroku/php/bin:/usr/local/bin:/usr/bin:/bin:/app/vendor/bin

So this is actually an easy fix, just need to fix PATH so it points to /app/subdir/… instead of /app/…

Would be nice if the buildpack system provided an option so we didn't have to make our own buildpacks to work around it

[jmorrell]

@atomkirk Sounds like you're trying to do something that's not supported cc @dzuelke

[jmike]

@Jan0707 have you tried running gulp on postinstall from within the internal package.json file, i.e. app/Resources/package.json in your case?

Another idea might be to install gulp with root package.json and update your configuration as such:

{
  "name": "App",
  "engines": {
    "node": "8.1.x"
  },
  "dependencies": {
    "gulp": "^3.9.1"
  },
  "scripts": {
    "postinstall": "npm install --prefix app/Resources && gulp --gulpfile app/Resources/gulpfile.js"
  }
}

Please note that you don't really need to provide the qualified file path for gulp - npm knows what to do.

[dzuelke]

Why a subdirectory, @atomkirk? Or, at least, why does the PHP runtime have to be in a subdirectory, and not just your application? A lot of moving parts are set up to look at $HOME/.heroku/… for buildpacks, and that includes how PHP is built and where it looks for its config files, where binaries look for other .sos, and so forth. It's not something that's easily changed, and so far I don't understand the use case for it.

[atomkirk]

Well, thanks for the response, but @jmorrell is right, I'm trying to do something unsupported. I kind of took us off topic. Sounds like if you want to support the original issue, the buildpack will probably have to run in the subdirectory and then PATH point at where it put the .heroku folder (or if the buildpack puts the .heroku folder in the repo root, then change nothing)

[gkatsanos]

This feature is what I was looking for. I have a client and a server which have two independent package files, and I want them to be sibling directories and not nested. Therefore I need a way to instruct heroku to start from a specific package file which is not located at the root of my repository.

[ivanovivelin]

Hi, I seem to face the same issue when running under sub directory =>

"postinstall": "npm install --prefix server && npm install --prefix ./server/client",

The entire project is inside server folder , including the client ... any help

[jmorrell]

@IIvanov8888 could you open a support ticket at help.heroku.com?

[wmaciejak]

@gablg1 for me your solution doesn't work.

After valid deploy I see in console

app[client.1]: bash: npm: command not found. 

The same situation with node. Even If I change $PATH variable, the issue still occurs.

[ClayShentrup]

I have a similar issue. I want to run an Ember Fastboot app without having to make a separate repo for my API backend (which is Rails in my case). Ideally I'd be able to specify multiple web processes for a single app. Since I can't do that, I have to make two separate apps for the frontend and backend. But I don't want to create multiple repos for this. I just want to be able to tell one app to use one subdirectory and tell the other app to use the other subdirectory from the git project root.

[WeishiZeng]

As mentioned in the ticket, one can use git subtree push --prefix {app_subfolder} heroku master to push a subfolder only. It would only work if the subfolder is self-contained

[Bnaya]

@WeishiZeng can you please elaborate? :)

[WeishiZeng]

@Bnaya The git command is used to push a subfolder to remote. If the subfolder happens to be a self-contained app recognized by heroku, then it'll be deployed.

[javidjamae]

One note to add to the conversation is that the git subtree push solutions only work when you're directly pushing your app to Heroku, but it doesn't solve the problem when you're setting up a Heroku CI Pipeline. In that case, the filesystem is readonly and (AFAIK) there is no way to control what subdirectory of the code is used for the test execution, deployment, etc.

I encountered this issue when trying to use a monorepo with a node.js app as I describe here: https://stackoverflow.com/questions/51449750/how-do-i-get-heroku-ci-to-run-tests-when-using-a-monorepo

[kris-campos]

has there not been an official solution for this yet? is there a ticket somewhere that we can follow?

[jmorrell]

@kris-campos I haven't had a chance to loop back on this yet, so there is no official solution. In the meantime I would look at Yarn workspaces: https://yarnpkg.com/blog/2017/08/02/introducing-workspaces/ which I believe npm is also in the process of implementing.

This will be the first thing I explore as a potential solution. If you try them please report back with how it worked for you.

[danielmahon]

@jmorrell I just starting switching to yarn workspaces & lerna. So far using https://github.com/timanovsky/subdir-heroku-buildpack I can get the package to run properly but I'm unable to access a shared module because it bascially wipes everything outside the sub directory so shared-code doesnt exist and is a local module, not published. Did you figure out a good way to solve this?

├── shared-code
├── server
└── frontend-app

[danielmahon]

@jmorrell

UPDATE: Well I tried to avoid it but looks like I need to use https://www.npmjs.com/package/yalc instead of yarn link. Since the symlinks don't seem to always work properly inside Heroku. I could drop yarn workspaces completely for yalc and it would definitely simplify/remove most of these steps but the point of this was to try to use yarn workspaces so I'll leave it for now and just use yalc inside Heroku.

This is my current working deployment setup using a local lerna / yarn workspaces monorepo:

root
├── buildpack-run.sh
├── lerna.json
├── package.json
├── packages
│   ├── app
│   │   ├── package.json
│   ├── graphql
│   │   ├── package.json
│   ├── native
│   │   └── package.json
│   ├── shared
│   │   ├── package.json
│   └── site
│       ├── package.json
└── yarn.lock

First, I'm using these 3 buildpacks:

https://github.com/weibeld/heroku-buildpack-run
https://github.com/timanovsky/subdir-heroku-buildpack
https://github.com/danielmahon/create-react-app-buildpack

I use heroku-buildpack-run to run this script to copy the "sibling" shared package into a lib folder within the main package to be deployed, as well as the root level yarn.lock which while it contains ALL the package dependencies, yarn install should still only match the dependencies from package.json.

#!/bin/bash

echo "       Copying shared modules"

mkdir -p packages/app/lib/@myscope
cp -R packages/shared packages/app/lib/@myscope
cp yarn.lock packages/app
# repeat for other simultaneous deployments

I use subdir-heroku-buildpack which just pulls out and replaces your root build folder with one specified at PROJECT_PATH. This is why I needed to copy the shared package from the first step into this one.

I use a forked create-react-app-buildpack because I needed to update it's own dependancy of heroku-buildpack-nodejs with a forked version of my own https://github.com/danielmahon/heroku-buildpack-nodejs that simply adds the --ignore-optional flag to the default yarn install command.

Maybe we can set --ignore-optional with an env variable so the buildpacks don't need forked?

In my package.json for the deployed module, I have the following npm scripts to yalc add the shared package that's now in the lib folder.

  "scripts": {
    "heroku-postbuild": "npm-run-all -s yalc:publish yalc:add ",
    "yalc:publish": "cd lib/@myscope/shared && yalc publish",
    "yalc:add": "yalc add @myscope/shared"
    ...
  },

I also needed to set the shared package as an optional dependency since it is unpublished.

  "optionalDependancies": {
    "@myscope/shared": "*"
  },

As of now, running lerna version --exact --force-publish in the root of the monorepo, properly updates all the package versions, and performs a git push which triggers a new build on Heroku for two apps which share the same local sibling package.

This works but obviously requires much more setup than I would like, let me know if anyone sees anything that can be simplified (I suppose a dedicated buildpack would help), until there is a better option.

[jmorrell]

@danielmahon I'm unclear on why you need the subdir buildpack. With a package.json listing the workspaces at the root you could push the whole repo up and get all of your dependencies installed.

There are a couple of things that would be missing:

• support for build commands like heroku-postbuild, but you could add the scripts at the root
• no automatically picking up the npm start command in the subdirectory, so this command needs to live at the package root

If you open a support ticket, I can help you simplify this setup

[jmorrell]

re: using yarn workspaces more generally

I got a support ticket recently from a user who had multiple services within the same repo, using yarn workspaces. They used an env var in each app ($APP_DIR) to direct that dyno to start that service.

One drawback to this is that it installs all of your dependencies for all of your applications, which was ballooning their app size over Heroku's limits. Here was the workaround we found.

tl;dr - delete all of the directories not needed by the app in $APP_DIR

---

Our temporary workaround was to delete folders we didn’t need in a heroku build script in our package.json. This works OK, but it means we have to maintain a list of folders to delete in each service which is kind of a pain.

To resolve this, you can get a map of how each workspace depends on the others using yarn workspaces info. In this example workspace-b depends on workspace-a, but workspace-a and workspace-c are independent:

❯ yarn workspaces info       
yarn workspaces v1.9.4
{
  "workspace-a": {
    "location": "workspace-a",
    "workspaceDependencies": [],
    "mismatchedWorkspaceDependencies": []
  },
  "workspace-b": {
    "location": "workspace-b",
    "workspaceDependencies": [
      "workspace-a"
    ],
    "mismatchedWorkspaceDependencies": []
  },
  "workspace-c": {
    "location": "workspace-c",
    "workspaceDependencies": [],
    "mismatchedWorkspaceDependencies": []
  }
}
✨  Done in 0.04s.

You could create a heroku-prebuild Node script that uses this output to delete any directories not required by $APP_DIR. Here is a quick sketch to what that might look like:

"scripts": {
  ...
  "heroku-prebuild": "node remove-workspaces.js"
}

remove-workspaces.js

const { exec } = require('child_process');
const app = process.env['APP_DIR']

exec('yarn workspaces info --json', (err, stdout, stderr) => {
  const output = JSON.parse(stdout);
  const info = JSON.parse(output.data);

  const dependencies = gatherDependencies(info, app);
  const unneeded = Object.keys(info).filter(i => !dependencies.includes(i));

  unneeded.forEach(i => exec(`rm -rf ${i}`));
});

// Gather all of the workspaces that `workspace` depends on
function gatherDependencies(info, workspace) {
  let deps = [workspace];
  let ws = [workspace];
  while (ws.length) {
    info[ws[0]].workspaceDependencies.forEach(w => {
      ws.push(w);
      deps.push(w);
    });
    ws.shift()
  }
  return deps;
}

This should leave only the workspaces needed by $APP_DIR, and yarn will only install the dependencies needed by that directory.

[jmorrell]

re: addressing this issue in a first-class way

Once npm adds support for workspaces making this first-class will become a top priority.

In the mean-time if you run into issues setting things up with workspaces, please comment here or open a support ticket, and let's work through what you found difficult or what edge cases you hit.

[danielmahon]

@jmorrell Thanks for the info! I will give your suggestions a try when I get a sec and let you know how it goes.

[danielmahon]

The much simpler setup for using a lerna monorepo with yarn workspaces and, in this case, create-react-app thanks to the previous suggestions by @jmorrell 🥇 !

In this example @scope/app is a create-react-app app, and @scope/server is a standard node server (like graphql-yoga). If you're not using CRA just ignore/remove those parts. When a new commit is pushed to a Heroku pipeline, both app and server will be deployed to their respective dynos based on the APP_WORKSPACE env variable. Both app and server can require the shared and "unpublished" @scope/shared module.

App Buildpack: mars/create-react-app (which also uses heroku/nodejs) Server Buildpack: heroku/nodejs (default)

App env APP_WORKSPACE=@scope/app Server env APP_WORKSPACE=@scope/server

root
├── build-workspaces.js
├── remove-workspaces.js
├── static.json
├── lerna.json
├── package.json
├── packages
│   ├── app
│   │   ├── package.json
│   ├── server
│   │   ├── package.json
│   ├── shared
│   │   ├── package.json
└── yarn.lock

$ > yarn workspaces info

{
  "@scope/app": {
    "location": "packages/app",
    "workspaceDependencies": [
      "@scope/shared"
    ],
    "mismatchedWorkspaceDependencies": []
  },
  "@scope/server": {
    "location": "packages/server",
    "workspaceDependencies": [
      "@scope/shared"
    ],
    "mismatchedWorkspaceDependencies": []
  },
  "@scope/shared": {
    "location": "packages/shared",
    "workspaceDependencies": [],
    "mismatchedWorkspaceDependencies": []
  }
}

package.json (root)

{
  "private": true,
  "workspaces": {
    "packages": [
      "packages/*"
    ]
  },
  "scripts": {
    // Required by create-react-app buildpack (this runs after heroku-postbuild)
    "build": "echo Packages were built in previous step...",
    // Removes unused workspaces for deployment
    "heroku-prebuild": "node remove-workspaces.js",
    // Builds target workspace and its workspaceDependencies
    "heroku-postbuild": "node build-workspaces.js"
  },
  "engines": {
    "node": "8.11.3",
    "yarn": "1.9.4"
  }
}

remove-workspaces.js

const { exec } = require('child_process');
const app = process.env['APP_WORKSPACE'];

exec('yarn workspaces info --json', (err, stdout, stderr) => {
  const output = JSON.parse(stdout);
  const info = JSON.parse(output.data);

  const dependencies = gatherDependencies(info, app);
  const unneeded = Object.keys(info)
    .filter(i => !dependencies.includes(i))
    // Notice we are referencing the "location" here, not the package name
    .map(key => info[key].location);

  console.log('\t', '----->', 'Pruning unused workspaces:', unneeded);
  unneeded.forEach(i => exec(`rm -rf ${i}`));
});

// Gather all of the workspaces that `workspace` depends on
function gatherDependencies(info, workspace) {
  let deps = [workspace];
  let ws = [workspace];
  while (ws.length) {
    info[ws[0]].workspaceDependencies.forEach(w => {
      ws.push(w);
      deps.push(w);
    });
    ws.shift();
  }
  return deps;
}

build-workspaces.js

const { exec, spawn } = require('child_process');
const app = process.env['APP_WORKSPACE'];

exec('yarn workspaces info --json', (err, stdout, stderr) => {
  const output = JSON.parse(stdout);
  const info = JSON.parse(output.data);

  const dependencies = gatherDependencies(info, info[app]);

  console.log('\n', '----->', 'Building workspaces:', app.split(','));

  dependencies.forEach(wp => {
    const build = spawn('yarn', ['build'], {
      cwd: __dirname + '/' + wp.location,
    });
    build.stdout.on('data', data => {
      console.log(data.toString());
    });
    build.stderr.on('data', data => {
      console.log(data.toString());
    });
    build.on('error', error => {
      throw error;
    });
  });
});

// Gather all of the workspaces that `workspace` depends on
function gatherDependencies(info, workspace) {
  let deps = [workspace];
  let ws = [workspace];
  while (ws.length) {
    ws[0].workspaceDependencies.forEach(w => {
      ws.push(info[w]);
      deps.push(info[w]);
    });
    ws.shift();
  }
  return deps;
}

static.json (placed in root)

{
  "root": "packages/app/build/",
 // ...
}

I think I included all the important parts, I had to customize things a bit more for my specific needs but overall this is definitely easier than my previous method.

[alexpetralia]

Hi was there ever a solution here? Or is the only solution: https://github.com/heroku/heroku-buildpack-nodejs/issues/385#issuecomment-422981921

Running npm install --prefix server && npm install --prefix frontend-app did not work for me because Vue CLI 3 uses vue-cli-service from node_modules, and this is not read since it it is in a subdirectory.

[alexpetralia]

I figured out a really hacky but simple solution if this works for anyone else.

1. Include this in your Procfile: release: cd <YOUR_APP> && npm install && npm run build

Here you cd into your subdir, npm install because otherwise it will throw an error saying it doesn't have node_modules, and finally npm run build to build your source files

Heroku will say you need a package.json in your root.

2. Include a package.json with the contents: { } (ie. empty)

Now Heroku will detect your package.json, move on with the build, and later execute the npm commands in your Procfile.

Edit: Also, if it is relevant, I have NPM_CONFIG_PRODUCTION set to False

[jmorrell]

Hi @alexpetralia!

Hi was there ever a solution here?

The solution will likely be some additional functionality around workspaces. Link to yarn docs: https://yarnpkg.com/lang/en/docs/workspaces/

In the case where you have two apps in two directories: server and frontend-app, you could add a package.json at the root that looked like this:

{
  "workspaces": ["server", "frontend-app"]
  // additional config like node, npm version or global build scripts would go here
}

and the buildpack would take care of making sure those dependencies are installed, and the build scripts executed in each of those workspaces.

I want to ship this change first: https://github.com/heroku/heroku-buildpack-nodejs/issues/583 so that your example would work with no additional configuration of build scripts.

Additionally, npm has not yet shipped support for workspaces, but it is on their roadmap for the next few months. My fear with supporting them now is that the implementation might subtly differ from npm's implementation once it ships.

For now I recommend using yarn workspaces, and adding a build script with cd commands

[jmorrell]

@alexpetralia

1. Include this in your Procfile: release: cd <YOUR_APP> && npm install && npm run build

I am unsure how this is working for you. It is not possible to modify your built image during release phase.

https://devcenter.heroku.com/articles/release-phase#design-considerations

Not suggested for asset compilation or other tasks requiring filesystem persistence

Release phase is not suitable for tasks that require filesystem persistence, as filesystem changes during release phase will not be deployed to your app’s dyno formation (the dyno filesystem is ephemeral). Asset compilation should happen during builds. Release phase can be used to upload the compiled assets to a CDN.

This will work if you add cd <YOUR_APP> && npm install && npm run build as a heroku-postbuild script in your root package.json:

{
  scripts: {
    "heroku-postbuild": "cd <YOUR_APP> && npm install && npm run build` as a `heroku-postbuild"
  }
}

[alexpetralia]

Interesting. I'm reluctant to touch what's working for me now but should it break in the future, I'll try your fix. Thanks for checking however.

On Mon, Feb 18, 2019, 19:40 Jeremy Morrell <notifications@github.com wrote:

@alexpetralia https://github.com/alexpetralia

1. Include this in your Procfile: release: cd && npm install && npm run build

I am unsure how this is working for you. It is not possible to modify your built image during release phase.

https://devcenter.heroku.com/articles/release-phase#design-considerations

Not suggested for asset compilation or other tasks requiring filesystem persistence

Release phase is not suitable for tasks that require filesystem persistence, as filesystem changes during release phase will not be deployed to your app’s dyno formation (the dyno filesystem is ephemeral). Asset compilation should happen during builds. Release phase can be used to upload the compiled assets to a CDN.

This will work if you add cd && npm install && npm run build as a heroku-postbuild script in your root package.json:

{

scripts: {

"heroku-postbuild": "cd <YOUR_APP> && npm install && npm run build` as a `heroku-postbuild"

}

}

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/heroku/heroku-buildpack-nodejs/issues/385#issuecomment-464933576, or mute the thread https://github.com/notifications/unsubscribe-auth/AJhyei4WHqeSKdffNrKTI9W7hqQ6_AI4ks5vO0fxgaJpZM4MsQgS .

[alexpetralia]

Ok - for posterity. @jmorrell perhaps unsurprisingly was right: the release phase specified in your Procfile does not share the same file system as the implicit build phase which executes before your Procfile.

In other words, running npm build run or python manage.py collect-static in the release line of your Procfile does nothing because those file system operations are essentially run in a different file system (which is discarded). When you heroku run bash, you'll notice that there are no folders for dist or staticfiles because those were created during the release phase and discarded. (At least this is my understanding).

So that means, as @jmorrell said, any file system operations MUST be done during the implicit build phase (before the Procfile).

This means that:

(1) npm run build must be run first (2) Then python manage.py collectstatic must be run after

Which means that the Python buildpack must run AFTER your nodejs buildpack. You need to fix the order in your Heroku buildpack settings.

So now our nodejs buildpack will run first, see the package.json file, which according to @jmorrell looks like this:

{
  scripts: {
    "heroku-postbuild": "cd <YOUR_FRONTEND> && npm install && npm run build"
  }
}

Hopefully this buildpack runs successfully - it installs all your node_modules (npm install) and then builds your src (npm run build).

Then we move onto the python buildpack. Here we need to make sure that DISABLE_COLLECTSTATIC=0 so that django will pull files from your dist folder into whatever your STATIC_ROOT is, probably something like staticfiles.

Then and only then will all your buildpacks be done, Heroku will move onto your Procfile, and then execute your release and/or web phases.

All this being said, this still seems like a much easier solution (though getting here wasn't easy) than the whole workspaces deal above. Here we just drop our frontend in a subfolder, add a package.json to our root directory and finally make sure our buildpacks are in order.

Anyway, that's what worked for me. Hope it helps.

[87vrvk9k]

@danielmahon You're a god, tried so many different buildpacks before stumbling upon this. Builds seem quick with this approach too.

[isaachinman]

Since the symlinks don't seem to always work properly inside Heroku

Is this issue being tracked anywhere? We should be able to use npm link and yarn link in Heroku contexts - this workflow is very typical for package/library development.

[LawJolla]

It's crazy to me how hard it is to run apps from a monorepo with shared code. I had to modify @danielmahon 's code (thank you!) because my shared code dependencies are themselves a tree which depend on order.

My question is if I have two static websites, it seems like it's not possible to run a monorepo on Heroku because you're only allowed one static.json file and root does not take environment variables.

Is that correct? Or is there a way to run multiple static websites from a monorepo?

[jmorrell]

Or is there a way to run multiple static websites from a monorepo?

@LawJolla a Heroku app can only have one service bind to port 80 and respond to web traffic. There may be hacky ways of solving this problem, but no official one.

One way others have worked around this is by creating multiple apps with different env vars to point to different sub-directories. You can do this within the same Pipeline: https://devcenter.heroku.com/articles/pipelines#multiple-apps-in-a-pipeline-stage

[jmorrell]

Since the symlinks don't seem to always work properly inside Heroku Is this issue being tracked anywhere?

@isaachinman @danielmahon I don't know why symlinks would not work properly, but I would be happy to dig into a repro if you have one.

[LawJolla]

Or is there a way to run multiple static websites from a monorepo?

@LawJolla a Heroku app can only have one service bind to port 80 and respond to web traffic. There may be hacky ways of solving this problem, but no official one.

One way others have worked around this is by creating multiple apps with different env vars to point to different sub-directories. You can do this within the same Pipeline: https://devcenter.heroku.com/articles/pipelines#multiple-apps-in-a-pipeline-stage

Thanks @jmorrell ,

I am only having one service bind to port 80. In fact I have four services binding. --packages -admin-client -admin-server -customer-client -customer-server static.json

The servers are running fine.

The problem is both clients are create react apps. And apparently Heroku only allows one static.json file, so I can only specify root to one of them, e.g. "packages/admin-client/build"

How do I get "packages/customer-client/build" to route correctly?

Again, it's absolutely wild that it's this hard to deploy a Yarn Workspaces project.

[jmorrell]

The problem is both clients are create react apps. And apparently Heroku only allows one static.json file, so I can only specify root to one of them, e.g. "packages/admin-client/build"

This is more a concern of the create-react-app buildpack or the underlying heroku-buildpack-static which configures nginx for you. These buildpacks are designed with one app in mind, not monorepos, so it's not surprising that they run into issues in that context.

You can also use the serve module or express.static (or many others) to have more control over how static assets are served.

Though you will still only be able to run one service that serves web traffic per app. This is a long-standing platform design / limitation.

Again, it's absolutely wild that it's this hard to deploy a Yarn Workspaces project.

I have my own list, but what would you like to see change here?

[isaachinman]

@jmorrell Unfortunately I don't have time to put together a minimal repro, but next-i18next at this release will reproduce the issue.

We've got a nested example app which attempts to consume the top-level package via yarn link in build scripts.

If you build and run this via Heroku, you'll see the error.

[LawJolla]

@jmorrell

I put node in-front of the create react app for the work around.

I have my own list, but what would you like to see change here?

Zeit Now is doing a great job. https://zeit.co/examples/monorepo/ . However, they're only stateless servers, and I need stateful servers.

So, here's my dream list: A Yarn workspaces buildpack. That buildpack will understand the packages, how to build them if needed (probably just calling yarn build as it does now), and how to cache the modules (I have a 10 minute build every time because Heroku isn't caching any of the build). Then a config file that links the app packages to a heroku app, e.g.

{
  apps: [{ package: "@namespace/app1", app: "@app-heroku-app"}...]
}

And then the big trick... figure out what diffed from the code push and deploy accordingly. For instance, if @namespace/shared-code changed, then every app that depends on shared-code is rebuilt and deployed. If both @namespace/app1 and @namespace/app2 changed, both are built and deployed.

Thanks for your time and consideration!

[jmorrell]

how to cache the modules (I have a 10 minute build every time because Heroku isn't caching any of the build)

This is another work-around, but you could likely benefit by setting custom cache directories: https://devcenter.heroku.com/articles/nodejs-support#custom-caching

I'm experimenting with better caching strategies which should help here.

A Yarn workspaces buildpack.

I largely agree with your vision there. Thank you for the feedback

[jtushman]

@jmorrell -- I am running the same setup as many of the people on this thread: yarn workspaces

➜ heroku buildpacks
=== quala-success Buildpack URLs
1. https://github.com/timanovsky/subdir-heroku-buildpack
2. heroku/nodejs

My package structure:

app/
   package.json
   client/
   backend/       (a yarn workspace)
       package.json
       Procfile
       utils/
       data/
       server/

server/package.json

  "scripts": {
    "heroku-postbuild": "yarn workspace utils build && yarn workspace data build  && yarn workspace server build && yarn workspace cli build"
  },

Procfile

web: yarn workspace server start

I got everything running nicely EXCEPT -- the node_modules caching.

in my root package.json I have:

  "cacheDirectories": [
    "node_modules",
    "backend/node_modules"
  ],

but keep saying that backend/node_modules is empty -- and keeps on fetching everything on each deploy, and takes greater than 2 mins. Would love some help tuning this.

I do have an open ticket logged with support