ember-cli-deploy / ember-cli-deploy-revision-data

An ember-cli-plugin to create a unique revision key for the build
MIT License
22 stars 57 forks source link

Git info is generated in the dist dir #52

Open marcoow opened 7 years ago

marcoow commented 7 years ago

The Git info is currently generated in the dist dir which causes problems if that dir is outside of the repo.

In our case we're deploying from a Docker container and mount the host's tmp dir as the tmp dir in the container. As that dir is a volume then and no regular folder any git operations performed in that dir won't work, leading to the error:

fatal: Not a git repository (or any parent up to mount point /usr/src/app/tmp)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

/usr/src/app/node_modules/ember-cli-deploy-revision-data/lib/scm-data-generators/git.js:16
        var info = log.latest;
                      ^

TypeError: Cannot read property 'latest' of null
    at /usr/src/app/node_modules/ember-cli-deploy-revision-data/lib/scm-data-generators/git.js:16:23
    at Git.<anonymous> (/usr/src/app/node_modules/simple-git/src/git.js:1276:13)
    at Git.<anonymous> (/usr/src/app/node_modules/simple-git/src/git.js:1192:21)
    at emitTwo (events.js:106:13)
    at ChildProcess.emit (events.js:191:7)
    at maybeClose (internal/child_process.js:885:16)
    at Socket.<anonymous> (internal/child_process.js:334:11)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at Pipe._handle.close [as _onclose] (net.js:501:12)
error Command failed with exit code 1.

The error in git.js being a consequence of the first 2 lines really:

fatal: Not a git repository (or any parent up to mount point /usr/src/app/tmp)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

export GIT_DISCOVERY_ACROSS_FILESYSTEM = 1 actually does fix the problem for us but it seems suspicious that the git info is generated from the dist dir as opposed to the root of the repo or working directory that ember deploy is run from.

achambers commented 7 years ago

I suspect that this line should probably pass the project root (context.project.root() from memory) in as the path as opposed to the dist dir. I can't think of any reason why the root of simpeGit should start at the dist dir

mcfiredrill commented 7 years ago

I'm not sure but maybe I'm running into this same issue trying to use the ember js buildpack on heroku?

remote:                                                                                                                                                                  [213/1804]
remote: moment-timezone#0.4.1 bower_components/moment-timezone
remote: └── moment#2.10.6
remote:
remote: emojione#2.1.4 bower_components/emojione
remote: -----> Caching bower cache
remote: -----> Building ember assets
remote: DEPRECATION: Overriding init without calling this._super is deprecated. Please call `this._super.init && this._super.init.apply(this, arguments);` addon: `undefined`
remote:     at Object.<anonymous> (/tmp/build_ab268438ea5477419a8de38efdbb1282/node_modules/ember-cli-deploy/lib/tasks/read-config.js:9:23)
remote: DEPRECATION: Overriding init without calling this._super is deprecated. Please call `this._super.init && this._super.init.apply(this, arguments);` addon: `undefined`
remote:     at Object.<anonymous> (/tmp/build_ab268438ea5477419a8de38efdbb1282/node_modules/ember-cli-deploy/lib/tasks/pipeline.js:5:23)
remote: DEPRECATION: Overriding init without calling this._super is deprecated. Please call `this._super.init && this._super.init.apply(this, arguments);` addon: `undefined`
remote:     at Object.<anonymous> (/tmp/build_ab268438ea5477419a8de38efdbb1282/node_modules/ember-cli-deploy/lib/tasks/deploy.js:4:23)
remote: [BABEL] Note: The code generator has deoptimised the styling of "datafruits13/emojiStrategy.js" as it exceeds the max of "100KB".
remote: [WARN] (broccoli-uglify-sourcemap) Minifying: `assets/vendor.js` took: 23938ms (more than 20,000ms)
remote: cleaning up...
remote: fatal: Not a git repository (or any parent up to mount point /tmp)
remote: Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
remote:
remote: /tmp/build_ab268438ea5477419a8de38efdbb1282/node_modules/ember-cli-deploy-revision-data/lib/scm-data-generators/git.js:16
remote:         var info = log.latest;
remote:                       ^
remote:
remote: TypeError: Cannot read property 'latest' of null
remote:     at /tmp/build_ab268438ea5477419a8de38efdbb1282/node_modules/ember-cli-deploy-revision-data/lib/scm-data-generators/git.js:16:23
remote:     at Git.<anonymous> (/tmp/build_ab268438ea5477419a8de38efdbb1282/node_modules/simple-git/src/git.js:1272:13)
remote:     at Git.<anonymous> (/tmp/build_ab268438ea5477419a8de38efdbb1282/node_modules/simple-git/src/git.js:1188Error running: ember deploy production 2>&1:21)
remote:     at emitTwo (events.js:106:13)
remote:     at ChildProcess.emit (events.js:192:7)
remote:     at maybeClose (in
remote: ternal/child_process.js:890:16)
remote:     at Socket.<anonymous> (internal/child_process.js:334:11)
remote:     at emitOne (events.js:96:13)
remote:     at Socket.emit (events.js:189:7)
remote:     at Pipe._handle.close [as _onclose] (net.js:501:12)
remote: -----> Failed trying to compile heroku/ember-cli-deploy
remote:  !     Push rejected, failed to compile emberjs app.
remote:
remote:  !     Push failed
remote: Verifying deploy....
remote:
remote: !       Push rejected to tranquil-retreat-63814.
remote:
To https://git.heroku.com/tranquil-retreat-63814.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'https://git.heroku.com/tranquil-retreat-63814.git'

Setting GIT_DISCOVERY_ACROSS_FILESYSTEM to 1 did not seem to fix the error for me. However the message remote: Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). went away when I did.

marcoow commented 7 years ago

That looks very much like the same issue.

mcfiredrill commented 7 years ago

So looks like I don't need any of the ember-cli-deploy dependencies for heroku, so this seems to be a non issue for me personally now.

FWIW I attempted to implement @achambers ' proposed fix, but the project property wasn't available on the context for me.

achambers commented 7 years ago

@mcfiredrill said:

FWIW I attempted to implement @achambers ' proposed fix, but the project property wasn't available on the context for me.

I find this strange as you can see that it's put in to the context here -> https://github.com/ember-cli-deploy/ember-cli-deploy/blob/master/lib/tasks/pipeline.js#L64

Weird.

mcfiredrill commented 7 years ago

Did you see my patch? Perhaps I'm doing it wrong. @achambers https://github.com/mcfiredrill/ember-cli-deploy-revision-data/commit/975cb5603232bc088f4aaae756374fe7db1c1387

achambers commented 7 years ago

@mcfiredrill I just saw it then. Are you sure the project is missing?

I got the root prop slightly wrong. It's actually project.getProjectRoot()

Could that have been your issue?

mcfiredrill commented 7 years ago

@achambers I think the error was unknown property 'root' of undefined or such , but I can try again when I have time

achambers commented 7 years ago

Hmmm. That's surprising to me. I'll have a look when I get to work

thejohnnybot commented 7 years ago

I ran into this issue today. Any updates on it?

mirague commented 7 years ago

Ran into this issue today as well, running on Heroku:

fatal: Not a git repository (or any of the parent directories): .git
/tmp/build_e41570486597db179c1b9f1ab3674332/trailgames-bazaar-653ceb9c2a13116f49db7ce4de496a4a80836ff8/node_modules/ember-cli-deploy-revision-data/lib/scm-data-generators/git.js:16
        var info = log.latest;
                      ^
TypeError: Cannot read property 'latest' of null
    at /tmp/build_e41570486597db179c1b9f1ab3674332/trailgames-bazaar-653ceb9c2a13116f49db7ce4de496a4a80836ff8/node_modules/ember-cli-deploy-revision-data/lib/scm-data-generators/git.js:16:23
    at Git.<anonymous> (/tmp/build_e41570486597db179c1b9f1ab3674332/trailgames-bazaar-653ceb9c2a13116f49db7ce4de496a4a80836ff8/node_modules/simple-git/src/git.js:1325:13)
    at Git.<anonymous> (/tmp/build_e41570486597db179c1b9f1ab3674332/trailgames-bazaar-653ceb9c2a13116f49db7ce4de496a4a80836ff8/node_modules/simple-git/src/git.js:1236:21)
    at emitTwo (events.js:106:13)
    at ChildProcess.emit (events.js:194:7)
    at maybeClose (internal/child_process.js:899:16)
    at Socket.<anonymous> (internal/child_process.js:342:11)
    at emitOne (events.js:96:13)
    at Socket.emit (eError running: ember deploy production 2>&1
vents.js:191:7)
    at Pipe._handle.close [as _onclose] (net.js:509:12)
-----> Failed trying to compile heroku/ember-cli-deploy
 !     Push rejected, failed to compile emberjs app.
 !     Push failed

@achambers @marcoow

achambers commented 7 years ago

@mirague Fancy looking in to the suggested fix I mentioned further up and submitting a PR?

mirague commented 7 years ago

@achambers On Heroku I know for a fact they remove the .git directory when building the slug which is probably why @mcfiredrill and us are seeing these errors.

As for diving into this I'm not currently at capacity to do so, I recently started with Ember and got into the deploying only today.

I'm trying to think of ideas how we can expose the latest git tags inside the Heroku build process.. I know there's an env variable SOURCE_VERSION which contains the related git hash. We could use this for the git-commit strategy. But the tag still remains a question..

Update: I can get the version from package.json in this case, as opposed to using tag. If there's a custom resolver this can be fixed.

mirague commented 7 years ago

@achambers I'm willing to give it a shot if you can help me out a bit. Any first steps/pointers to get me started?

ssendev commented 7 years ago

https://github.com/mcfiredrill/ember-cli-deploy-revision-data/commit/975cb5603232bc088f4aaae756374fe7db1c1387 worked for me on 1.0.0

mirague commented 7 years ago

@ssendev On Heroku?

ssendev commented 7 years ago

@mirague local with tmp -> /tmp/ember-cli

ssendev commented 7 years ago

@achambers getProjectRoot() is static so is not available on project https://ember-cli.com/api/classes/Project.html#method_getProjectRoot the constructor sets root but it's not entirely clear whether it's public or not since only one private property is documented.

mirague commented 7 years ago

To solve the Heroku scenario we might want to be able to pass in a custom generator, basically a method that resolves with the revisionKey? Like I said, Heroku doesn't have a .git folder so reading from that context isn't possible. They do however set an environment variable SOURCE_VERSION with the full commit hash which can be used.

We could have something like this:

  // in any case we prefer the git-commit resolver
  ENV['revision-data'] = {
    type: 'git-commit'
  };

  // Heroku doesn't have the .git context, use the custom resolver
  if (process.env.SOURCE_VERSION) {
    ENV['revision-data'] = {
      type: 'custom',
      revisionKey: function () {
        return process.env.SOURCE_VERSION;
      }
    }
  }

cc @achambers

flexyford commented 7 years ago

I just ran into this today for Heroku as well. I did not attempt the getProjectRoot solution, so I can't speak to that. I did modify @mirague's solution above.

My workaround uses type: git-commit in development environment, and SOURCE_VERSION in staging and production

  // Heroku doesn't have the .git context, thereforew we choose
  // type and scm value which don't depend on .git repos or require('git-repo-info')
  const isHerokuDeploy = !!process.env.SOURCE_VERSION;
  if (isHerokuDeploy) {
    ENV["revision-data"] = {
      type: 'file-hash'
      scm: null             
    };
  } else {
    ENV["revision-data"] = {
      type: 'git-commit' // <= The `type` you want to use if is not heroku deploy
    };
  }

Then later on in another deployment hook . . .

const sha = process.env.SOURCE_VERSION || context.revisionData.scm.sha;
vihai commented 4 years ago

No news on this issue? I've just been bitten by it...