issue when deploying to Heroku

[acandael]

I just implemented the middleman-search_engine_sitemap gem to my Middleman project and it works fine locally.

However when I push my changes to Heroku, I get this build error:

error  tmp/sitemap.xml
There were errors during this build, re-run with `middleman build --verbose` to see the full    exception.

rake aborted! Command failed with status (1): [middleman build...] /tmp/build_b3d8907b-d401-4248-91c2-027fd6b69c1d/Rakefile:3:in `block (2 levels) in <top (required)>'

!
!     Precompiling assets failed.
! 

!     Push rejected, failed to compile Ruby app

 To git@heroku.com:morning-inlet-7066.git
  ! [remote rejected] master -> master (pre-receive hook declined)
  error: failed to push some refs to 'git@heroku.com:morning-inlet-7066.git'

Does anyone have a clue what the problem might be? My repository is at: https://github.com/acandael/personal-site

thanks for your help,

Anthony

[Aupajo]

Hey, Anthony.

Just looking at your commit, it looks like you're using a different gem than this one (middleman-sitemap vs middleman-search_engine_sitemap).

If you're having issues with that gem, you can get in touch with the author.

Alternatively, you're welcome to try my version, which has a few more options and might be better suited to your needs – check out the README!

[acandael]

Hi Aupajo, yes, in the meantime I tried another gem (middleman-sitemap). But I'll try 'middleman-search_engine_sitemap again and let you know if I have the same deployment issue.

greetings,

Anthony

[acandael]

I installed the middleman-search_engine_sitemap again and having the same issue when deploying to Heroku:

error  tmp/sitemap.xml
   There were errors during this build, re-run with `middleman build --verbose` to see the full exception.
   rake aborted!
   Command failed with status (1): [middleman build...]
   /tmp/build_e4ce3b06-a4cb-474c-83ae-10973c4d89bc/Rakefile:3:in `block (2 levels) in <top (required)>'
   Tasks: TOP => assets:precompile
   (See full trace by running task with --trace)
 !
 !     Precompiling assets failed.
 !

 !     Push rejected, failed to compile Ruby app

my repository is at: https://github.com/acandael/personal-site

thanks for looking into this,

Anthony

[Aupajo]

Sorry, Anthony! I missed your initial commit where you used this gem.

I've managed to reproduce the error on Heroku. I'm looking into it now.

---

Issue from Heroku:

   error  tmp/sitemap.xml
       undefined method `call' for nil:NilClass
       /tmp/build_c9a2bb7e-2f2b-4ea4-a9bd-33990e811f47/vendor/bundle/ruby/2.0.0/gems/tilt-1.4.1/lib/tilt/builder.rb:22:in `evaluate'
       /tmp/build_c9a2bb7e-2f2b-4ea4-a9bd-33990e811f47/vendor/bundle/ruby/2.0.0/gems/tilt-1.4.1/lib/tilt/template.rb:103:in `render'

The call comes from evaluate in Tilt's Builder template handler:

xml = ::Builder::XmlMarkup.new(:indent => 2)
data.call(xml)

So nil is data which, from the comments, is the “template source; loaded from a file or given directly.”

Looks like the template isn't being passed to Tilt properly.

Paths seem to be accessible on Heroku:

irb(main)> Middleman::SearchEngineSitemap::TEMPLATES_DIR
=> "/app/vendor/bundle/ruby/2.0.0/gems/middleman-search_engine_sitemap-1.1.0/lib/middleman/search_engine_sitemap/templates"
irb(main)> `ls /app/vendor/bundle/ruby/2.0.0/gems/middleman-search_engine_sitemap-1.1.0/lib/middleman/search_engine_sitemap/templates`
=> "sitemap.xml.builder\n"

On Heroku, during the build, the TEMPLATES_DIR is referenced as:

Relative: vendor/bundle/ruby/2.0.0/gems/middleman-search_engine_sitemap-1.1.0/lib/middleman/search_engine_sitemap/templates (exists)
Absolute: /tmp/build_f8b51737-151f-4bbd-86c8-3703c9337d3f/vendor/bundle/ruby/2.0.0/gems/middleman-search_engine_sitemap-1.1.0/lib/middleman/search_engine_sitemap/templates (exists)

The weirdness seems to kick in with the file watcher.

after_configuration do
  puts files.known_paths.inspect
end

Locally:

#<Set: {#<Pathname:../../../.gem/ruby/2.1.1/gems/middleman-search_engine_sitemap-1.1.0/lib/middleman/search_engine_sitemap/templates/sitemap.xml.builder>}>

Heroku:

#<Set: {}>

Looks like the gem template paths don't get registered on Heroku. Something may be happening with the file watcher, or maybe after_configuration never gets called.

Can't relocate the gem to a lib directory and get the same issue. Suggests something about Heroku's gem paths.

[acandael]

Thanks a lot!

Anthony

From: Pete Nicholls notifications@github.com Reply-To: Aupajo/middleman-search_engine_sitemap <reply+i-34655518-d6446a0e0246ca8834a57ca19adf49e18a76d688-1520187@reply.git hub.com> Date: Monday 2 June 2014 04:02 To: Aupajo/middleman-search_engine_sitemap middleman-search_engine_sitemap@noreply.github.com Cc: Anthony Candaele anthony.candaele@webmove.be Subject: Re: [middleman-search_engine_sitemap] issue when deploying to Heroku (#2)

Sorry, Anthony! I missed your initial commit with the gem.

I've managed to reproduce the error on Heroku. I'm looking into it now.

‹ Reply to this email directly or view it on GitHub https://github.com/Aupajo/middleman-search_engine_sitemap/issues/2#issuecom ment-44796588 .

[mdaguete]

I've the same problem with RHEL 6.5

== Request: /sitemap.xml error build/sitemap.xml undefined method call' for nil:NilClass /opt/moviles/deploy/vendor/bundle/ruby/2.0.0/gems/tilt-1.4.1/lib/tilt/builder.rb:22:inevaluate' /opt/moviles/deploy//vendor/bundle/ruby/2.0.0/gems/tilt-1.4.1/lib/tilt/template.rb:103:in render' /opt/moviles/deploy//vendor/bundle/ruby/2.0.0/gems/middleman-core-3.3.3/lib/middleman-core/core_extensions/rendering.rb:274:inrender_individual_file'

[mdaguete]

The bug is due /^vendor(\/|$)/ present in regex ignore list in middleman filewatcher, i've tested it removing the regex. Add to config.rb and now it works ok

#Filewatcher ignore list
set :file_watcher_ignore,[
    /^bin(\/|$)/,
    /^\.bundle(\/|$)/,
#        /^vendor(\/|$)/,
    /^node_modules(\/|$)/,
    /^\.sass-cache(\/|$)/,
    /^\.cache(\/|$)/,
    /^\.git(\/|$)/,
    /^\.gitignore$/,
    /\.DS_Store/,
    /^\.rbenv-.*$/,
    /^Gemfile$/,
    /^Gemfile\.lock$/,
    /~$/,
    /(^|\/)\.?#/,
    /^tmp\//
  ]

[Aupajo]

Thanks, @mdaguete, that's really helpful. Did the sitemap still work for you when you made that change?

Need to think of a way to combat this more generally.

[mdaguete]

@Aupajo the site map works ok with the change, no problem at all

[acandael]

Hi mdaguete, I ran into exactly the same issue. Your solution fixed it. Thanks, Anthony

[bassjacob]

@Aupajo Not intending to resurrect a seemingly dead thread, but I experienced the same problem as above, deploying to Heroku. @mdaguete your config.rb fix worked to fix the error.

Thank you.

[pzi]

Hi!

I am experiencing the same error during the middleman build step of my middleman template on travis-ci: https://travis-ci.org/pzi/middleman-basis#L311

So I'd give this a +1 :neutral_face:

Thanks for your gem/work!

Cheers, Patrik

[Aupajo]

Hi @pzi,

You can try @mdaguete's approach above. I can't build this into my gem, because modifying the global file watcher ignore list seems like a Bad Idea™ for any library to do. What I probably need to do is reach out to the Middleman team and see if there's any new way around this/anything planned for a future release.

[pzi]

Hey @Aupajo,

Yeah, I figured it would be a Bad Idea™ :) I even felt bad for adding it to my config.rb and making Travis-CI work more by outputting all those extra/unnecessary lines of code.

I wonder why it's not blowing up when building the site locally. I am using the same ruby version and gems. It only fails on Travis-CI.

Thanks for your help though. Hope that's gonna work out fine!

[Aupajo]

I think it's to do with the way Heroku bundles gems into vendor.

[pzi]

Hmm, I am not using Heroku though? I just use Travis-CI to run bundle exec middleman build --verbose to make sure it all works. Not deploying it to anywhere :|

[Aupajo]

Oh, sorry, mis-read your comment. Does Travis do the same? Can you see anything like bundle install --path ... anywhere in your Travis log?

[pzi]

Hmm, no... this is the bundler line: https://travis-ci.org/pzi/middleman-basis#L89

[Aupajo]

That build looks like it's passing?

[pzi]

Yeah, because of the ignore in config.rb https://github.com/pzi/middleman-basis/blob/develop/config.rb#L91

This is a failed build: https://travis-ci.org/pzi/middleman-basis/builds/54708444

[Aupajo]

Ah, thanks! I wonder if there's some Bundler config. Will chat with the Travis folk.

[pzi]

Could probably do some trickery with commands: http://docs.travis-ci.com/user/languages/ruby/#Dependency-Management and http://docs.travis-ci.com/user/languages/ruby/#Custom-Bundler-arguments-and-Gemfile-locations

[Aupajo]

@pzi From that first page:

By default, gems are installed into vendor/bundle in your project's root directory.

[pzi]

Hmm, so I guess I could do bundler_args: --path bundle in my .travis.yml :)

Didn't see that. Thanks!

[Aupajo]

No problem! Hope it helps the next person.

[pzi]

Hmm, ultimately, it has the same effect on ci as before. Except we don't need that hack any more (which feels slightly better). https://travis-ci.org/pzi/middleman-basis/builds/54819693

[pzi]

Hmm, I came across this message: https://groups.google.com/d/msg/travis-ci/XID21_FwgX8/DhW1xU7XlCEJ

And because I have a Gemfile.lock in my repo it will do bundle install --jobs=3 --retry=3 --deployment which causes the problem. So I have a feeling that Heroku would probably do the same given it's a hosting platform.

TL;DR

I felt dirty with bundler_args: --path bundle so I overwrote the args that Travis adds, but left out --deployment. Resulting in bundler_args: --jobs=3 --retry=3 and a properly passing build without hacks.

[renatocarvalho]

I'm having this issue as well.

[Aupajo]

Have a workaround for the file watcher ignore issue – now running into one where the rendered template is returning nil.

Middleman::CoreExtensions::Rendering:

def render_individual_file(path, locs={}, opts={}, context=self, &block)
  ...
  body = if opts[:template_body]
    opts.delete(:template_body)
  else
    template_data_for_file(path)
  end
  ...
end

body is returning nil. The template_data_for_file method defined in Rendering is not actually being called – the one defined in FrontMatter is, instead.

Middleman::CoreExtensions::FrontMatter:

def template_data_for_file(path)
  extensions[:frontmatter].data(path).last
end

def data(path)
  ...
  data, content = frontmatter_and_content(p) # {}, nil
  ...
end

def frontmatter_and_content(path)
  ...
  data = {}
  return [data, nil] if !app.files.exists?(full_path) || ::Middleman::Util.binary?(full_path) # returning here
  ...
end

app.files.exists?(full_path) is returning false.

Returning full circle to the file watcher:

def exists?(path)
  p = Pathname(path)
  p = p.relative_path_from(Pathname(@app.root)) unless p.relative?
  @known_paths.include?(p)
end

@known_paths:

@known_paths
=> #<Set: {#<Pathname:config.rb>,
 #<Pathname:source/images/background.png>,
 #<Pathname:source/images/middleman.png>,
 #<Pathname:source/index.html.erb>,
 #<Pathname:source/javascripts/all.js>,
 #<Pathname:source/layouts/layout.erb>,
 #<Pathname:source/stylesheets/all.css>,
 #<Pathname:source/stylesheets/normalize.css>}>

Looks like the reload trick isn't working any more.

To be continued…

[hirefrank]

Having the same problem on CircleCi...

[hirefrank]

This worked for me on CicleCI:

dependencies:
  override:
    - bundle install --path=gemset: # note ':' here
        timeout: 180 # fail if command has no output for 3 minutes

  # we automatically cache and restore many dependencies between
  # builds. If you need to, you can add custom paths to cache:
  cache_directories:
    - "gemset"   # relative to the build directory
    #- "~/custom_2" # relative to the user's home directory

[Aupajo]

Still tracking this issue on the Middleman repo: https://github.com/middleman/middleman/issues/1217

At this point, it looks like it won't be an issue in 4.0, but I'm not overly hopeful that it can be resolved in 3.x.

[Aupajo]

As far as I can tell, this issue has now been addressed and fixed in Middleman 4.

If you're on Middleman 4, you can update your project by altering your Gemfile as follows and then running bundle:

gem 'middleman-search_engine_sitemap', '~> 1.4.0'

You should no longer need the --path or file_watcher_ignore workarounds.

Let me know if you run into any issues with the new version!