Middleman V4 Compatibility

[olliekav]

Has there been start on making this extension compatible with v4? The extension seems to be found okay and activated but the CLI command cannot be found.

Could not find command "contentful".

The documentation on what has changed is so light for MM at the moment I can't find what has been changed that might cause this.

[dlitvakb]

Hey @olliekav,

Thanks for the input.

We haven't looked into migrating yet.

Probably will be done as a next step, after we add support for Webhooks (which is a feature that's about to be released).

[olliekav]

@dlitvakb sounds good. Thanks for the prompt reply.

[wanderingcrow]

MM is now v4, just bumping this.

contentful_middleman (>= 0) ruby depends on
      middleman-core (~> 3.3) ruby

Thanks for the work on this gem, btw. I use it all the time.

[dlitvakb]

Hey @javaporter,

We'll be doing a migration to v4 really soon, there are a few things we are finishing before moving onto this, though next week we're probably starting.

Thank you for your interest and support.

Hope all of the new lately introduced features have been put onto good use :smile:

[dlitvakb]

Hey @javaporter @olliekav,

I've added a new branch with support for Middleman v4 -> here

This will not be released in RubyGems yet, but you're free to use it and test it.

Please let us know if you find any bugs in this new version.

[wanderingcrow]

@dlitvakb Awesome, thanks. I'll check it out this week.

[wmakley]

I'm getting the error with the experimental branch still:

Could not find command "contentful".

[dlitvakb]

@wmakley just pushed an updated version.

Turns out there is a lot more in the v4 transition that what Middleman docs say.

[wmakley]

Awesome! Maybe this one is my mistake and not an issue with v4, but it's hard to tell where the initial error propagated from. I am still learning the contentful gem.

$ bundle exec middleman contentful
/Users/william/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/bundler/gems/contentful_middleman-5e6541ab1c21/lib/contentful_middleman/mappers/base.rb:67:in `map_entry_full': undefined method `each' for nil:NilClass (NoMethodError)
    from /Users/william/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/bundler/gems/contentful_middleman-5e6541ab1c21/lib/contentful_middleman/mappers/base.rb:22:in `map'
    from /Users/william/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/bundler/gems/contentful_middleman-5e6541ab1c21/lib/contentful_middleman/import_task.rb:36:in `block in local_data_files'
    from /Users/william/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/contentful-0.8.0/lib/contentful/resource/array_like.rb:15:in `each'
    from /Users/william/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/gems/contentful-0.8.0/lib/contentful/resource/array_like.rb:15:in `each_item'
    from /Users/william/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/bundler/gems/contentful_middleman-5e6541ab1c21/lib/contentful_middleman/import_task.rb:27:in `map'
    from /Users/william/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/bundler/gems/contentful_middleman-5e6541ab1c21/lib/contentful_middleman/import_task.rb:27:in `local_data_files'
    from /Users/william/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/bundler/gems/contentful_middleman-5e6541ab1c21/lib/contentful_middleman/import_task.rb:14:in `run'
    from /Users/william/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/bundler/gems/contentful_middleman-5e6541ab1c21/lib/contentful_middleman/commands/contentful.rb:44:in `block in contentful'
    from /Users/william/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/bundler/gems/contentful_middleman-5e6541ab1c21/lib/contentful_middleman/commands/contentful.rb:42:in `each'
    from /Users/william/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/bundler/gems/contentful_middleman-5e6541ab1c21/lib/contentful_middleman/commands/contentful.rb:42:in `reduce'
    from /Users/william/.rbenv/versions/2.2.4/lib/ruby/gems/2.2.0/bundler/gems/contentful_middleman-5e6541ab1c21/lib/contentful_middleman/commands/contentful.rb:42:in `contentful'

Possibly relevant config.rb:

activate :contentful do |f|
  f.use_preview_api = true
  f.space         = { blog: 'SECRET' }
  f.access_token  = 'SECRET'
  f.cda_query     = { content_type: 'ID', include: 1 }
  f.content_types = {
    post: 'ID'
  }
end

[dlitvakb]

Hey @wmakley,

2 small questions:

1. For f.space, what is called 'SECRET' in your pasted config.rb is the space_id right?
2. Have you tried this with the v3 version and encountered the same issue? Or it is exclusive to v4?

Thanks

[wmakley]

1. Correct.
2. I set up a project for v3, but I could not get it to work at all: I set up config.rb exactly as shown, but kept getting an error message that I needed to call "activate :contentful", even though I verified that my config.rb was calling it.

[dlitvakb]

As for the "activate :contentful" part... it happened to me once, got it solved by deleting the Gemfile.lock file and running bundle install again, make sure that for the v3 project you are running on v0.8.0 or the master branch, the upgrade to MMv4 is backwards incompatible (though the config.rb should remain the same)

[wmakley]

Did you mean a different gem for 0.8.0? contentful_middleman doesn't have a version 0.8.0.

[dlitvakb]

Sorry @wmakley , I meant 1.3.0

[wanderingcrow]

@dlitvakb I'm won't have a chance to really bang on this until I use it on a site in production (or start a new site), but I did do a quick demo and so far, everything seems to be working for me. I set something up in case it helps other people: https://github.com/javaporter/mm4-contentful

[wmakley]

Still getting this error with v3: "You need to activate the contentful extension in config.rb before you can import data from Contentful". I guess I will make a new issue for it.

[wmakley]

Finally got v3 working, looks the problem with v4 is my fault. Same error:

lib/contentful_middleman/mappers/base.rb:67:in `map_entry_full': undefined method `each' for nil:NilClass (NoMethodError)

I assume there is some empty response that is propagating a little too far. Maybe the API isn't working for me.

[dlitvakb]

@wmakley can you open a new ticket regarding this issue?

I need some more information that's not related to this issue (mm4 integration).

[dlitvakb]

@javaporter Thanks! That's awesome!

[philipjabenton]

I'm hitting a problem I need a hand with...

I'm playing with Middleman & Contentful integration with sample Contentful content. I've pulled down data which for this example i'll call Posts.

I am trying to use Middleman Dynamic Pages to create a page for each post. I have this code in my config.rb file which works fine:

data.content.post.each do |id,post| proxy "post/#{post.slug}.html", "post/template.html", :locals => { :post => post }, :ignore => true end

Where I hit a problem is during build/deployment to Netlify because I do not store the data pulled in from Contentful in my github repo. I'd like to pull in the data before build upon deployment.

I'm fairly new to Middleman but I am presuming the proxy block in config.rb is being executed in some way before build causing the error - "undefined method `content'"

I can get this working if I move the code into a configure :build block but I need to be able to pull in the data whilst developing.

Any ideas?

[dlitvakb]

Hey @philipbenton,

Simple fix! As the config.rb gets loaded before calling the contentful extension, you need to make sure that code doesn't get executed unless data is present, middleman auto-defines methods for it's data object based on the contents found inside it's /data folder, therefore, if there is no data, you need to avoid calling them. To do so, change your code to the following:

if data.respond_to? :content
  data.content.post.each do |id,post|
    proxy "post/#{post.slug}.html", "post/template.html", :locals => { :post => post }, :ignore => true
  end
end

[philipjabenton]

@dlitvakb afraid that hasn't helped. If I call middleman contentful or middleman I still get that same error:

/Users/philipbenton/.rvm/gems/ruby-2.3.0/gems/middleman-core-4.1.6/lib/middleman-core/core_extensions/data.rb:176:inmethod_missing': undefined method content' for #<Middleman::CoreExtensions::Data::DataStore:0x007fba9c1c2a60> (NoMethodError)

[dlitvakb]

Hey @philipbenton,

Can we move this conversation to a separate thread?

Please include a copy of your config.rb file, with any sensitive data obfuscated, in the new issue.

Cheers

[philipjabenton]

@dlitvakb Just done that. Thanks.

[shoshi]

Right now, it looks like asset descriptions are not included in v4, but they were present in v3: https://github.com/contentful/contentful_middleman/blob/dl/upgrade-to-v4/lib/contentful_middleman/mappers/base.rb#L54

Is there a plan for this? Or an alternative way of accessing descriptions? (I wonder if translations caused complications.)

[dlitvakb]

Hey @shoshi looks like something I might have accidentally deleted (or never backported). I will add it shortly.

[dlitvakb]

Hey @shoshi,

This has been backported now

Cheers

[shoshi]

@dlitvakb Just tested this and it's working great! Thanks so much!

[hubertron]

curious when full v4 support will be rolled out. Don't want to put an experimental branch into prod.

[dlitvakb]

@hubertron It is not experimental anymore, though a full release is not yet planned as we still need to figure out a way to keep both versions and maintain them simultaneously.

Currently development is running for both in parallel, with full releases only for the v3 version, but this one is completely stable.

[macandcheese]

Is it possible to host the v4 branch on RubyGems? Thanks in advance.

[dlitvakb]

Hey @macandcheese,

I'd need to check out how to have multiple incompatible versions released simultaneously, but I guess I could release it as a separate gem, something like contentful_middleman_v4 without much problems, and keep it like that until the majority of the community moves to v4.

[macandcheese]

@dlitvakb - appreciate the quick response. Thank you!

[davedub]

Any chance you could go ahead with hosting a v4 branch on RubyGems? I can't seem to download and install the experimental branch locally for a Middleman 4 site without getting an error.

gem install --local '/path/to/gemfile/contentful_middleman_v4-1.5.0

[dlitvakb]

@davedub I'll look into how to do this without breaking versioning

Cheers

[shoshi]

For anybody waiting on hosting the v4 branch on RubyGems, (if your use case allows) you can add this to your Gemfile in the meantime:

gem 'contentful_middleman', :git => 'https://github.com/contentful/contentful_middleman.git', :branch => 'dl/upgrade-to-v4'

[kisin]

any news about supporting middleman v4?

[dlitvakb]

Hey @kisin,

As mentioned in comments above, the branch is completely stable and works perfectly with v4 and you can use it currently by pointing your Gemfile to the dl/upgrade-to-v4 branch of this repository.

There is currently a still ongoing research on how to publish both versions (v3 and v4 compatible) without coming into conflicting gems/versions.

Cheers

[kisin]

its not working for me... i'm using ruby-2.2.2, middleman 4.2.1 and branch dl/upgrade-to-v4.

if i'm using configuration like this its only create a space-hash file:

activate :contentful do |f|
  f.access_token  = "access_token"
  f.space         = { authors: "space" }
end

and if i'm using configuration with content type and query then its just stuck:

activate :contentful do |f|
  f.access_token  = "access_token"
  f.space         = { authors: "space" }
  f.cda_query     = { content_type: 'id', include: 1 }
  f.content_types = { author: 'id' }
end

[dlitvakb]

@kisin can you provide the stack trace when running middleman contentful. Also, let's move this to a separate issue.

Cheers

[dlitvakb]

Hello everyone following this thread,

As of today there are breaking changes introduced into the master branch due to the update to the 2.0.0 CDA SDK. This also includes some new niceties that will make your contentful_middleman installation nicer (like removing the need of custom mappers for sys attributes and other bug fixes).

That said, I'd like to hear some opinions regarding what to do with the middleman v4 branch, would you like these changes applied onto it and get all the benefits and less bugs? Or would you prefer a new branch to include these changes to keep compatibility and avoid breaking current codebases?

There are a few more changes that are going to be added before doing a formal release of the v3 gem, in particular options for configuring output path and file names.

Please answer either with reply emojis - to the comments I'll post below - or with a longer form answer. This way, we can make an informed decision based on your feedback.

Cheers

[dlitvakb]

Break Compatibility and keep development aligned

[dlitvakb]

Create new v4 branch and current v4 branch stays as the 1.x compatible.

(or the other way around, create new v4-1.x-stable branch, and keep this one aligned)

[wanderingcrow]

Thanks for the continued hard work on this gem. I use the v4 version 100% of the time now and it's a crucial part of my workflow. Glad to see it's still in active development.

[allanwhite]

Appreciate the hard work, it's really useful. I tend to be more willing to break things to maintain compatibility with an important service like Contentful; it tends to mean greater stability in the future, and more rapid adoption of new features.

[dlitvakb]

Ok, after giving it some time to collect feedback. What I'm going to do is create a v4-1.x-stable branch with the CURRENT (as of 2017-04-14) state of the branch. That branch will no longer receive any updates unless there's a security flaw found.

This current branch will be updated to receive the newest updates from master and will keep parity moving forwards.

I once again warn everybody that THIS IS A BREAKING CHANGE, and that if you want to keep your current installation as is, switch to the new v4-1.x-stable branch.

The new branch will be created at Tuesday, April 18th, 2017 - 10AM GMT-3:00. With the current branch updated shortly afterwards.

Cheers

[dlitvakb]

v4-1.x-stable branch successfully created.

Starting work on merging upstream master with dl/upgrade-to-v4.

[dlitvakb]

dl/upgrade-to-v4 has been merged with upstream master 🎉

Happy hacking everybody!

[kaishiro]

So, this change ended up breaking a bunch of our builds. The issue we're having is that the a few of the builds are dependent on the all_entries_page_size option, and this option no longer seems present in the "new" v4-1.x-stable. As far as I can tell this leaves people who were on the old dl/upgrade-to-v4 branch but dependent on that option somewhat stranded unless I'm missing something.

[dlitvakb]

Hey @kaishiro,

Looks like you're correct, there are 3 missing commits.

I'll add them now, I'm really sorry for that, will update ASAP.

Cheers

Update: This has been pushed now, sorry for the inconvenience.