Put Contentful data into a blog?

[mrh-is]

Is there a way to pipe data from Contentful into a middleman-blog? I can generate pages wherever, but I can't get the blog to take them. Any proxys end up running after blog runs, so they're not available to the blogs.

[dlitvakb]

Hey @mrh-is,

Have you tried https://github.com/contentful/contentful-middleman-pages ??

You could then do something like:

class PostMapper < ContentfulMiddleman::Mapper::Base
  def map(context, entry)
    super
    publish_date  = entry.publish_date
    context.year  = publish_date.year
    context.month = if publish_date.month >= 10 then publish_date.month else "0#{publish_date.month}" end
    context.day   = if publish_date.day >= 10 then publish_date.day else "0#{publish_date.day}" end
    context.updatedAt = entry.sys[:updatedAt]

    if defined? entry.old_file_name and (not entry.old_file_name.nil?)
      context.slug = entry.old_file_name
    else
      context.slug = entry.title.parameterize
    end

    context.published = true
  end
end

activate :contentful do |f|
  f.space           = { blog: ENV['CONTENTFUL_SPACE_ID'] }
  f.access_token    = ENV['CONTENTFUL_ACCESS_TOKEN']
  f.use_preview_api = use_preview_api
  f.cda_query       = { content_type: ENV['CONTENTFUL_POST_CONTENT_TYPE_ID'], include: 1, limit: 1000 }
  f.content_types   = { post: { id: ENV['CONTENTFUL_POST_CONTENT_TYPE_ID'], mapper: PostMapper } }
end

activate :contentful_pages do |extension|
  extension.data      = 'blog.post'
  extension.prefix    = 'blog'
  extension.template  = 'article.html.haml'
  extension.permalink = '{year}-{month}-{day}-{slug}.html'
end

activate :blog do |blog|
  blog.permalink = "blog/:year/:month/:day/:title"
  blog.sources   = "blog/:year-:month-:day-:title"
  blog.paginate = true
  blog.page_link = "page/:num"
  blog.publish_future_dated = true
end

Hope this helps

Cheers

[mrh-is]

Yeah, that looks great! But it doesn't look like it supports MM v4, and won't for a while. Thoughts on that? I tried to get proxies working, but I wasn't able to get them into the blogs. (This may be veering into a middleman-blog issue if that's the route I need to go.)

[dlitvakb]

Hey @mrh-is,

Sorry to tell you that the pages repo is not getting an update to v4 currently... next month I'm flying to the office and I can discuss personally about this, as there is starting to be better support for v4 lately and many have been requesting it.

If possible, try to talk with the people maintaining the blog extensions and check if they can provide you with a better solution than the one I described in the issue you referenced.

Hope you can solve your issue shortly

Cheers

[mrh-is]

We fixed this! We redirected contentful_middleman to put the YAML files into source/data, which makes those resources available to middleman-blog. We also patched the file writer to output files that look like blog entries, i.e., front matter in a YAML block followed by Markdown content.

[dlitvakb]

Hey @mrh-is,

Glad to hear that you solved this!

I'm in Berlin now and for 2 more weeks, so if you have any feature ideas, it's the best time to bring them up!

Cheers

[mrh-is]

Our patches, which could probably be made options (and I could probably just PR):

module Middleman
  module Cli
    class Contentful
      remove_const :MIDDLEMAN_LOCAL_DATA_FOLDER
      MIDDLEMAN_LOCAL_DATA_FOLDER = 'source/data'
    end
  end
end

module ContentfulMiddleman
  class Context
    def to_yaml
      full_hash = hashize
      content = full_hash.delete(:content)
      full_hash.deep_stringify_keys.to_yaml + "---\n\n" + content
    end
  end
  module LocalData
    class File
      def local_data_file_path
        base_path = LocalData::Store.base_path
        ::File.join(base_path, @path + ".html.md")
      end
    end 
  end
end

We also had to patch all_entries because getting 1000 entries at a time was too big (>7MB):

module ContentfulMiddleman
  class Instance
    def all_entries(cda_query)
      all = []
      query = cda_query.clone
      query[:order] = 'sys.createdAt' unless query.key?(:order)
      num_entries = client.entries(limit: 1).total
      entries_per_page = 100

      ((num_entries / entries_per_page) + 1).times do |i|
        query[:limit] = entries_per_page
        query[:skip] = i * entries_per_page
        page = client.entries(query)
        page.each { |entry| all << entry }
      end

      all
    end
  end
end

Any to all of these might be the wrong way to do these. ¯(ツ)/¯

[dlitvakb]

Hey @mrh-is,

Looks like we could probably add a configurable data path, and also a configurable amount of entries per page when using all entries, regarding the YAML processing, I don't understand what you did.

I'll add this to our backlog

Cheers

[mrh-is]

Awesome, hopefully those help someone else!

The YAML changes basically just move you from:

---
id: post_1
title: "Cool Stuff"
content: "Lots of cool words go here."

to

---
id: post_1
title: "Cool Stuff"
---

Lots of cool words go here.

That way we didn't have to rewrite our existing blog templates to work with Contentful. We also changed them to .html.md files instead of .yaml files, again so the normal middleman-blog processing would work.

[dlitvakb]

Ahh that makes sense!

I'll look into adding custom transformers as well, though I don't think we'll dive into that huge rabbit hole, but maybe adding them as a custom extension like we currently do with mappers might work

Cheers

[alexandra-aas]

@dlitvakb @mrh-is I'm looking to use contentful as the source of my blog posts. I see that patches were created, but I'm unsure how to actually implement this. I have contentful working and can grab the data, but I'm unsure how to actually use the data as the blog source. Any guidance would help. Thanks.