search

jwadolowski / cookbook-cq

Chef cookbook for Adobe CQ (aka AEM)
Other
20 stars 23 forks source link
adobe aem automation chef cookbook cq hacktoberfest ruby

readme

CQ/AEM Chef cookbook

This cookbook deploys and configures Adobe Experience Manager (AEM), formerly known as CQ.

FYI, it is not called aem-coobkook, because when I started development there was no AEM yet (it was known as CQ at that time). Nowadays the aem name seems to be taken anyways, so I no longer have a choice.

Table of contents

Supported platforms

Operating systems

Chef versions

IMPORTANT

Since version 2.0.0 only Chef Infra Client 16.x is supported. Please stick to version 1.x if you're using older version of Chef!

AEM/CQ versions

Getting started

TBD

Attributes

For default values please refer to appropriate files.

default.rb

To set Java related attributes please refer to java cookbook. By default it installs Oracle's JDK7.

author.rb

All attributes in this file refer to CQ/AEM author instance (node['cq']['author'] namespace).

publish.rb

All attributes in this file refer to CQ/AEM publish instance (node['cq']['publish'] namespace).

Recipes

default.rb

Installs core dependencies (Ruby gems and OS packages).

commons.rb

Takes care of common elements of every CQ/AEM deployment, including:

author.rb

Installs CQ/AEM author instance.

publish.rb

Installs CQ/AEM publish instance.

Custom resources

All CQ/AEM related resource are idempotent, so action won't be taken if not required.

Whenever you need to deploy 2 or more CQ/AEM instances on a single server please make sure you named all your custom resources differently, as you may get unexpected results otherwise (i.e. when CQ/AEM restart is required afterwards). Please find cq_package example below:

Bad:

cq_package 'package1' do
  instance "http://localhost:#{node['cq']['author']['port']}"

  action :deploy
end

cq_package 'package1' do
  instance "http://localhost:#{node['cq']['publish']['port']}"

  action :deploy
end

Good:

cq_package 'Author: package1' do
  instance "http://localhost:#{node['cq']['author']['port']}"

  action :deploy
end

cq_package 'Publish: package1' do
  instance "http://localhost:#{node['cq']['publish']['port']}"

  action :deploy
end

cq_package

Allows for CRX package manipulation using CRX Package Manager API.

Key features:

Actions

If you'd like to upload and install a package, in most cases please use deploy action instead of combined upload and install. Detailed explanation can be found below.

Properties

Usage

cq_package 'Slice 4.2.1' do
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  source 'https://oss.sonatype.org/content/groups/public/com/cognifide/slice'\
    '/slice-assembly/4.2.1/slice-assembly-4.2.1-cq.zip'

  action :upload
end

First cq_package resource will download Slice package from provided URL and upload it to defined AEM Author instance.

cq_package 'Upgrade to Oak 1.0.13' do
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  source 'https://artifacts.example.com/aem/6.0.0/hotfixes'\
    '/cq-6.0.0-hotfix-6316-1.1.zip'
  http_user 'john'
  http_pass 'passw0rd'

  action :upload
end

Second resource does the same as the first one, but for Oak 1.0.13 hotfix. The only difference is that provided URL requires basic auth, hence the http_user and http_pass properties.

cq_package 'ACS AEM Commons 1.10.2' do
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  source 'https://github.com/Adobe-Consulting-Services/acs-aem-commons'\
    '/releases/download/acs-aem-commons-1.10.2'\
    '/acs-aem-commons-content-1.10.2.zip'

  action [:upload, :install]
end

Third package shows how to combine multiple actions in a single cq_package resource usage.

cq_package 'AEM6 hotfix 6316' do
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  source node['cq']['packages']['aem6']['hf6316']
  recursive_install true

  action :deploy

  notifies :restart, 'service[cq60-author]', :immediately
end

4th cq_package presents how to use deploy action that combines both upload and install in a single execution. This is preferred way of doing package deployment, in particular for those that require AEM service restart as soon as installation is completed. recursive_install was also used here, which is required for majority of hotfixes and every service pack.

cq_package 'Geometrixx All' do
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  source "http://localhost:#{node['cq']['author']['port']}/etc/packages"\
    '/day/cq60/product/cq-geometrixx-all-pkg-5.7.476.zip'
  http_user node['cq']['author']['credentials']['login']
  http_pass node['cq']['author']['credentials']['password']

  action :uninstall
end

Next example describes usage of uninstall action. In this particular case operation was executed against Geometrixx package.

cq_package 'Not really well-thought-out package' do
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  source node['cq']['packages']['myapp']
  http_user node['cq']['author']['credentials']['login']
  http_pass node['cq']['author']['credentials']['password']
  rescue_mode true

  action :deploy

  notifies :restart, 'service[cq60-author]', :immediately
end

6th cq_package presents usage of rescue_mode property. Imagine that this package provides new OSGi bundles and right after its installation some serious issue occurs (i.e. unresolvable OSGi dependency, conflict or cycle). As a result of this event all bundles will be turned off and effectively instance will stop responding or start serving 404s for all resources (including /system/console). The java process though will still be running. The only solution to that problem is AEM restart, after which all work perfectly fine again. Without rescue_mode property cq_package provider will keep checking OSGi bundles to detect their stable state, but none of these attempts will end successfully, as nothing is reachable over HTTP. Eventually Chef run will be aborted. If rescue_mode was activated (set to true) then after error_state_barrier unsuccessful attempts an error will be printed and the processing will be continued (restart of cq60-author service in this case).

cq_package 'Author: Service Pack 2 (upload)' do
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  source node['cq']['packages']['aem6']['sp2']

  action :upload
end

cq_package 'Author: Service Pack 2 (install)' do
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  source node['cq']['packages']['aem6']['sp2']
  recursive_install true
  rescue_mode true
  same_state_barrier 12
  error_state_barrier 12
  max_attempts 36

  action :install

  notifies :restart, 'service[cq60-author]', :immediately
end

7th & 8th cq_package resources explain how to deal with AEM instance restarts after package installation and tune post installation OSGi stability checks.

Moreover it explains how to use combination of upload and install instead of deploy. Such procedure might be required sometimes, i.e. when some extra steps have to be done after package upload, but before its installation.

Please notice that both resources were named differently on purpose to avoid resource merge and 2 restarts. If you'd use:

cq_package 'Author: Service Pack 2' do
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  source node['cq']['packages']['aem6']['sp2']
  recursive_install true

  action [:upload, :install]

  notifies :restart, 'service[cq60-author]', :immediately
end

or

cq_package 'Author: Service Pack 2' do
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  source node['cq']['packages']['aem6']['sp2']

  action :upload
end

cq_package 'Author: Service Pack 2' do
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  source node['cq']['packages']['aem6']['sp2']
  recursive_install true

  action :install

  notifies :restart, 'service[cq60-author]', :immediately
end

two restarts will be triggered.

In the first case during compile phase Chef will generate 2 resources with the same name, but different actions.

In second example restart still will be triggered after upload, even if it's not explicitly defined during 1st usage (upload action). The reason is quite simple - both resources are named the same (Author: Service Pack 2) and Chef will treat this as a single resource on resource collection. It means that notify parameter will be silently merged to the resource with upload action during compile phase.

cq_osgi_config

Provides an interface for CRUD operations in OSGi configs.

Actions

For regular (non-factory, single instance) configs:

For factory configs:

Properties

Compatibility matrix

Property Regular OSGi config Factory OSGi config
pid :white_check_mark: :white_check_mark:
username :white_check_mark: :white_check_mark:
password :white_check_mark: :white_check_mark:
instance :white_check_mark: :white_check_mark:
factory_pid :x: :white_check_mark:
properties :white_check_mark: :white_check_mark:
append :white_check_mark: :white_check_mark:
apply_all :white_check_mark: :white_check_mark:
include_missing :white_check_mark: :white_check_mark:
unique_fields :x: :white_check_mark:
count :x: :white_check_mark:
enforce_count :x: :white_check_mark:
force :white_check_mark: :x:

Usage

Regular OSGi configs

cq_osgi_config 'Root Mapping' do
  pid 'com.day.cq.commons.servlets.RootMappingServlet'
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  properties('rootmapping.target' => '/welcome.html')

  action :create
end

Root Mapping resource sets / redirect to /welcome.html if it's not already set.

cq_osgi_config 'Event Admin' do
  pid 'org.apache.felix.eventadmin.impl.EventAdmin'
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  append true
  properties(
    'org.apache.felix.eventadmin.IgnoreTimeout' => ['com.example*']
  )

  action :create
end

Event Admin merges defined properties with the ones that are already set (because of append property). This is how Event Admin will look like before:

ID VALUE
org.apache.felix.eventadmin.ThreadPoolSize 20
org.apache.felix.eventadmin.Timeout 5000
org.apache.felix.eventadmin.RequireTopic true
org.apache.felix.eventadmin.IgnoreTimeout ["org.apache.felix*","com.adobe*"]

and after Chef run:

ID VALUE
org.apache.felix.eventadmin.ThreadPoolSize 20
org.apache.felix.eventadmin.Timeout 5000
org.apache.felix.eventadmin.RequireTopic true
org.apache.felix.eventadmin.IgnoreTimeout ["com.adobe*","com.example*","org.apache.felix*"] 
cq_osgi_config 'OAuth Twitter' do
  pid 'com.adobe.granite.auth.oauth.impl.TwitterProviderImpl'
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  properties(
    'oauth.provider.id' => 'twitter'
  )

  action :delete
end

Properties of OAuth Twitter will be restored to default values if any of them was previously modified (explicitly set).

cq_osgi_config 'Promotion Manager' do
  pid 'com.adobe.cq.commerce.impl.promotion.PromotionManagerImpl'
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  force true

  action :delete
end

Promotion Manager will behave as OAuth Twitter with one exception - it will happen on every Chef run due to force property.

Factory OSGi configs

cq_osgi_config 'Custom Logger' do
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  factory_pid 'org.apache.sling.commons.log.LogManager.factory.config'
  properties(
    'org.apache.sling.commons.log.level' => 'error',
    'org.apache.sling.commons.log.file' => 'logs/custom.log',
    'org.apache.sling.commons.log.pattern' =>
      '{0,date,dd.MM.yyyy HH:mm:ss.SSS} *{4}* [{2}] {3} {5}',
    'org.apache.sling.commons.log.names' => [
      'com.example.custom1',
      'com.example.custom2'
    ]
  )
  unique_fields ['org.apache.sling.commons.log.file']

  action :create
end

Custom Logger resource will create a new logger according to defined properties. org.apache.sling.commons.log.file is a virtual identifier of given OSGi config instance (specified by user). If instance with such "ID" already exists nothing happens. Otherwise a brand new configuration will be created. Please keep in mind that there's no need to specify any UUID in resource definition.

cq_osgi_config 'Job Queue' do
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  factory_pid 'org.apache.sling.event.jobs.QueueConfiguration'
  properties(
    'queue.name' => 'Granite Workflow Timeout Queue',
    'queue.type' => 'TOPIC_ROUND_ROBIN',
    'queue.topics' => ['com/adobe/granite/workflow/timeout/job'],
    'queue.maxparallel' => -1,
    'queue.retries' => 10,
    'queue.retrydelay' => 2000,
    'queue.priority' => 'MIN',
    'service.ranking' => 0
  )
  unique_fields ['queue.name']

  action :delete
end

Job Queue resource will delete factory instance of org.apache.sling.event.jobs.QueueConfiguration that has queue.name set to Granite Workflow Timeout Queue. Presence of additional properties doesn't matter in this case and will be completely ignored.

cq_osgi_bundle

Adds ability to stop and start OSGi bundles

Actions

Properties

Usage

cq_osgi_bundle 'Author: org.eclipse.equinox.region' do
  symbolic_name 'org.eclipse.equinox.region'
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  same_state_barrier 3
  sleep_time 5

  action :stop
end

First example stops org.eclipse.equinox.region AEM author instance. Since there's just a few dependencies on this bundle, number of post-stop checks have been limited, as there's no point to wait for so long.

cq_osgi_bundle 'com.adobe.xmp.worker.files.native.fragment.linux' do
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"

  action :start
end

Second instance of cq_osgi_bundle is fairly simple, as it just starts com.adobe.xmp.worker.files.native.fragment.linux bundle.

cq_osgi_component

Please keep in mind that OSGi components used to get back to their original state after AEM instance restart. If you disabled one, most probably it'll become enabled after instance restart.

Actions

Properties

Usage

cq_osgi_component 'Author: com.example.my.component' do
  pid 'com.example.my.component'
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"

  action :enable
end

cq_osgi_component 'Author: com.project.email.servlet' do
  pid 'com.project.email.servlet'
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"

  action :disable
end

Both examples are self-explanatory. First one enables com.example.my.component component if it's in disabled state. Second one will disable com.project.email.servlet component, but only if it's state is not already disabled.

cq_user

Exposes a resource for CQ/AEM user management. Supports:

Actions

Properties

Compatibility matrix

Property admin user All other users
id :white_check_mark: :white_check_mark:
username :white_check_mark: :white_check_mark:
password :white_check_mark: :white_check_mark:
instance :white_check_mark: :white_check_mark:
email :white_check_mark: :white_check_mark:
first_name :white_check_mark: :white_check_mark:
last_name :white_check_mark: :white_check_mark:
phone_number :white_check_mark: :white_check_mark:
job_title :white_check_mark: :white_check_mark:
street :white_check_mark: :white_check_mark:
mobile :white_check_mark: :white_check_mark:
city :white_check_mark: :white_check_mark:
postal_code :white_check_mark: :white_check_mark:
country :white_check_mark: :white_check_mark:
state :white_check_mark: :white_check_mark:
gender :white_check_mark: :white_check_mark:
about :white_check_mark: :white_check_mark:
user_password :x: :white_check_mark:
enabled :x: :white_check_mark:
old_password :white_check_mark: :x:

Usage

cq_user 'admin' do
  username node['cq']['author']['credentials']['login']
  password 'd4rk_kn1ght'
  instance "http://localhost:#{node['cq']['author']['port']}"

  first_name 'Bruce'
  last_name 'Wayne'
  old_password 'passw0rd'

  action :modify
end

Modify action on cq_user 'admin' resource will change CQ/AEM admin password to d4rk_kn1ght if the current one is either passw0rd or admin (the latter is automatically checked if both password and old_password are incorrect). Moreover admin first name and last name will be updated (to Bruce and Wayne respectively) if needed.

cq_user 'author' do
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"

  first_name 'Peter'
  last_name 'Parker'
  job_title 'Spiderman'
  gender 'male'
  enabled false
  user_password 'sp1d3r'

  action :modify
end

Second example (cq_user 'author') also updates user password, but this time the old one doesn't have to be specified, as this operation will be executed on admin rights (auth credentials: username/password). Additionally auhtor's profile will be updated and user will be disabled (enabled false), so you won't be able to log in as this user anymore.

cq_jcr

CRUD operations on JCR nodes.

Actions

Properties

Usage

cq_jcr '/content/test_node' do
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  properties(
    'property_one' => 'first',
    'property_two' => 'second',
    'property_three' => ['item1', 'item2', 'item3']
  )

  action :create
end

Create action on cq_jcr '/content/test_node' will create such node with given properties if it doesn't exist yet. Otherwise its properties will be updated if necessary. By default append is set to true, which means existing properties of /content/test_node will stay untouched unless the same properties are specified in your cq_jcr resource.

cq_jcr '/content/geometrixx/en/products/jcr:content' do
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  append false
  properties(
    'jcr:primaryType' => 'cq:PageContent',
    'jcr:title' => 'New title',
    'subtitle' => 'New subtitle',
    'new_property' => 'Random value'
  )

  action :create
end

2nd example sets append property to false, which means that all properties except those specified in your resource should be removed. It will act as a full overwrite (keep in mind that some properties are protected and can't be deleted, moreover Sling API automatically adds things like jcr:createdBy).

cq_jcr '/content/dam/geometrixx-media/articles/en/2012' do
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"

  action :delete
end

Next example is quite simple - /content/dam/geometrixx-media/articles/en/2012 will get deleted if it exists. Otherwise warning message will be printed.

cq_jcr '/content/geometrixx/en/services/certification/jcr:content' do
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  properties(
    'jcr:title' => 'New Certification Services',
    'brand_new_prop' => 'ValueX'
  )

  action :modify
end

Last cq_jcr resource uses :modify action. It applies updates to existing nodes only. If specified path does not exist warning message will be displayed.

cq_start_guard

Allows you to wait for full AEM instance start before moving on with subsequent operations. It periodically sends HTTP request to AEM and compares response (both status code and body) with expected state. As soon as defined requirements are met the resource stops its job.

Actions

Properties

Usage

service 'cq64-author' do
  supports status: true, restart: true
  action :start

  notifies :run, "cq_start_guard[cq64-author]", :immediately
end

cq_start_guard 'cq64-author' do
  instance "http://localhost:#{node['cq']['author']['port']}"
  path node['cq']['author']['healthcheck']['resource']
  expected_code node['cq']['author']['healthcheck']['response_code']
  expected_body node['cq']['author']['healthcheck']['response_body']
  timeout node['cq']['service']['start_timeout']

  action :nothing
end

Whenever AEM gets (re)started run cq_start_guard and wait until /libs/granite/core/content/login.html returns 200 response code

service 'cq64-author' do
  supports status: true, restart: true
  action :start

  notifies :run, "cq_start_guard[cq64-author]", :immediately
end

cq_start_guard 'cq64-author' do
  instance "http://localhost:#{node['cq']['author']['port']}"
  path '/bin/healthchecks/instance'
  expected_code '200'
  expected_body '{"status": "ok"}'
  timeout 900
  http_timeout 5
  interval 10

  action :nothing
end

Right after restart of cq64-author service send notification to cq_start_guard and wait until /bin/healthchecks/instance returns 200 code and {"status": "ok"} JSON in the body. Don't spend more than 15 minutes on such health check. Requests will be send every 10 seconds, however each HTTP call can't last more than 5 seconds.

cq_clientlib_cache

This resource enables invalidation/rebuilt of internal clientlib cache in AEM. Please keep in mind that cq_clientlib_cache is not idempotent and it is generally recommended to trigger it via notify from other resources.

Actions

Properties

Usage

cq_package 'Custom AEM app' do
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"
  source 'http://artifacts.example.org/app/1.0/myapp-1.0.zip'
  recursive_install true

  action :deploy

  notifies :invalidate, 'cq_clientlib_cache[invalidation]', :delayed
end

cq_clientlib_cache 'invalidation' do
  username node['cq']['author']['credentials']['login']
  password node['cq']['author']['credentials']['password']
  instance "http://localhost:#{node['cq']['author']['port']}"

  action :nothing
end

Testing

TBD

Author

Jakub Wadolowski (jakub.wadolowski@cognifide.com)