teracy-dev roadmap for v0.6.0

[hoatle]

v0.6.0

• [x] project cleanup: remove Chef-specific provisioner to another project as the optional cookbook
• [x] all Vagrant settings support
• [ ] test coverage should be increased, expect >= 90% test coverage
• [ ] better docs organizations and add more docs (user guide, dev guide, reference guide)
• [ ] should support different providers (for example, if I prefer VMWare, Parallel or other types of providers like AWS, DO, GCE, bare metal remote host, etc.) (supported via teracy-dev-core or can create extensions to support)
• [ ] installer for easier on board usage (lower priority, possibly v1.0)
• [x] default setup for faster speed, low memory footprint (kernel, core and its extensions)
• [x] easier config management with yaml files
• [x] extension mechanism to customize, extend teracy-dev without any limitation

v0.6.0

• [ ] final release

v0.6.0-c1, c2

• [ ] bug fixes if any

v0.6.0-b2

• [ ] test coverage should be increased, expect >= 90% test coverage
• [ ] user guide, dev guide, reference guide is available

  v0.6.0-b1

• [x] add Location::Manager.add_synch for extension support
• [ ] load /teracy-dev-ext.rb if avaiable
• [ ] better docs organization, update docs
• [ ] update the travis-ci configuration for check ruby code style + unit tests
• [ ] dev mode for docs using k8s

v0.6.0-a6

• [x] bug fix for git_sync

v0.6.0-a5

• [x] support register processors, configurators with weight
• [x] support "remote" location config from Location::GitSynch
• [x] add support for extension dependency validation
• [x] support _id update with deprecation
• [x] mask specified secret values within log messages

v0.6.0-a4

• bug fixes + minor improvements

v0.6.0-a3

• bug fixes + minor improvements

v0.6.0-a2

• bug fixes + minor improvements

v0.6.0-a1

• [x] clean up
• [x] upgrade to use the latest version of vagrant, virtualbox
• [x] support all vagrant settings
• [x] cluster support (multiple-machine)
• [x] easier configuration (completely new way of configuration with yaml files)
• [x] teracy-dev extension introduced to customize, extend teracy-dev without any limitation

[hoatle]

let's use https://www.hashicorp.com/blog/hashicorp-vagrant-2-0/ for v0.6.0

[hoatle]

this is the proposal for yaml instead of json:

# vars must be defined to be used
# format:
# key: ${ENV_VAR}
# key: ${ENV_VAR-:default}
# and "#{key}" can be used for settings values if available
variables:
  node_name_prefix: ${NODE_NAME_PREFIX:-node}
  node_hostname_prefix: ${NODE_HOSTNAME_PREFIX:-teracy-dev}
  some_value: ${MY_VAR:-value}-affix

# vagrant related settings
vagrant:
  require_version: ">= 2.1.0"
  plugins:
    - _id: "0"
      name: vagrant-gatling-rsync
      required: true
      enabled: true
      config_key: gatling
      options:
        latency: 0.5
        time_format: "%H:%M:%S"
        rsync_on_startup: true
    - _id: "1"
      name: vagrant-rsync-back
      required: true
      enabled: true
    - _id: "2"
      name: vagrant-hostmanager
      required: true
      enabled: true
      config_key: hostmanger
      options:
        manage_host: true
        manage_guest: true
  # usually defined from workspace/dev-setup/
  preload_extension_paths:
  postload_extension_paths:
  config_paths:

# teracy-dev related settings
teracy-dev:
  require_version: ">= 0.6.0-a1-SNAPSHOT"
  config_lookup_path: workspace/dev-setup/ # will look for workspace/dev-setup/config_default.yaml

# default settings for all nodes
default:
  vm:
    box: bento/ubuntu-16.04
    box_version:
    box_url:
  providers:
    - _id: "0"
      name: virtualbox
      gui: false
      memory: 1024
      description: "teracy-dev #{Time.now.getutc.to_i}"
  provisioners:
    - _id: "0"
      type: chef_solo
      verion: "13.1.31"
      log_level: info # one of debug, info, warn, error, fatal. Default: info
      cookbooks_path:
        - vendor-cookbooks
        - main-cookbooks
      roles_path: roles
      nodes_path: nodes
      data_bags_path: data_bags
      run_list:
        - vim
        - teracy-dev
      json:
        docker:
          enabled: false
          version: 1.8.0
    - _id: "1"
      type: ansible_local
      # etc

# specific nodes, each node will override the default
nodes:
  - name: "#{node_name_prefix}-1"
    is_primary: true
    vm:
      hostname: "#{NODE_HOSTNAME_PREFIX}-1.local"
  - name: "#{node_name_prefix}-2"
    vm:
      hostname: "#{NODE_HOSTNAME_PREFIX}-2.local"

[hoatle]

the Vagrantfile could start with the following skeleton to work with the above yaml file:

# -*- mode: ruby -*-
# vi: set ft=ruby :
require "rubygems"
require 'yaml'

# teracy-dev version
VERSION='0.6.0-a1-SNAPSHOT'

load File.dirname(__FILE__) + '/lib/utility.rb'

$logger = get_logger()

# learn from https://github.com/hashicorp/vagrant/blob/ee5656da37c75e896201d389c79da199114603c2/lib/vagrant.rb#L216
def require_version(*requirements)
  $logger.info("teracy-dev version: #{VERSION}")
  req = Gem::Requirement.new(*requirements)
  if req.satisfied_by?(Gem::Version.new(VERSION))
      $logger.debug("version requirements satisfied!")
      return
  end
  $logger.error("teracy-dev version requirements: #{requirements}")
  exit 1
end

settings = YAML.load_file(File.join(File.dirname(__FILE__), 'config_default.yaml'))

Vagrant.require_version settings["vagrant"]["require_version"]
require_version(settings['teracy-dev']['require_version'])

# process settings override mechanism here

$logger.debug("settings: #{settings}")

# pre-load-ext list here

Vagrant.configure("2") do |config|
  settings["nodes"].each do |node_config|
    $logger.debug("node_config: #{node_config}")
    config.vm.define node_config["name"] do |node|

    end
  end
  # post-load-ext list here
end

[hoatle]

the config override mechanism is basically the same with v0.5.0, we have "config_default.yaml" and "config_override.yaml" files respectively.

[hoatle]

this is the working branch from my repo, we'll collaborate on that branch until we have a working version to be merged: https://github.com/hoatle/teracy-dev/tree/tasks/v0.6.0

[hoatle]

https://github.com/hoatle/teracy-dev/blob/tasks/v0.6.0/lib/settings_processor.rb is ready to be implemented, it has the same override mechanism as v0.5.0 with additional variables_pipeline to be implemented

[hoatle]

@phuonglm please validate the initial stuff and idea/spec from https://github.com/hoatle/teracy-dev/tree/tasks/v0.6.0 to organize implementation effort for the team.

[hoatle]

make sure all the settings from Vagrantfile must be implemented via our config file https://www.vagrantup.com/docs/index.html we have default_config.yaml and override_config.yaml to override the default one.

[hoatle]

https://github.com/hoatle/teracy-dev/blob/tasks/v0.6.0/lib/configurator.rb is ready to be implemented

[hoatle]

for easier development, $ LOG_LEVEL=debug vagrant status should be very helpful.

You can export $ export LOG_LEVEL=debug and then $ vagrant status

[hoatle]

https://github.com/hoatle/teracy-dev/tree/tasks/v0.6.0 is ready for preview

[hoatle]

teracy-dev v0.6 rational:

• slow dev pace, technical debt exists, slow to release a new version
• some issues/features not easy to implement with the existing code base
• extension features should be used to keep the kernel, core minimal and no limitation to extend the core with extensions
• easy to support different types of dev tools, dev envs, and dev workflow

teracy-dev v0.6 requirements:

• minimal kernel (nucleus) with the extension mechanism
• a teracy-dev-core extension is used to support all features of Vagrant
• other extensions to extend the teracy-dev-core extension (teracy-dev-essential, teracy-dev-k8s, teracy-dev-docker (to support existing teracy-dev v0.5 migration + upgrade), etc)
• create different extensions and dev-setups with v0.6 to see how things work and to adjust reasonably for v0.6
• migrate existing teracy-dev v0.5 setup to v0.6 setup to see how things work and to adjust reasonably for v0.6
• broken changes from v0.5 to v0.6 are expected