Migrate OracleLinux7.9/8.x image to Aliyun

[qiaofeng1227]

Requirement

1. Can use password and pairkey when create VM
2. Can automatic mount all the default system disk when create VM
3. Can reset password on Cloud console
4. Can resize system disk on Cloud console
5. remove waagent and waagent init
6. Reset Time Zone / UTC time

refer

API:https://help.aliyun.com/document_detail/25542.html doc:https://help.aliyun.com/document_detail/25464.html

[chendelin1982]

System disk can't auto increase suggest use growpart refer to: https://blog.csdn.net/just_a_litte_body/article/details/109991898

[qiaofeng1227]

btrfs disk auto increase

btrfs filesystem resize max /data
btrfs filesystem resize max /

[chendelin1982]

LVM resize solution confirmed, refer to: https://support.websoft9.com/docs/linux/zh/admin-file.html#lvm We need a automatic script to like this: 1, 2, it need to test

The suitable solution is that we should set the cloud.cfg

timezone: "Asia/Shanghai" 

growpart:
  mode: auto
  devices: [/dev/vda2]
  ignore_growroot_disabled: false

runcmd:
  - [pvresize,/dev/vda2]
  - [lvextend,-l,+100%FREE,/dev/rootvg/rootlv]
  - [xfs_growfs,/dev/rootvg/rootlv]

and we need to run cloud-init clean before download VHD

[chendelin1982]

Rest password issue

1. cloud-init
2. KeyPair and Password mutually exclusive（互斥）?
3. root locked at /etc/shadow

   root:*LOCK*:14600::::::

   romove all cloud-init cach

   cloud-init clean

time zone solution

# chronyd need to re-install because of service error
yum erase chronyd
yum install chronyd -y

#oraclelinux8
yum erase chrony
yum install chrony -y

timedatectl set-ntp yes

# Set time zone
timedatectl set-timezone "Asia/Shanghai" 
timedatectl set-local-rtc yes
systemctl restart chronyd

#?
hwclock --localtime

Remove Azure agent

systemctl disable waagent
waagent deprovision+root --force

Delete azure cfg

rm -rf /etc/cloud/cloud.cfg.d/10_updates_policy.cfg
rm -rf /etc/cloud/cloud.cfg.d/10-azure-kvp.cfg

[chendelin1982]

2021-11-23 09:12:18,330 - stages.py[DEBUG]: Running module set-passwords (<module 'cloudinit.config.cc_set_passwords' from '/usr/lib/python3.6/site-packages/cloudinit/config/cc_set_passwords.py'>) with frequency once-per-instance
2021-11-23 09:12:18,330 - handlers.py[DEBUG]: start: modules-config/config-set-passwords: running config-set-passwords with frequency once-per-instance
2021-11-23 09:12:18,331 - util.py[DEBUG]: Writing to /var/lib/cloud/instances/iid-datasource-none/sem/config_set_passwords - wb: [644] 25 bytes
2021-11-23 09:12:18,331 - helpers.py[DEBUG]: Running config-set-passwords using lock (<FileLock using file '/var/lib/cloud/instances/iid-datasource-none/sem/config_set_passwords'>)
2021-11-23 09:12:18,331 - util.py[DEBUG]: Reading from /etc/ssh/sshd_config (quiet=False)
2021-11-23 09:12:18,331 - util.py[DEBUG]: Read 4270 bytes from /etc/ssh/sshd_config
2021-11-23 09:12:18,331 - ssh_util.py[DEBUG]: line 70: option PasswordAuthentication already set to yes
2021-11-23 09:12:18,331 - cc_set_passwords.py[DEBUG]: No need to restart SSH service, PasswordAuthentication not updated.
2021-11-23 09:12:18,331 - handlers.py[DEBUG]: finish: modules-config/config-set-passwords: SUCCESS: config-set-passwords ran successfully

cloud-init 默认时区必须修正，否则日志文件显示的是 UTC 时间

[qiaofeng1227]

It's OK at our timezone set methon

[root@iZj6c46sxkb44piei7ekgkZ ~]# date
Sat Nov 27 10:07:53 CST 2021
[root@iZj6c46sxkb44piei7ekgkZ ~]# timedatectl
      Local time: Sat 2021-11-27 10:08:00 CST
  Universal time: Sat 2021-11-27 02:08:00 UTC
        RTC time: Sat 2021-11-27 10:07:59
       Time zone: Asia/Shanghai (CST, +0800)
     NTP enabled: yes
NTP synchronized: yes
 RTC in local TZ: yes
      DST active: n/a

Warning: The system is configured to read the RTC time in the local time zone.
         This mode can not be fully supported. It will create various problems
         with time zone changes and daylight saving time adjustments. The RTC
         time is never updated, it relies on external facilities to maintain it.
         If at all possible, use RTC in UTC by calling
         'timedatectl set-local-rtc 0'.

but cloud-init log time is UTC-time, because log output time is UTC-time

[root@iZj6ce6cov921yxkpw6hjcZ ~]# cd /usr/lib/python3.6/site-packages/cloudinit
[root@iZj6ce6cov921yxkpw6hjcZ cloudinit]# ls
analyze           cloud.py  cs_utils.py       dmi.py        features.py  handlers     __init__.py  net         persistence.py  reporting    settings.py        sources      subp.py        type_utils.py  util.py
apport.py         cmd       dhclient_hook.py  ec2_utils.py  filters      helpers.py   log.py       netinfo.py  __pycache__     safeyaml.py  signal_handler.py  ssh_util.py  templater.py   url_helper.py  version.py
atomic_helper.py  config    distros           event.py      gpg.py       importer.py  mergers      patcher.py  registry.py     serial.py    simpletable.py     stages.py    temp_utils.py  user_data.py   warnings.py
[root@iZj6ce6cov921yxkpw6hjcZ cloudinit]# cat log.py
# Copyright (C) 2012 Canonical Ltd.
# Copyright (C) 2012 Hewlett-Packard Development Company, L.P.
# Copyright (C) 2012 Yahoo! Inc.
#
# Author: Scott Moser <scott.moser@canonical.com>
# Author: Juerg Haefliger <juerg.haefliger@hp.com>
# Author: Joshua Harlow <harlowja@yahoo-inc.com>
#
# This file is part of cloud-init. See LICENSE file for license information.

import collections
import io
import logging
import logging.config
import logging.handlers
import os
import sys
import time

# Logging levels for easy access
CRITICAL = logging.CRITICAL
FATAL = logging.FATAL
ERROR = logging.ERROR
WARNING = logging.WARNING
WARN = logging.WARN
INFO = logging.INFO
DEBUG = logging.DEBUG
NOTSET = logging.NOTSET

# Default basic format
DEF_CON_FORMAT = '%(asctime)s - %(filename)s[%(levelname)s]: %(message)s'

# Always format logging timestamps as UTC time
logging.Formatter.converter = time.gmtime

def setupBasicLogging(level=DEBUG, formatter=None):
    if not formatter:
        formatter = logging.Formatter(DEF_CON_FORMAT)
    root = logging.getLogger()
    for handler in root.handlers:
        if hasattr(handler, 'stream') and hasattr(handler.stream, 'name'):
            if handler.stream.name == '<stderr>':
                handler.setLevel(level)
                return
    # Didn't have an existing stderr handler; create a new handler
    console = logging.StreamHandler(sys.stderr)
    console.setFormatter(formatter)
    console.setLevel(level)
    root.addHandler(console)
    root.setLevel(level)

def flushLoggers(root):
    if not root:
        return
    for h in root.handlers:
        if isinstance(h, (logging.StreamHandler)):
            try:
                h.flush()
            except IOError:
                pass
    flushLoggers(root.parent)

def setupLogging(cfg=None):
    # See if the config provides any logging conf...
    if not cfg:
        cfg = {}

    log_cfgs = []
    log_cfg = cfg.get('logcfg')
    if log_cfg and isinstance(log_cfg, str):
        # If there is a 'logcfg' entry in the config,
        # respect it, it is the old keyname
        log_cfgs.append(str(log_cfg))
    elif "log_cfgs" in cfg:
        for a_cfg in cfg['log_cfgs']:
            if isinstance(a_cfg, str):
                log_cfgs.append(a_cfg)
            elif isinstance(a_cfg, (collections.Iterable)):
                cfg_str = [str(c) for c in a_cfg]
                log_cfgs.append('\n'.join(cfg_str))
            else:
                log_cfgs.append(str(a_cfg))

    # See if any of them actually load...
    am_tried = 0
    for log_cfg in log_cfgs:
        try:
            am_tried += 1
            # Assume its just a string if not a filename
            if log_cfg.startswith("/") and os.path.isfile(log_cfg):
                # Leave it as a file and do not make it look like
                # something that is a file (but is really a buffer that
                # is acting as a file)
                pass
            else:
                log_cfg = io.StringIO(log_cfg)
            # Attempt to load its config
            logging.config.fileConfig(log_cfg)
            # The first one to work wins!
            return
        except Exception:
            # We do not write any logs of this here, because the default
            # configuration includes an attempt at using /dev/log, followed
            # up by writing to a file.  /dev/log will not exist in very early
            # boot, so an exception on that is expected.
            pass

    # If it didn't work, at least setup a basic logger (if desired)
    basic_enabled = cfg.get('log_basic', True)

    sys.stderr.write(("WARN: no logging configured!"
                      " (tried %s configs)\n") % (am_tried))
    if basic_enabled:
        sys.stderr.write("Setting up basic logging...\n")
        setupBasicLogging()

def getLogger(name='cloudinit'):
    return logging.getLogger(name)

def _resetLogger(log):
    """Remove all current handlers, unset log level and add a NullHandler.

    (Adding the NullHandler avoids "No handlers could be found for logger XXX"
    messages.)
    """
    if not log:
        return
    handlers = list(log.handlers)
    for h in handlers:
        h.flush()
        h.close()
        log.removeHandler(h)
    log.setLevel(NOTSET)
    log.addHandler(logging.NullHandler())

def resetLogging():
    _resetLogger(logging.getLogger())
    _resetLogger(getLogger())

resetLogging()

# vi: ts=4 expandtab

log.py : row 38

[chendelin1982]

split to:

Websoft9/role_os#21 Websoft9/StackHub#108