search

HWCloudEngine / hybrid_cloud

6 stars 12 forks source link

[CI] Build a CI Enviroment #17

Open arthur-wangfeng opened 8 years ago

arthur-wangfeng commented 8 years ago

Goal:

Build a collaborative development environments.

  1. Can be used by T-Div team
  2. Support code/document review
  3. Support nightly integration test
  4. Support deploy/configure testbed automatically
  5. Maintain projects on github

Guide: https://github.com/Hybrid-Cloud/hybrid_cloud/wiki/How-to-work-with-Hybrid-Cloud-CI https://github.com/Hybrid-Cloud/hybrid_cloud/wiki/gerrit-brief-user-guide

Gerrit Url: http://202.96.135.238:8000

Plan:

Milestone Date Projects on Github Review: Gerrit automation: Jenkins Test: Tempest
_Wang Feng_ _Li Liangyu_ _Li Bin_ _Wu Jing_
1/26 1.Confirm current status of Gerrit; 2.Make Detailed Plan Make detailed plan make detailed plan
1/27 Split Codes to different projects 搭建新的gerrit环境，并验证基于http的用户注册功能 搭建新的jenkins环境，并验证从git上获取代码的配置过程
1/28 搭建简单openstack环境， 利用环境进一步熟悉tempest具体细节
1/29 熟悉并验证项目导入、项目及用户权限分配、提交流程（开发者提交->reviewer检视->提交/驳回）等
1/30 deploy hybrid cloud testbed manually 验证基于openId的用户注册功能 熟悉并验证jenkins的配置项目 串讲 tempest
Weekend 1/31
2/01
2/02
2/03
2/04
2/05 Register all team members
CI-PoC 2/06 下周与jenkins联调各项功能，主要验证整个流程：开发者提交->reviewer检视->jenkins用例测试->提交/驳回 jenkins的脚本触发以及自动部署，并触发脚本过程 对接齐飞环境， 由于fsp keystone鉴权环节做了修改，需要修改tempest源代码，对接成功
Holiday 2/07
Holiday 2/08
Holiday 2/09
Holiday 2/09
Holiday 2/10
Holiday 2/11
Holiday 2/12
Holiday 2/13
2/14 对接王博环境成功，上传image镜像，测试image-list flavor-list简单用例成功
2/15 gerrit认证方式已支持github账号登录，需各位开发同事注册 复杂用例测试失败，fsp代码做了修改， tempest需要进行适配
2/16 在gerrit创建工程，验证基本流程 在测试jenkins从jacket工程获取代码，并在演示环境上尝试部署
2/17 为注册用户分配权限，验证常用权限功能 jenkins测试jacket工程的备份以及安装
2/18 验证测试gerrit提交、审核各种场景下功能，如多分枝提交、冲突解决等 尝试构建有关jacket项目模块的测试用例工程
2/19 验证测试gerrit提交、审核各种场景下功能，如多分枝提交、冲突解决等，完善gerrit使用指导书 尝试构建有关jacket项目模块的测试用例工程
Weekend 2/20
Weekend 2/21
2/22
2/23
2/24
2/25
2/26
2/27
Weekend 2/28
CI-Delivery 2/29
drormizrachi commented 8 years ago

Hi

Here are some recommended python code analysis tools that might be integrated with Jenkins: (1) https://www.python.org/dev/peps/pep-0008/ - to check python style conversion (2) https://pypi.python.org/pypi/pyflakes- check errors in your code (3) http://www.pylint.org/ - need a lot of tuning, else you'll have a lot of false positive

drormizrachi commented 8 years ago

You also need to make sure that your stable build will be deployed to T-DIV LAB too

drormizrachi commented 8 years ago

Last, T-DIV needs an access to the Gerrit

arthur-wangfeng commented 8 years ago

status Update on 2016/2/2:

  1. Projects on Github (Wang Feng)
    • cleaned/merged abandoned projects,
    • changed team members privilege: team leader can admin all projects, project Hybrid-Cloud-Patches-For-Tricircle is locked to ordinary developers (read-only).
  2. Build Test-bed (Wang Feng):
    • Finish installation of basic mngt nodes. configuring now.
    • accomplish this week (Plan)
  3. Gerrit (Li Liangyu):
    • Installed a new public gerrit (Done).
    • adding member and projects (In progress)
  4. Jenkins (Li Bin):
    • Installed a new Jenkins. (Done)
    • Auto deploy _legacy_ project: script, etc. (This week)
    • Design automation script framework for _future_ projects. (This week, In progress)
  5. Tempest (Wu Jing):
    • ramp up in original openstack (Done)
    • Demo in hybrid cloud environment. (Plan, In progress)
arthur-wangfeng commented 8 years ago

Status updated on 2016/2/16:

In general, about a week delayed, due to holiday and team member's leave .

  1. Common (Wang Feng)
    • cleaned/merged abandoned projects. (Done)
    • changed team members privilege: team leader can admin all projects, project Hybrid-Cloud-Patches-For-Tricircle is locked to ordinary developers (read-only). (Done)
    • write CI guide and review. (In progress)
  2. Build Test-bed (Wang Feng):
    • Test-bed support all features except Disaster Recovery. (progress 90%. accomplish in Feb.)
  3. Gerrit (Li Liangyu):
    • Installed a new public gerrit (Done)
    • write user guide. (Done)
    • support github openid, user can register by themself. (Done)
    • configure projects (In progress)
  4. Jenkins (Li Bin):
    • Installed a new Jenkins. (Done)
    • Auto deploy _legacy_ project: script, etc. (In progress)
    • Design automation script framework for _future_ projects. (In progress, working on PoC)
  5. Tempest (Wu Jing):
    • ramp up in original openstack (Done)
    • Integrating with hybrid cloud environment. (In progress, working on PoC, solving some configuration issues)
arthur-wangfeng commented 8 years ago

Status updated on 2016/2/23:

In general, about a week delayed, due to holiday and team member's leave .

  1. Common (Wang Feng)
    • cleaned/merged abandoned projects. (Done)
    • changed team members privilege: team leader can admin all projects, project Hybrid-Cloud-Patches-For-Tricircle is locked to ordinary developers (read-only). (Done)
    • write CI guide and review. (In progress)
    • Setup up a meeting to introduce whole CI flow. (This week)
    • Hands-on training. (In progress, Wang Gang is preparing hardware)
  2. Build Test-bed (Wang Feng):
    • Test-bed support all features except Disaster Recovery. (progress 90%. accomplish in Feb.)
  3. Gerrit (Li Liangyu): (Done). It works now.
    • Installed a new public gerrit
    • write user guide.
    • support github openid, user can register by themself.
    • configure projects
  4. Jenkins (Li Bin):
    • Installed a new Jenkins. (Done)
    • Auto deploy _legacy_ project: script, etc. (In progress)
    • Design automation script framework for _future_ projects. (need review)
    • Invoke Tempest and send result via email. (PoC done)
  5. Tempest (Wu Jing):
    • ramp up in original openstack (Done)
    • Integrating with hybrid cloud environment . (PoC done)
    • Write documents and introduce to team members. (In Progress)
    • Optimize: fix bugs and Solve some configuration issues. (In Progress)
    • support multi-process?
  6. Develop test case (all developers): (Planned, started after CI Introduction meeting)
  7. Build clustered Liberty version openstack testbed.
    • build Opensource Liberty version testbed, without cluster feature. (in Progress, )
    • ramp up for cps (Wang feng, Libin) (in Progress)
drormizrachi commented 8 years ago

Please let us know when the setup scripts are ready and work, so we can run it on our lab too, thanks in advance

arthur-wangfeng commented 8 years ago

Status updated on 2016/3/1:

Summary:

  1. CI platform is built and introduced to team, need to push team use this CI: develop installation scripts, test use case, etc
  2. Ramping up for cps for building clustered Liberty version openstack testbed
  3. Hands-on training. (On-hold due to work conflicts, decision need. platform is ready.)

Plan:

  1. Start to build clustered Liberty version openstack testbed.
  2. Optimize CI platform according requirements. Push team using this platform

Decision Needed

  1. CI team consists of: . a. 3 long term dedicated members: liu longzhu and 2 others. who are responsible for CI maintaining, testing. b. 1~3 short term dedicated members (rotating position, 2~3 months): Libin, Liliangyu, Wujing. who are responsible for short-term CI development and demo environment maintaining.

    The developers responsible for developing test case are members of feature teams, not members of CI team. The feature team leaders are the only contact person for CI team.

    1. Hand-on lab training, to learn how to deploy hybrid cloud environment. a. an one-week dedicated guy from each feature team to join this training b. Qifei need 1 week to train them, and he need a tester continue his current work for a week.
    2. some actions make team use CI procedure: a. collect commits and source code lines on github project to evaluate teams' workload b. review design and codes on gerrit c. evaluate the quality of developers and teams' deliveries using tempest testing report.

Detailed Progress

  1. Common (Wang Feng)
    • cleaned/merged abandoned projects. (Done)
    • changed team members privilege: team leader can admin all projects, project Hybrid-Cloud-Patches-For-Tricircle is locked to ordinary developers (read-only). (Done)
    • write CI guide and review. (Done)
    • Setup up a meeting to introduce whole CI flow. (Done)
    • Hands-on training. (on-hold)
  2. Build Test-bed (Wang Feng):
    • Test-bed support all features except Disaster Recovery. (progress 90%. accomplish in Feb.)
  3. Gerrit (Li Liangyu): (Done). It works now.
    • Installed a new public gerrit
    • write user guide.
    • support github openid, user can register by themself.
    • configure projects
  4. Jenkins (Li Bin):
    • Installed a new Jenkins. (Done)
    • Auto deploy _legacy_ project: script, etc. (In progress)
    • Design automation script framework for _future_ projects. (need review)
    • Invoke Tempest and send result via email. (PoC done)
  5. Tempest (Wu Jing):
    • ramp up in original openstack (Done)
    • Integrating with hybrid cloud environment . (PoC done)
    • Write documents and introduce to team members. (done)
    • smoking test issue. (In Progress)
    • support multi-process?
  6. Develop test case (all developers): (Planned, started after CI Introduction meeting)
  7. Build clustered Liberty version openstack testbed.
    • build Opensource Liberty version testbed, without cluster feature. (in Progress, )
    • ramp up for cps (Wang feng, Libin) (in Progress)
arthur-wangfeng commented 8 years ago

add link of gerrit, jenkins.etc

gerrit: 202.96.135.238:8000

arthur-wangfeng commented 8 years ago

Status updated on 2016/3/15:

Summary:

  1. CI platform is built and introduced to team, need to push team use this CI: develop installation scripts, test use case, etc
  2. building clustered Liberty version openstack testbed
  3. Hands-on training. (On-hold due to work conflicts, decision need. platform is ready.)

Plan of this week:

  1. Accomplish installation scripts development. Start nightly testing
  2. Accomplish Liberty deployment
  3. (every feature team) Start test use case development
  4. Try to use DevStack as developers' component test environment. (CMS, Hufeiran)
arthur-wangfeng commented 8 years ago

Status updated on 2016/3/22:

Summary:

  1. Accomplish installation scripts development. Start nightly testing
  2. Deployed a standalone Liberty VM, integrating with FS-cascading. (liulongzhu)
  3. fixed some ci-env problems.
  4. ramp up with k8s testing framework.

Plan of this week:

  1. automatically deploy agent in hypervm
  2. integrate Liberty with FS-cascading
  3. Push feature team modify test use case
  4. report loc and commits number of each team/developer daily.
  5. Try to use DevStack as developers' component test environment. (CMS, Hufeiran)

On hold

  1. Hands-on training. (On-hold due to work conflicts, decision need. platform is ready.)
arthur-wangfeng commented 8 years ago

Status updated on 2016/3/29:

Summary:

  1. ensured integrating Liberty with FS-cascading is feasible.
  2. support automatically deploy agent in hypervm on aws.
  3. Determined test use cases of computing and cms
  4. Tried report loc and commits number of each team/developer daily.
  5. built a k8s testing framework, need more time to analyze.

Plan of this week:

  1. finish integration of Liberty and FS-cascading (before April, 8th)
  2. automatically deploy agent in hypervm on vcloud
  3. Push feature team modify test use case, need be done (before April, 8th)
  4. report loc and commits number of each team/developer daily from this week

Decision Needed

  1. add testing use case development to feature team's plan and review the status on weekly meeting.

On hold

  1. Hands-on training. (**On-hold due to work conflicts, decision need. platform is ready.
  2. Try to use DevStack as developers' component test environment. (CMS, Hufeiran)
arthur-wangfeng commented 8 years ago

Status updated on 2016/4/5:

Assigned this task to Hu Feiran, because Wang Feng relocated to T-div.

Summary:

  1. Deploying opensource testbed: integrating Liberty and FS-cascading _(ongoing, before April, 8th)_
  2. Automatically deploy agent in hypervm on vcloud _(3 days Delayed, need to extend pyvcloud api for exporting-vapp-as-template)_
  3. Push feature team modify test use case, need be done _(ongoing,before April, 8th)_
  4. report loc and commits number of each team/developer daily from this week _(Done)_

Plan of this week:

  1. Deploying opensource testbed: integrating Liberty and FS-cascading _(before April, 8th)_
  2. Automatically deploy agent in hypervm on vcloud _(This Week)_
  3. Push feature team modify test use case. _(ongoing,before April, 8th)_
  4. analyze murano testing framework, for hybrid cloud docker feature; _(This week)_
  5. Build and release cascading&cascaded images automatically. _(before April, 15th)_

Decision Needed

  1. Who maintain demo environments? A special team or who deployed them?

On hold

  1. Hands-on training. (**On-hold due to work conflicts, decision need. platform is ready.
  2. Try to use DevStack as developers' component test environment. (CMS, Hufeiran)