search

opencontainers / runtime-tools

OCI Runtime Tools
https://www.opencontainers.org/
Apache License 2.0
434 stars 143 forks source link

way to integrate runtime validation with travis.ci #527

Open liangchenye opened 7 years ago

liangchenye commented 7 years ago

Once https://github.com/opencontainers/runtime-tools/pull/439 been merged, we will have a very flexible way to verify/certify a runtime. I wrote a demo to explain how we can integrate it with travis.ci. The approach is similar with code coverage which does these:

  1. using prove validate/*.t > output.txt to get a testing report
  2. send the testing report to a server
  3. add the icon to 'README.md' to show the certification result

I wrote a demo project to show how to achieve this:

  1. a ok demo and a fail demo. In the travis, I generate a 'ok.txt' and 'fail.txt'.
  2. In the travis, I upload them to a server -- 'ocihub.org' which I applied.
  3. In README.md, I added two icons, one is 'ok', one is 'fail'.

In the future, 'runc', 'rkt', runv' and other runtime projects could apply this approach. So both the maintainer/user could be easily know a runtime is conformed with oci spec.

The server code is very simple, I reuse isula/ihub framework.

wking commented 7 years ago

On Wed, Nov 29, 2017 at 11:56:53AM +0000, 梁辰晔 (Liang Chenye) wrote:

  1. In the travis, I upload them to a server -- 'ocihub.org' which I applied.

How will this be secured? Will the admin will hand out per-project credentials to project admins who put those credentials in Travis repository settings 1 and use those settings for their POST?

liangchenye commented 7 years ago

Yes, the admin will send the 'token' to the user, the user can put it in the travis environment. Taking codecov for example, a user could login by his/her github account. codecov will generate the token and send it to the user automatically. I don't know how many man-power it will cost to develop such a system.

The certification progress is more official. In that case, we may not need to develop a 'token-generate' system. The certification team could just generate it by hand and hand out pre-project credentials to project admins.

alban commented 6 years ago

I've run the validation tests on runc with CircleCI: https://github.com/opencontainers/runc/pull/1757