search

ipfs-inactive / dev-team-enablement

[ARCHIVED] Dev Team Enablement Working Group
3 stars 1 forks source link

Setup Jenkinsfile for sharness to pull down go-ipfs to run with sharness #44

Open victorb opened 6 years ago

victorb commented 6 years ago

As a first step with sharness, we should add it to Jenkins now when it's under the IPFS organization.

Plan is to:

cc @chriscool

chriscool commented 6 years ago
  • Have Jenkins find the latest successful build of go-ipfs:

I guess that if the sharness test is triggered by a successful build of go-ipfs, then it should be easy to find the build that is triggering it.

  • Clone sharness

Should be easy: git clone https://github.com/ipfs/ipfs-sharness-tests.git

  • Put the go-ipfs binary in the right place

Should be easy: cp ... ipfs-sharness-tests/bin/

  • Run sharness

Should be easy: cd ipfs-sharness-tests; make progressive=t

Though we have to implement what progressive=t does, see https://github.com/ipfs/testing/issues/46

  • Save the tests results

Sharness will have saved them in a folder during the above step, but we now need to save them to be able to decide if the next sharness test will pass or not. See https://github.com/ipfs/testing/issues/46 where it is discussed if they should be saved per branch or even per commit for some branches.

  • Figure out how to set Commit Status on other projects (for when sharness will be triggered from go-ipfs PR, we need to set cross-project Commit Status)

I am not sure how this works on GitHub.

chriscool commented 6 years ago

Also I think that as Sharness is saving the test results, it is better if Sharness also interprets existing test results and decide if the whole status of trying to pass the test suite is PASS or FAIL. So Sharness will need to have the previous test results, so before running Sharness, the existing test results have to be copied somewhere into the ipfs-sharness-tests folder.

There after running each test Sharness will read the corresponding information in the existing test results and either fail the test (if it should have passed), or update the test results if the test passed, or do nothing if it is ok with the test failing.