hyperledger / indy-test-automation

Automation tools for testing of Indy Project components.
https://wiki.hyperledger.org/display/indy
Apache License 2.0
6 stars 23 forks source link

Test migration to VDR / Askar / CredX + DCO #125

Closed VladimirWork closed 1 year ago

VladimirWork commented 1 year ago

Indy-node system tests migration from Indy SDK (libindy) to Indy VDR, Aries Askar and Indy CredX libraries. Sovtoken-related tests are kept untouched since libsovtoken depends on libindy, not indy-vdr. Test pipeline after all changes: https://github.com/VladimirWork/indy-test-automation/actions/runs/4135106879

WadeBarnes commented 1 year ago

Apologies it's taken so long to get to this review. I've got two instances of the tests running in a real world scenario.

The run with the payment tests enabled has encountered some test failures. Would you be able to have a look?

The run with the payment tests disabled has only encountered one test failure, but that looks like a timeout. I'll rerun the tests again once complete, I expect the test will pass when rerun. Edit: All the tests passed when rerun as anticipated.

~I'm still in the process of reviewing the code.~

VladimirWork commented 1 year ago

@WadeBarnes Hi, PR is ready - we updated error handling and split tests and pipelines into 2 parts - new VDR-based and old token-related according to the selected option:

Option 2: Keep sovrin token tests untouched and still depending on libsovtoken and libindy We’ve decided that Option 2 is the best way to proceed. It will be a Sovrin effort to migrate the payment related tests to use indy-vdr and to upgrade libsovtoken to use indy-vdr.

VladimirWork commented 1 year ago

@WadeBarnes

Regarding payment pipeline failure:
Dockerfile.ubuntu-2004:110
--------------------
 109 |     RUN echo "SOVRIN_INSTALL: ${SOVRIN_INSTALL}"
 110 | >>> RUN if [ "${SOVRIN_INSTALL}" = "yes" ]; then \
 111 | >>>         sovrin_package=$(find ./ -name "sovrin_*.deb" | head -n 1); \
 112 | >>>         echo "Installing ${sovrin_package} ..."; \
 113 | >>>         dpkg -i "${sovrin_package}"; \
 114 | >>>     fi
 115 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c if [ \"${SOVRIN_INSTALL}\" = \"yes\" ]; then         sovrin_package=$(find ./ -name \"sovrin_*.deb\" | head -n 1);         echo \"Installing ${sovrin_package} ...\";         dpkg -i \"${sovrin_package}\";     fi" did not complete successfully: exit code: 1

Please recheck payment pipeline with the latest changes from this PR.

pSchlarb commented 1 year ago

I am also not sure how these changes to the pipeline affect or conflict witht the changes from https://github.com/hyperledger/indy-test-automation/pull/126 and https://github.com/sovrin-foundation/sovrin/pull/348

And the upcomming node integration. Sorry i hadn't had the time to check for it. I hope i get this week some time to check it.

VladimirWork commented 1 year ago

@WadeBarnes I've forked Sovrin main and run tests from [VladimirWork:vdr-askar-migration-copy] (https://github.com/VladimirWork/indy-test-automation/tree/vdr-askar-migration-copy) with payments, here is the result: https://github.com/VladimirWork/sovrin/actions/runs/4391344953 with 2 failures: TestEndorserSuite (as expected) and TestAdHocSuite with some strange exit code - looks like GHA runner issues (yes, it's green on the second attempt: https://github.com/VladimirWork/sovrin/actions/runs/4391344953/jobs/7710164070), I'll rerun this job, but it looks like it was intermittent failure that doesn't relate to test code or env scripts in the indy test automation repo.

WadeBarnes commented 1 year ago

Thanks @VladimirWork. I'm rerunning things here to confirm the results; https://github.com/WadeBarnes/sovrin/actions/runs/4406874031

VladimirWork commented 1 year ago

@WadeBarnes Your pipeline also looks good, I fixed conflicts with main and switched jobs back to hyperledger repo.

WadeBarnes commented 1 year ago

@VladimirWork, Test results confirmed. Results as expected. Do you want to squash any of the commit history to clean things up a bit before I merge?

VladimirWork commented 1 year ago

@WadeBarnes I squashed some unnecessary commits, I believe now PR is ready for merging.