search

endojs / endo

Endo is a distributed secure JavaScript sandbox, based on SES
Apache License 2.0
762 stars 68 forks source link

Agoric Integration Testing #1866

Open kriskowal opened 7 months ago

kriskowal commented 7 months ago

What is the Problem Being Solved?

Agoric periodically syncs the Endo repository with the Agoric SDK repository and this process often involves fixing unforeseen compatibility issues. Agoric tests are often overconstrained. Running some “canary” Agoric SDK tests in Endo CI and requiring them to pass should help us discover these unintended breaking changes early.

Description of the Design

Agoric SDK already has the ability to CI against Endo packages on any branch. We should reconstruct some of this infrastructure in the Endo CI workflows. This involves checking out Agoric SDK and using its scripts to pack up Endo as if it were released and patching them into Agoric SDK with Yarn resolutions. We’ve recently discovered that care is needed to ensure all Endo packages have their types generated before packing them up.

Security Considerations

The Endo org should not hold Agoric org CI keys.

Scaling Considerations

My hope is that we can find a sufficient subset of Agoric SDK tests that will reveal incompatibility without prolonging CI in Endo. Similarly, we would not want to impose the “Test in Windows” constraint on Agoric SDK.

Test Plan

The feature is the tests. Tests are the feature.

Upgrade Considerations

mhofman commented 3 months ago

To summarize what I've written in Slack before: we can use a repository_dispatch workflow trigger, then poll the workflow runs to get the result of the triggered jobs. It can be difficult to match the workflows to their trigger, and some community suggestion can help:

We already have endo branch override logic setup in agoric-sdk (but it does not currently cover docker based tests)