hadronous / pic-js

An Internet Computer Protocol canister testing library for TypeScript and JavaScript.
https://hadronous.github.io/pic-js/
Apache License 2.0
11 stars 6 forks source link

Simulated errors during intercanister calls #25

Closed infu closed 9 months ago

infu commented 9 months ago

Not sure if this issue is for this repo or pocket-ic repo. I've managed to force an error "could not perform remote call" when one canister calls another using pocket-ic, but it's not a very reliable simulation and takes a lot of CPU. Perhaps we can have something like this:

canone.forceErrorWhenCalling(cantwo, errCode);
await canone.doCall(); // it calls cantwo inside

and also simulate the case when canone calls cantwo but doesn't receive a callback

canone.noCallback(cantwo);
await canone.doCall();
nathanosdev commented 9 months ago

Unfortunately in both of these cases, there's nothing that pic-js can do to simulate these errors because it doesn't have anything to do with the execution of the canisters and that would be on the pocketic side.

What kind of error are you trying to simulate when canone calls cantwo but fails? Is this meant to emulate a protocol failure that cannot deliver the message? Or is it meant to emulate the target canister trapping when it receives the call?

Similarly for the callback scenario, is this the protocol failing to deliver the callback? Or is the target canister just not sending the callback because it had some issue?

If it's essentially a misbehaving canister that you want to emulate, then you could put together "mock" canisters that would misbehave however you want. One mock canister could trap when you call it and another mock canister could accept the call but then never send the callback.

If it's protocol-level errors that you want to emulate then I'd recommend opening a ticket on the pocketic repo to see what they think.

infu commented 9 months ago

Thanks for the tips. So far these tests work great for multi-canister scenarios. I just want to get it working as realistic as possible. Like overloading a canister and checking what errors occur. Otherwise, we are just testing logic, but can't stress test. I can't easily overload a canister, because pocket-ic is too fast, and there is something different when making calls from the tests to canisters. Something like this will just make 1000 calls and they won't be parallel. So Instead I've made a canister that sends the calls and then I finally got the error I am looking for.

  it("parallel calls", async () => {

    let re = await Promise.all(Array(1000).fill(0).map(async (_,idx) => {
      let uid = createIdentity('something'+idx);
      user.setIdentity(uid);
      let rez = await can.dosomething();
      expect(rez).toEqual({ok:null});
    }));

  });

cantwo in my case is a ledger and I am testing our ledger middleware in canone. I can't change the ledger easily - using icrc ledger from SNSW. So - not trying to emulate misbehaving canister, but protocol errors.

nathanosdev commented 9 months ago

I've spoken also with the PocketIC team and there doesn't seem to be an easy way to solve this. Internally PocketIC is using the same replica code that's used on mainnet, so forcing calls to fail in some way would require changes to the replica. Feel free to open a new ticket on the pocketic repo if you'd like to contrinue the discussion there, but there's nothing that I can do on the pic-js side so I'll close this ticket.