fediverse-devnet / feditest

A testing framework for distributed, heterogeneous systems communicating with complex protocols, such as the Fediverse
https://feditest.org/
MIT License
32 stars 6 forks source link

[WIP] Experimental Mastodon API Node #178

Closed steve-bate closed 3 months ago

steve-bate commented 4 months ago

This has some node-specific unit tests that exercise several of the ActivityPubNode API methods. I didn't try to run it with the current AP tests and I expect there will be some issues with that. I needed to make local changes to the tests when I tested an earlier version of the Node.

jernst commented 3 months ago

Overall, I think this is basically what I have in mind. It would be nice if we could manage to do without hard-coding a specific number of actors or having to pre-provision them before the test run because that kinds of limits what kinds of tests we can write. Not sure how realistic it is to programmatically do that, however.

What about we pick a single, simple test as the next target. Say https://github.com/fediverse-devnet/feditest-tests-fediverse/blob/develop/tests/fediverse/follow/leader_accepts_follow_request.py or maybe just send a Note to a previously unrelated other Actor on a different server. And make this work in a fully-automated way on UBOS with two Mastodon instances talking to each other. That would prove the point that we can make it work and expanding it from there should be straightforward?

steve-bate commented 3 months ago

Overall, I think this is basically what I have in mind. It would be nice if we could manage to do without hard-coding a specific number of actors or having to pre-provision them before the test run because that kinds of limits what kinds of tests we can write. Not sure how realistic it is to programmatically do that, however.

Do you mean "hard-coded" in the provisioning infrastructure (versus the node implementation or configuration)?

What about we pick a single, simple test as the next target. Say https://github.com/fediverse-devnet/feditest-tests-fediverse/blob/develop/tests/fediverse/follow/leader_accepts_follow_request.py or maybe just send a Note to a previously unrelated other Actor on a different server. And make this work in a fully-automated way on UBOS with two Mastodon instances talking to each other. That would prove the point that we can make it work and expanding it from there should be straightforward?

FWIW, in an older version of the framework, I had that test and a few others running with a single manually-provisioned Mastodon providing both actors. This isn't federated behavior, but it tests the Mastodon node functionality.

jernst commented 3 months ago

I think we can do the automatic account and token provisioning, see discussion here: https://matrix.to/#/!uHqAjmOtrLtidOiczC:matrix.org/$_PUyfFvVjG8GaaNpbFAMxr0iAQq24zXU-b-Yow9CMq8 . The API doesn't have a "now create me a new account" factory method right now because I figured that might be difficult, and so currently it only asks for an existing account, but it might be very advantageous if we can do that.

steve-bate commented 3 months ago

I think we can do the automatic account and token provisioning, see discussion here: https://matrix.to/#/!uHqAjmOtrLtidOiczC:matrix.org/$_PUyfFvVjG8GaaNpbFAMxr0iAQq24zXU-b-Yow9CMq8 . The API doesn't have a "now create me a new account" factory method right now because I figured that might be difficult, and so currently it only asks for an existing account, but it might be very advantageous if we can do that.

I agree it could be useful, where it's possible. I'm thinking this Mastodon node could be used for any server that implements the Mastodon API (at least the methods we need). However, AFAIK, only Mastodon itself implements tootctl.

After we convert the API node into a mixin class and add it to the UBOS node, it's possible the UBOS node driver could do whatever it does to create the actors and add those to the parameters to satisfy the mixin expectations.

steve-bate commented 3 months ago

Unless you think it might break existing functionality, I'd like to merge the existing Mastodon API node implementation and the test example code in feditest-tests-fediverse. After that, I can convert the node to a mixin class that can be used for both UBOS and non UBOS nodes.