Add tests for engine and processors

[glihm]

Torii engine is in charge of pulling the events and processing each one of them using the corresponding processor.

More tests have to be added to ensure the modifications made to the engine and processors are correctly done.

The engine is here: https://github.com/dojoengine/dojo/blob/22072dd62792fd3625d5c02c1f10254b6b10f9f5/crates/torii/core/src/engine.rs The processors are here: https://github.com/dojoengine/dojo/blob/22072dd62792fd3625d5c02c1f10254b6b10f9f5/crates/torii/core/src/processors/mod.rs

Two approaches to write those tests:

1. Mocking: by not using any underlying node, all processors can first be tested with mocked events, constructed by end, to ensure we can extract correctly the information for a well formatted event.
2. To test the logic around the event fetching and pagination, the starknet provider can also be mocked, or Katana test runner may be use.

This issue can be tackled in two PR if necessary.

[mubarak23]

@glihm i will love to work on this issue

[glihm]

Hey @mubarak23, thanks for the will to contribute! Do you have already an idea on how you will tackle that? Do you have any question on the event content for testing?

[mubarak23]

@glihm i have some questions,

• What are the key components of an event
• How do events relate to other entities
• What criteria are used to validate whether an event is valid or should be processed

[glihm]

What are the key components of an event

An event is composed of two arrays of felts. keys and data. In the case of keys, the first element is always the event selector (which is a hash that identifies the name of the event). data contains all fields being serialized, if they are not annotated with #[key] attribute (which put them into keys).

How do events relate to other entities

To which entities are you referring to?

What criteria are used to validate whether an event is valid or should be processed

This depends on the processor. In Torii, you'll see that each processor implementation should have a validate function: https://github.com/dojoengine/dojo/blob/22072dd62792fd3625d5c02c1f10254b6b10f9f5/crates/torii/core/src/processors/store_set_record.rs#L26

The goal of this issue is to organize tests around this.

The principal challenge here is that, generally the processor require to fetch data from the DB and store data to the DB. Ideally, we want those tests to be as unit as we can. Which could imply:

1. Refactoring the code to have smaller and easily testable function that are not using the DB (for example, a function that only parse the entity from data)
2. Doing a test as we have in sql_test.rs where we have to run a katana instance, initialize a DB and work around that.

I would say 1 is better, but if too challenging, you can write tests for each of the processors using 2. The downside of two is that the migration is considered very slow, so this will significantly impact the CI.

[mubarak23]

@glihm i will like to tackle this issue using second approach [2] by following these outlines

• Valid event with correct keys and data.
• Invalid event with incorrect keys or data.
• Error scenarios such as database errors or deserialization failures. but i will love to hear your though process on how we can achieve 1

[glihm]

Hum, sorry @mubarak23 I've not given any followup here. Will unassign please let us know if you are still willing to work on that.

[PedroRosalba]

@glihm Can I work on this?

[glihm]

@PedroRosalba any idea on how you would tackle that? If yes, all yours!

[g4titanx]

hi @glihm, i would love to work on this

[glihm]

hi @glihm, i would love to work on this

All yours, let us know if you have any question. 👍

[g4titanx]

Two approaches to write those tests:

1. Mocking: by not using any underlying node, all processors can first be tested with mocked events, constructed by end, to ensure we can extract correctly the information for a well formatted event.
2. To test the logic around the event fetching and pagination, the starknet provider can also be mocked, or Katana test runner may be use.

This issue can be tackled in two PR if necessary.

@glihm, i would suggest an hybrid approach. for testing the core logic of individual processors (BlockProcessor, EventProcessor and TransactionProcessor). we would mock events and providers to verify that the processors can correctly handle and process events.

then, for testing the overall flow of the engine, including event fetching, processing, and pagination. we could use a test runner like Katana to simulate the blockchain environment and ensure that the engine interacts correctly with the blockchain and processes events as expected.