EPIC: Simulator - Githubissues

tac0turtle commented 1 year ago

Summary

This EPIC is the start of a tracking issue for features we would like to have in a simulator. We should identify the pitfalls of the current design, if any, and aim to provide a better user experience

One feature that would be nice is extend the simulator from a dumb fuzzer to a more intelligent fuzzer where improper messages would be caught if they errored.

Work Breakdown

TBD

I'd like to invite people to comment what features they would like to see from the simulator

alexanderbez commented 1 year ago

I know dYdX wants https://github.com/cosmos/cosmos-sdk/issues/13843

robert-zaremba commented 1 year ago

Error reporting. Today, tracing an error in simulations is a super unfriendly.
Better high level functions to register and handle operations. Today, there is lot of boilerplate to deliver an operation.

gjermundgaraba commented 1 year ago

I have implemented simulations from 0 this week, and I have some feedback:

I want much, much less boilerplate. I'd rather the simulation framework was much less flexible and just worked if I implemented a couple of interfaces. In particular, the whole thing falls apart after you implement the module simulation interfaces. It's so much stuff to wire together afterwards. It is also not great that I essentially have to duplicate a lot of logic into the operation functions so they don't crash and burn.
The whole setup of the runsim binary is confusing and seems unnecessary, especially since it doesn't output good error messages. I think I would much prefer to not have an extra binary, unless it somehow becomes the key to remove all the boilerplate needed.
I would like to have all things simulator related implementations be in one place. I don't want this concept spread all over the chain code. It is messy and hard to keep straight in your head.

If anyone is interested, I can link the PR for this change to illustrate how much appears to be needed, just get started.

Since this is more about features, I just leave my feedback on docs here at the bottom so they are somewhere at least:

Docs need to explain more deeply how simulations are supposed to work (or rather, how your implementations are supposed to behave)
Docs need to cover the whole process from A-Z, today it doesn't

alexanderbez commented 1 year ago

This is great feedback @gjermundgaraba, thank you!

elias-orijtech commented 10 months ago

If anyone is interested, I can link the PR for this change to illustrate how much appears to be needed, just get started.

@gjermundgaraba I'm working on the simulator rewrite I'd like to see the PR you did to integrate simulation.

gjermundgaraba commented 10 months ago

If anyone is interested, I can link the PR for this change to illustrate how much appears to be needed, just get started.

@gjermundgaraba I'm working on the simulator rewrite I'd like to see the PR you did to integrate simulation.

Happy to! Here it is :) https://github.com/EmpowerPlastic/empowerchain/pull/469

fmorency commented 1 month ago

I started adding simulation support in our chain and agree with @gjermundgaraba.

My main ask is a hands-on guide/tutorial that explains how to add simulator support to your CosmosSDK app.

Happy to help!

tac0turtle commented 1 month ago

thank you, @alpe is doing a cleanup right now with some new changes, after this we will create a tutorial on how to write simulations. :)

cosmos / cosmos-sdk

EPIC: Simulator #14753

Summary

Work Breakdown