EPIC: Twilight (0.47)

[tac0turtle]

Summary

This issue is meant to outline the next release of the sdk and QA required for all the changes.

Major Changes:

• [x] Introduction state changes for IS
• [x] Param module migration
  • params are handled by each module
  • introduction of a consensus param module to handle modifying tendermint consensus parameters
• [x] ABCI 1.0 (prepare & process proposal) & App side mempool
• [x] migrate from gogo/proto to cosmos/gogoproto
  • [ ] upgrade docs

Work Breakdown

• [ ] Audit changes (https://github.com/cosmos/cosmos-sdk/compare/release/v0.46.x...release/v0.47.x)
  • [x] ABCI
  • [x] #14259
  • [x] #13951
  • [x] #14104
  • [x] #14256
  • [x] #13904
  • [x] #14257
  • [x] Modules
  • [x] #13897
  • [x] #13906
  • [x] #13905
  • [x] #13902
  • [x] #13971
  • [x] #13938
  • [x] #14063
  • [x] #13939
  • [x] #14093
  • [x] #14212
  • [x] #13988
  • [x] #13991
  • [x] #14158
  • [x] #14255
  • [x] #14187
• [x] Release Beta/Alpha
• [ ] Public testnet
• [x] #14358
• [ ] https://github.com/cosmos/cosmos-sdk/pull/14294
• [ ] Upgrade chains with data from v0.46 to v0.47 (potentially the state of Umee)
• [ ] load test
• [ ] coordinate QA with tendermint for abci 1.0

Audit checklist

• please copy to a markdown to follow while you walk through the code

• 2 people should be assigned to each section

• the auditors should work in silo and then at the end compare notes of concerns or changes they believe should happen

• [ ] API audit

  • spec audit: check if the spec is complete.
  • Are Msg and Query methods and types well-named and organized?
  • Is everything well documented (inline godoc as well as /spec/ folder in module directory)
  • check the proto definition - make sure everything is in accordance to ADR-30 (at least 1 person, TODO assignee)
    • Check new fields and endpoints have the Since: cosmos-sdk 0.47 comment

• [ ] Completeness audit, fully implemented with tests

  • [ ] Genesis import and export of all state
  • [ ] Query services
  • [ ] CLI methods
  • [ ] All necessary migration scripts are present (if this is an upgrade of existing module)

• [ ] State machine audit

  • [ ] Read through MsgServer code and verify correctness upon visual inspection
  • [ ] Ensure all state machine code which could be confusing is properly commented
  • [ ] Make sure state machine logic matches Msg method documentation
  • [ ] Ensure that all state machine edge cases are covered with tests and that test coverage is sufficient (at least 90% coverage on module code)
  • [ ] Assess potential threats for each method including spam attacks and ensure that threats have been addressed sufficiently. This should be done by writing up threat assessment for each method. Specifically we should be paying attention to:
    • [ ] algorithmic complexity and places this could be exploited (ex. nested for loops)
    • [ ] charging gas complex computation (ex. for loops)
    • [ ] storage is safe (we don't pollute the state).
  • [ ] Assess potential risks of any new third party dependencies and decide whether a dependency audit is needed
  • [ ] Check correctness of simulation implementation if any

• [ ] Audit Changelog against commit log, ensuring all breaking changes, bug fixes, and improvements are properly documented.

There is a small testnet running as well to test against. Endpoints have been shared in slack.

Release Documentation

• Documentation
  • https://github.com/cosmos/cosmos-sdk/issues/13848
  • https://github.com/cosmos/cosmos-sdk/issues/13488
  • https://github.com/cosmos/cosmos-sdk/issues/13849
  • https://github.com/cosmos/cosmos-sdk/issues/11504

This will be a living epic until twilight is released.

---

Previous version EPIC: https://github.com/cosmos/cosmos-sdk/issues/11096, https://github.com/cosmos/cosmos-sdk/issues/11362

[alexanderbez]

I'd like to propose removing IS from Twilight.

[tac0turtle]

IS or LSM?

[alexanderbez]

Sorry, LSM.

[ethanfrey]

Is this the only issue tracking what is needed for 0.47 release? Or do you need to close all these issues: https://github.com/cosmos/cosmos-sdk/labels/R%3ATwilight ?

I ask, as it seemed 0.47 was more of a stable cleanup, while most of the remaining issues are hefty refactorings, nice to haves, or docs.

Docs are not blocking a 0.47.0-alpha. Which of these are needed for release?

• https://github.com/cosmos/cosmos-sdk/issues/13073 seems VERY big, and an additive change that could be added in eg 0.47.1 without affecting other modules.
• https://github.com/cosmos/cosmos-sdk/issues/12272 seems to need some larger rewrites to do it properly, and maybe best to put that into 0.48
• The protobuf annotations https://github.com/cosmos/cosmos-sdk/issues/13407 and https://github.com/cosmos/cosmos-sdk/issues/13405 are coming from someone who is not developing client tools (or at least nothing widely used). I would check with @pyramation if those help him, as he maintains telescope, which also generates the protobuf bindings for CosmJS now, as well as for custom chains. As his code covers 90%+ of all protobuf client usage - I would only add what he finds essential.

Besides those, it would be basically ready for rc and proper testing. I think https://github.com/cosmos/cosmos-sdk/issues/13408 is key and giving time for an rc and other repos to adapt to it and run together on some public testnets would be great for a solid 0.47.0 release

[robert-zaremba]

Why removing LSM? Isn't it ready? cc @alexanderbez

[pyramation]

• The protobuf annotations Add amino JSON proto annotations #13407 and Require cosmos.msg.v1 annotations #13405 are coming from someone who is not developing client tools (or at least nothing widely used). I would check with @pyramation if those help him, as he maintains telescope, which also generates the protobuf bindings for CosmJS now, as well as for custom chains. As his code covers 90%+ of all protobuf client usage - I would only add what he finds essential.

Definitely let's include (if possible) Add amino JSON proto annotations #13407 — I think it just needs last final approval from @ValarDragon in feat: Add proto annotations for Amino JSON #13501

[alexanderbez]

Why removing LSM? Isn't it ready? cc @alexanderbez

It still need review by core devs. We reviewed proto changes only IIRC.

[tac0turtle]

closing this, we are waiting on ibc to tag a alpa tag to do a public testnet now

[aaronc]

Let's just make sure the file descriptor bug fix (#14713) is tracked to go in before release

[alexanderbez]

Let's just make sure the file descriptor bug fix (#14713) is tracked to go in before release

If this is a critical blocker, technically this issue should not have been closed until that's merged.

[aaronc]

Just to restate the reason for considering it critical - fixing it will result in startup errors for chains with bad descriptors, IMHO better to not push that off to a patch release

[tac0turtle]

its tracked as critical in the project board. we will still run a testnet with ibc and wasm so there is still time. We can reopen if needed.

[alexanderbez]

Yeah sure, but the tracking issue epic shouldn't be closed until all critical items are addressed.

[aaronc]

Is there a reason we're not tracking releases with milestones now? IMHO it's pretty convenient for this sort of stuff

[tac0turtle]

Reopening, we track open issues for a release with the label. In this case it wasn't added to the issue

[julienrbrt]

https://github.com/cosmos/cosmos-sdk/issues/14713 has been closed.