These tasks involve enhancements and optimizations to the Access and Execution Data APIs in the flow-go repository. They encompass a range of improvements, including optional event encoding, standardized JSON encoding, addition of heartbeat responses, and enabling grpc compression for more efficient communication between nodes. The tasks require a combination of code modification, algorithm implementation, benchmarking, and integration work to achieve the desired outcomes. Each task addresses specific aspects of API functionality and contributes to overall system performance and efficiency.
[Access] Add heartbeat responses to event streaming API | Fixes: [onflow/flow-go#4215]
To resolve the issue, the state_stream API ([SubscribeEvents](https://github.com/onflow/flow-go/blob/dafd8fd29c476cacc497f79940412ba377f723fe/engine/access/state_stream/handler.go#L105) call) should be expanded by adding an additional option parameter to specify the heartbeat interval. The [EventsBackend](https://github.com/onflow/flow-go/blob/dafd8fd29c476cacc497f79940412ba377f723fe/engine/access/state_stream/backend_events.go) should be upgraded by returning periodic EventsResponse with the last block id, height, and empty events when no events are found from such block interval. Existing [backend_events_test.go](https://github.com/onflow/flow-go/blob/dafd8fd29c476cacc497f79940412ba377f723fe/engine/access/state_stream/backend_events.go) tests should be upgraded and new unit tests for event streaming should be added.
Research compression and how it can be integrated to access/execution nodes communication, to compress/decompress request/response data. We will consider the 4 most likely suited algorithms for this task: gzip, DEFLATE, LZ4, Snappy.
Implement those compressing algorithms ( gzip already implemented so + 3 new ones).
Implement a synthetic benchmark to measure the compression of those algorithms. Compare to choose the most efficient algorithm.
Prepare comparison doc, discuss with the team, and decide on the best option.
Integrate, and make mandatory compressing algorithm for access/execution nodes.
Impact
Proposed fixes will improve Access Node functionality's specific aspects and contribute to overall system performance and efficiency.
Milestones and funding
Milestone 1
Milestone
Deliverables
Timeline
Risks
USD proposal
1 - [Access] Make CCF encoded events optional via request parameter
Software Engineer with over 8 years of technical experience. Primary skills include C++(Standards 11,14,17), Rust, and Golang. I have been highly focused on blockchain development for the last few years. Here are a few examples of my projects that could be interesting when applying for this grant: - 2016-2019: Supply chain planning and management tools/applications for huge manufacturers and logistic businesses. Worked in a team of 20+ engineers. Gained experience in Golang. Build integrations between applications using REST API. - 2021-2022: Complex DeFi lending/borrowing system using Rust/Solana. Smart contracts design, implementation, and testing (using typescript). Project successfully works on the market and ranks in the top 100 of Coinmarketcap. - 2022-2023: Development of Domain name service using Rust on Elrond blockchain. Smart contracts / Backend / deployment.
andriyslisarchuk@gmail.com
Uliana
Blockchain Engineer
Software Engineer with over 5 years of technical experience. Primary skills include C++(Standards 11,14,17), Rust, and Golang. I have been highly focused on blockchain development from the very beginning of my career, here are a few examples of my projects: - 2018-2021: Creation of a multi-currency light wallet, Lightning Network-ready, that features its own DEX. C++ Development of blockchain specific features, creation of custom version of private lightning network using Golang. - 2021-2022: Complex DeFi lending/borrowing system using Rust/Solana. Smart contracts design, implementation, and testing (using typescript). - 2021-2023: Developing smart contracts for NFT Marketplace platform startup using Rust and Cosmos.
Grant category
Description
These tasks involve enhancements and optimizations to the Access and Execution Data APIs in the flow-go repository. They encompass a range of improvements, including optional event encoding, standardized JSON encoding, addition of heartbeat responses, and enabling grpc compression for more efficient communication between nodes. The tasks require a combination of code modification, algorithm implementation, benchmarking, and integration work to achieve the desired outcomes. Each task addresses specific aspects of API functionality and contributes to overall system performance and efficiency.
Proposed solution
[Access] Make CCF encoded events optional via request parameter | Fixes: [onflow/flow-go#4506] To implement the functionality requested in the issue, a list of endpoints from the
AccessAPIand theExecutionDataAPIshould be modified with an additional parameter, which will specify the encoding for events, as currently it always uses CCF to JSON conversion fromMessageToEventFromVersion. If not specified, default encoding should remain JSON. All related tests for modified endpoints should be adopted and extended to test new parameters.[Access] Add heartbeat responses to event streaming API | Fixes: [onflow/flow-go#4215] To resolve the issue, the state_stream API (
[SubscribeEvents](https://github.com/onflow/flow-go/blob/dafd8fd29c476cacc497f79940412ba377f723fe/engine/access/state_stream/handler.go#L105)call) should be expanded by adding an additional option parameter to specify the heartbeat interval. The[EventsBackend](https://github.com/onflow/flow-go/blob/dafd8fd29c476cacc497f79940412ba377f723fe/engine/access/state_stream/backend_events.go)should be upgraded by returning periodicEventsResponsewith the last block id, height, and empty events when no events are found from such block interval. Existing[backend_events_test.go](https://github.com/onflow/flow-go/blob/dafd8fd29c476cacc497f79940412ba377f723fe/engine/access/state_stream/backend_events.go)tests should be upgraded and new unit tests for event streaming should be added.[Access] Enable grpc compression | Fixes: [onflow/flow-go#4503]
Impact
Proposed fixes will improve Access Node functionality's specific aspects and contribute to overall system performance and efficiency.
Milestones and funding
Milestone 1
Milestone 2
Total funding proposed(USD): 19500
Team
Primary skills include C++(Standards 11,14,17), Rust, and Golang.
I have been highly focused on blockchain development for the last few years. Here are a few examples of my projects that could be interesting when applying for this grant:
-
2016-2019: Supply chain planning and management tools/applications for huge manufacturers and logistic businesses. Worked in a team of 20+ engineers. Gained experience in Golang. Build integrations between applications using REST API.-
2021-2022: Complex DeFi lending/borrowing system using Rust/Solana. Smart contracts design, implementation, and testing (using typescript). Project successfully works on the market and ranks in the top 100 of Coinmarketcap.-
2022-2023: Development of Domain name service using Rust on Elrond blockchain. Smart contracts / Backend / deployment.Primary skills include C++(Standards 11,14,17), Rust, and Golang.
I have been highly focused on blockchain development from the very beginning of my career, here are a few examples of my projects:
-
2018-2021: Creation of a multi-currency light wallet, Lightning Network-ready, that features its own DEX. C++ Development of blockchain specific features, creation of custom version of private lightning network using Golang.-
2021-2022: Complex DeFi lending/borrowing system using Rust/Solana. Smart contracts design, implementation, and testing (using typescript).-
2021-2023: Developing smart contracts for NFT Marketplace platform startup using Rust and Cosmos.