iggy-rs / iggy

Iggy is the persistent message streaming platform written in Rust, supporting QUIC, TCP and HTTP transport protocols, capable of processing millions of messages per second.
https://iggy.rs
MIT License
1.84k stars 85 forks source link

Implement new server storage #1016

Closed spetz closed 3 months ago

spetz commented 3 months ago

This is a new, custom implementation of the server storage based on a "command-sourcing" log, which is required for the future replication purposes. The existing streams/topic/partitions/segments storage remains unchanged. This implementation removes the usage of Sled embedded DB and comes with a built-in migration.

coveralls commented 3 months ago

Pull Request Test Coverage Report for Build 9741717848

Details


Changes Missing Coverage Covered Lines Changed/Added Lines %
sdk/src/binary/mapper.rs 19 20 95.0%
sdk/src/cli/personal_access_tokens/get_personal_access_tokens.rs 3 4 75.0%
sdk/src/cli/users/get_users.rs 1 2 50.0%
sdk/src/personal_access_tokens/create_personal_access_token.rs 17 18 94.44%
sdk/src/utils/duration.rs 4 5 80.0%
server/src/binary/handlers/consumer_groups/create_consumer_group_handler.rs 21 22 95.45%
server/src/channels/commands/clean_messages.rs 1 2 50.0%
server/src/compat/message_conversion/snapshots/message_snapshot.rs 0 1 0.0%
server/src/configs/displays.rs 1 2 50.0%
server/src/http/consumer_groups.rs 28 29 96.55%
<!-- Total: 2901 4466 64.96% -->
Files with Coverage Reduction New Missed Lines %
server/src/streaming/streams/storage.rs 1 92.31%
server/src/binary/handlers/consumer_groups/delete_consumer_group_handler.rs 1 96.15%
server/src/streaming/systems/system.rs 1 74.13%
sdk/src/command.rs 1 0.0%
server/src/http/jwt/storage.rs 1 71.64%
server/src/http/users.rs 1 94.0%
server/src/streaming/topics/consumer_groups.rs 1 88.52%
server/src/streaming/systems/users.rs 1 89.23%
server/src/configs/system.rs 1 86.09%
sdk/src/models/permissions.rs 2 86.33%
<!-- Total: 64 -->
Totals Coverage Status
Change from base Build 9741578452: -2.9%
Covered Lines: 23559
Relevant Lines: 29069

💛 - Coveralls
coveralls commented 3 months ago

Pull Request Test Coverage Report for Build 9747859729

Details


Changes Missing Coverage Covered Lines Changed/Added Lines %
sdk/src/binary/mapper.rs 19 20 95.0%
sdk/src/cli/personal_access_tokens/get_personal_access_tokens.rs 3 4 75.0%
sdk/src/cli/users/get_users.rs 1 2 50.0%
sdk/src/personal_access_tokens/create_personal_access_token.rs 17 18 94.44%
sdk/src/utils/duration.rs 4 5 80.0%
server/src/binary/handlers/consumer_groups/create_consumer_group_handler.rs 21 22 95.45%
server/src/channels/commands/clean_messages.rs 1 2 50.0%
server/src/compat/message_conversion/snapshots/message_snapshot.rs 0 1 0.0%
server/src/configs/displays.rs 1 2 50.0%
server/src/http/consumer_groups.rs 28 29 96.55%
<!-- Total: 2901 4483 64.71% -->
Files with Coverage Reduction New Missed Lines %
server/src/streaming/streams/storage.rs 1 92.31%
server/src/binary/handlers/consumer_groups/delete_consumer_group_handler.rs 1 96.15%
server/src/streaming/systems/system.rs 1 74.13%
sdk/src/command.rs 1 0.0%
server/src/http/jwt/storage.rs 1 71.64%
server/src/http/users.rs 1 94.0%
cli/src/credentials.rs 1 90.23%
server/src/streaming/topics/consumer_groups.rs 1 88.52%
server/src/streaming/systems/users.rs 1 89.23%
server/src/configs/system.rs 1 86.09%
<!-- Total: 65 -->
Totals Coverage Status
Change from base Build 9741578452: -3.0%
Covered Lines: 23558
Relevant Lines: 29086

💛 - Coveralls