iggy-rs / iggy

Iggy is the persistent message streaming platform written in Rust, supporting QUIC, TCP and HTTP transport protocols, capable of processing millions of messages per second.
https://iggy.rs
MIT License
1.84k stars 85 forks source link

Implement new server storage #1015

Closed spetz closed 3 months ago

spetz commented 3 months ago

This is a new, custom implementation of the server storage based on a "command-sourcing" log, which is required for the future replication purposes. The existing streams/topic/partitions/segments storage remains unchanged. This implementation removes the usage of Sled embedded DB and comes with a built-in migration.

coveralls commented 3 months ago

Pull Request Test Coverage Report for Build 9668042121

Details


Changes Missing Coverage Covered Lines Changed/Added Lines %
sdk/src/cli/personal_access_tokens/get_personal_access_tokens.rs 3 4 75.0%
sdk/src/cli/users/get_users.rs 1 2 50.0%
sdk/src/models/messages.rs 2 3 66.67%
sdk/src/personal_access_tokens/create_personal_access_token.rs 12 13 92.31%
sdk/src/utils/duration.rs 3 4 75.0%
server/src/binary/handlers/consumer_groups/create_consumer_group_handler.rs 22 23 95.65%
server/src/channels/commands/clean_messages.rs 1 2 50.0%
server/src/configs/displays.rs 1 2 50.0%
server/src/http/streams.rs 35 36 97.22%
server/src/state/file.rs 132 133 99.25%
<!-- Total: 1958 3422 57.22% -->
Files with Coverage Reduction New Missed Lines %
server/src/streaming/streams/storage.rs 1 92.31%
server/src/streaming/systems/messages.rs 1 65.79%
server/src/streaming/systems/system.rs 1 74.13%
server/src/http/jwt/storage.rs 1 71.64%
server/src/http/users.rs 1 93.81%
server/src/streaming/segments/time_index.rs 1 0.0%
server/src/streaming/partitions/messages.rs 1 82.87%
server/src/streaming/topics/consumer_groups.rs 1 88.52%
server/src/streaming/systems/users.rs 1 89.23%
server/src/configs/system.rs 1 86.09%
<!-- Total: 66 -->
Totals Coverage Status
Change from base Build 9659631488: -2.8%
Covered Lines: 23611
Relevant Lines: 29082

💛 - Coveralls
coveralls commented 3 months ago

Pull Request Test Coverage Report for Build 9668073157

Details


Changes Missing Coverage Covered Lines Changed/Added Lines %
sdk/src/cli/personal_access_tokens/get_personal_access_tokens.rs 3 4 75.0%
sdk/src/cli/users/get_users.rs 1 2 50.0%
sdk/src/models/messages.rs 2 3 66.67%
sdk/src/personal_access_tokens/create_personal_access_token.rs 12 13 92.31%
sdk/src/utils/duration.rs 3 4 75.0%
server/src/binary/handlers/consumer_groups/create_consumer_group_handler.rs 22 23 95.65%
server/src/channels/commands/clean_messages.rs 1 2 50.0%
server/src/configs/displays.rs 1 2 50.0%
server/src/http/consumer_groups.rs 36 37 97.3%
server/src/http/partitions.rs 24 25 96.0%
<!-- Total: 1960 3422 57.28% -->
Files with Coverage Reduction New Missed Lines %
server/src/streaming/streams/storage.rs 1 92.31%
server/src/streaming/systems/system.rs 1 74.13%
server/src/http/jwt/storage.rs 1 71.64%
server/src/http/users.rs 1 94.29%
server/src/streaming/segments/time_index.rs 1 0.0%
server/src/streaming/partitions/messages.rs 1 82.87%
server/src/streaming/topics/consumer_groups.rs 1 88.52%
server/src/streaming/systems/users.rs 1 89.23%
server/src/configs/system.rs 1 86.09%
sdk/src/models/permissions.rs 2 86.33%
<!-- Total: 65 -->
Totals Coverage Status
Change from base Build 9659631488: -2.7%
Covered Lines: 23614
Relevant Lines: 29082

💛 - Coveralls