nats-io / nats-server

High-Performance server for NATS.io, the cloud and edge native messaging system.
https://nats.io
Apache License 2.0
15.99k stars 1.41k forks source link

limit maxBytes with compression calculate uncompressed size #5905

Open paragor opened 2 months ago

paragor commented 2 months ago

Observed behavior

I have stream with compression and limit maxBytes 6Gi

apiVersion: jetstream.nats.io/v1beta2
kind: Stream
metadata:
  name: logs
spec:
  name: logs
  subjects: ["logs.*"]
  storage: file
  maxAge: 12h
  maxBytes: 6442450944
  replicas: 3
  account: logs
  compression: s2

but actualy disk space occupied 536.8M 5.5 gigabytes underutilized

/data/jetstream/$G/streams/logs # df -h /data
Filesystem                Size      Used Available Use% Mounted on
/dev/vdb                  9.7G    536.8M      9.2G   5% /data
/data/jetstream/$G/streams/logs # curl localhost:7777/metrics -s | grep nats_stream_total_bytes
# HELP nats_stream_total_bytes Total stored bytes from a stream
# TYPE nats_stream_total_bytes gauge
nats_stream_total_bytes{account="$G",account_id="$G",cluster="nats",domain="",is_meta_leader="true",is_stream_leader="true",meta_leader="nats-0",server_id="NC6ARVXMQOBMIQNT3ULUK2MVNLX43DPZD43YKA3OMKHGTXQD55MIP357",server_name="nats-0",stream_leader="nats-0",stream_name="logs"} 6.442450821e+09
> nats stream info logs
Information for Stream logs created 2024-09-17 22:50:57

              Subjects: logs.*
              Replicas: 3
               Storage: File
           Compression: S2 Compression

Options:

             Retention: Limits
       Acknowledgments: true
        Discard Policy: Old
      Duplicate Window: 2m0s
     Allows Msg Delete: true
          Allows Purge: true
        Allows Rollups: false

Limits:

      Maximum Messages: unlimited
   Maximum Per Subject: unlimited
         Maximum Bytes: 6.0 GiB
           Maximum Age: 12h0m0s
  Maximum Message Size: unlimited
     Maximum Consumers: unlimited

Cluster Information:

                  Name: nats
                Leader: nats-0
               Replica: nats-1, current, seen 21ms ago
               Replica: nats-2, current, seen 21ms ago

State:

              Messages: 6,157,140
                 Bytes: 6.0 GiB
        First Sequence: 27,979,737 @ 2024-09-19 04:05:43
         Last Sequence: 34,136,876 @ 2024-09-19 13:56:38
      Active Consumers: 0
    Number of Subjects: 11

Expected behavior

nats calculate maxBytes for limits as actual size on disk

Server and client version

nats-server: v2.10.19

Host environment

No response

Steps to reproduce

No response

ripienaar commented 2 months ago

Limits are always calculated on the message size rather than how the storage would store it. We have had a few issues filed around this in the past but this is working as designed