jsAccountLimits::MaxBytesRequired is a boolean, should it be int64_t?

[NostraMagister]

Observed behavior

The NATS C Client structure jsAccountInfo has a member Limits that holds a jsAccountLimits structure.

This structure in turn has a member MaxBytesRequired of which the datatype is a bool.

Shouldn't that member be defined as int64_t as are the other jsAccountInfo::Limits::... members?

Expected behavior

Expected it to be a numeric variable

Server and client version

NATS C Client

Host environment

Windows/Linux VS C/C++

Steps to reproduce

Just check the structure jsAccountLimits::MaxBytesRequired

[kozlovic]

No, this is really a boolean. It is an account's limit that forces all streams to require a MaxBytes value to be set. In other word, if this boolean is enabled for the account limit and a stream is created without MaxBytes value, the stream creation will fail.

[NostraMagister]

Thank you.

MaxByteSettingRequired might be a name that could emphasize 'booleaness'.

[kozlovic]

This is coming from the server/jwt's library field names. We can't change that now so as to not break backward compatibility. What is missing, is brief documentation after each field :-)

[NostraMagister]

For what it's worth, I have started to write documentation like the one below. If that helps feel free to cut & paste and fill in some ?????. This is for nats.h more specifically jsStreamConfig. Ill have the same for consumers. English is my 3rd language, so some sentence may sound a little strange.

/**

• Configuration of a JetStream stream.
• The creation (#js_AddStream) and updating (#js_UpdateStream) of a stream require a
• configuration as an input argument. This configuration can be newly created
• and initialized (#jsStreamConfig_Init) and sensible defaults will be supplied for most members
• as indicated in the documentation of these individual members.
• The stream configuration can also be obtained through the stream name (#js_GetStreamInfo) as
• the 'Config' member of the jsStreamInfo structure. This structure must be destroyed by the application
• sooner or later with the #jsStreamInfo_Destroy and hence will destroy the imbedded Config member as
• applicable. The application does NOT own the imbedded strings in that case. If the Config is needed to
• be used with #js_UpdateStream then the structure must be copied first and only the strings that would be
• overwritten in that copy remain owned by the application.
• The members of a configuration may be updated before giving the configuration to one of
• the #js_AddStream or #js_UpdateStream functions. The application remains owner and deallocation
• wise responsible, of all strings, which when allocated on the heap may need deallocation. The
• stream configuration is no longer needed after the function returns, the stream name suffices to
• add consumers to the stream, to delete the stream with #js_DeleteStream or to use the JetStream
• specific persistent synchronous and asynchronous publish and subscription functions.
• From an architecture perspective one or more JetStream streams, by means of this jsStreamConfig configuration,
• are unique entities in a JetStream Context (jsCtx). Just as multiple context allow fine grained tuning by applying
• different policies, JetStream streams allow the same. Consumers are associated with individual JetStream stream.
• A typical use is to allocate a jsStreamConfig pointer and then either allocate a jsStreamConfig structure and initialize it or retrieve a jsStreamConfig as explained above and make a copy of it to point to. Next the configuration can be updated with
• the settings as whished and passed to the add or update stream function. On return the application needs to
• deallocate the jsStreamConfig copy and sub-strings as applicable. This approach allows to manage large collections of
• stream, based on retrieved data, for instance from KeyValue stores or databases, without the need to keep
• al configuration data in memory. Keeping stream names suffices as the rest can be retrieved.

typedef struct jsStreamConfig { /* A zero-terminated string with the name of the stream. It may contain the alphanumeric

• values (a-z, A-Z, 0-9) and the underscore '_'. It is advised to keep the stream name under
• 32 characters, because the JetStream storage directories will include the account, stream name,
• and consumer name. The application remains the owner of the strings in the jsStreamConfig structure
• and is responsible for deallocation as applicable for the type of string (stack/heap allocated).
• The stream name must be unique within all streams created with the SAME JetStream context (jsCtx).
• It is the only stream related parameter that the application needs for JetStream publish and subscribe
• functions or to associate consumers to the stream.
• If however the configuration structure is needed, it can be retrieved with #js_GetStreamInfo where it is
• present in the .Config member of the retuned jsStreamInfo structure. The jsStreamConfig stream structure
• can then be copied, the copy updated and used with the #UpdateStream function.
• Finally, the stream name can be used to delete the stream with #DeleteStream.

• */
  const char              *Name;
  
  /* A zero-terminated string with a humanly readable description of the intended use of the stream. This string

• can contain most characters as it isn't used in any technical capacity. There is NO default setting. / const char Description;

  /* A array of zero-terminated strings with subjects. If there are no subjects provided the library will use the stream 

• name as the default subject. The default setting is ????? */ const char **Subjects;

  /* An positive value, or zero, indicated the number of elements in the Subjects array. If the value is zero there will

• still by at least one subject because the library will use the stream Name as the default topic. */ int SubjectsLen;

  /* A variable holding one of the values as defined by jsRetentionPolicy. The retention policy tells the NATS server 

• when to discard messages for this specific stream and within authorized boundaries deviate from higher priority
• settings. The might themselves be overwriting by consumer settings for the same parameter. At the moment of writing
• these are the possibilities.
• jsRetentionPolicy::js_InterestPolicy : Remove a message if all interested observers have received it
• jsRetentionPolicy::js_LimitsPolicy : Keep messages until a limit (e.g. MaxMsg, MaxByte or MaxAge) has been reached.
• jsRetentionPolicy::js_WorkQueuePolicy : delete message if first worker/consumer on a queue received it

• The default setting is jsRetentionPolicy:: ????? */ jsRetentionPolicy Retention;

  /* A positive numeric value indicating the maximum consumers that may be associated with this specific stream. If exceeded 
     the library will refuse the addition with #js_AddConsumer or tools such as the NATS CLI. The default value is ?????
  */
  int64_t                 MaxConsumers;
  
  /* A positive numeric value indicating what the maximum number of pending messages is that may be awaiting in the 

• queue for this specific stream. Exceeding this this number will lead to message processing according to set policies,

• rather then processing it the normal way. The default value is ????? */ int64_t MaxMsgs;

  /* A positive numeric value that is an additional restriction on waiting messages by specifying it as a volume of bytes. 

• NATS is message agnostic and messages do not have fixed sizes, nor fixed maximum sizes, and hence can largely differ
• in the memory they occupy. Therefore setting limits for messages waiting in the queue of this stream to be processed

• just be number of messages may be insufficient. The default value is ????? */ int64_t MaxBytes;

  /* A positive numeric variable, or zero, to indicate that messages must be deleted from the queue waiting for processing 

• if they are older then a certain age. The age is set in nanoseconds and compared to the current time at the moment of

• evaluation. The default value is ????? */ int64_t MaxAge;

  /* A positive numeric value that allows to set the maximum number of messages that a single subject, as defined in the 

• Subjects array member, may have waiting for processing. This parameter must be carefully set because it applies to ALL

• subjects for this specific stream. The default value is ????? */ int64_t MaxMsgsPerSubject;

  /* A positive message setting the maximum allowed sized of a single message. NATS is message agnostic and hence the body of

• a message is a pointer and a size. Besides that NATS keeps metadata on a per message basis, such as headers. This parameter

• refers to the complete size of the message in bytes. The default is ????? NOTE: Verify & confirm if this refers to the complete msg size and not just the msg body size? */ int32_t MaxMsgSize;

  /* A variable holding one of the values as defined by jsDiscardPolicy. 

• jsDiscardPolicy::js_DiscardNew will throw away the NEWEST message if limits are reached to store a new on.
• jsDiscardPolicy::js_DiscardOld will throw away the OLDEST message if limits are reached to store a new one.
• Discarding per subject requires KeyValue alike semantics to discard new messages on a subject per subject basis.

• See also the DiscardNewPerSubject member of this configuration. The default is ????? */ jsDiscardPolicy Discard;

  jsStorageType           Storage;
  int64_t                 Replicas;
  
  /* Boolean variable that allows to suppress acknowledgement. By default when a message is publish on a stream, it is acknowledge 

• by a callback, if one was provided with the used asynchronous publish function or a publish function waits for acknowledgement
• before returning if it is synchronous. Turning this default behavior off by setting this variable to true might increase
• performance at the risk of not knowing if messages reached persistency. Application that do not need to check messages from an
• application level list or republish in the callback, might turn this on. Such application might want to explore if they need
• persistency and may be better of with NATS core.

• The default is false. */ bool NoAck;

  const char              *Template;
  int64_t                 Duplicates;
  jsPlacement             *Placement;
  jsStreamSource          *Mirror;
  jsStreamSource          **Sources;
  int                     SourcesLen;
  
  /* A Boolean variable allowing to seal this specific stream so that no messages can get in or out anymore.

• The messages that are waiting for processing are thereby frozen in place if this is set to true.

• The default is false. */ bool Sealed;

  /* A Boolean variable allowing to deny the deletion of messages from this specific stream to functions that

• can do that, for instance by using the Message Sequence Number. The default is false.
• TODO: It is not clear if this also apply to automatic deletion as part of automatic purging based

• on purging parameters that apply to the stream. */ bool DenyDelete;

  /* A Boolean variable allowing to deny the purging of messages from this specific stream to functions that

• for instance by applying filters on message properties such as sequence number, age, etc. The default is false.
• TODO: It is not clear if this also apply to automatic purging as part of automatic purging based

• on purging parameters that apply to the stream, mostly after a new connection. */ bool DenyPurge;

  /* A Boolean variable allowing to deny roll-up operations. Roll-up operations uses 'special' formatted message 

• headers (that are in the metadata of a message) and that are used to select other messages and purge them.
• These other messages that can be affected and purge are only those that are older then the message that is
• used for the roll-up. Hence it does not affect messages that arrived later then the moment that the roll-up
• operation was started. The default is ?????

•  */
  bool                    AllowRollup;
  
  /* A jsRePublish structure to allows source subjects to be mapped to a destination subject for republishing.

• jsRePublish::Destination : char *
• jsRePublish::Source : char *
• jsRePublish::HeadersOnly : bool, allows only to republish the header, not the body. If receiver is interested
• in the body, the message number can be used to retrieve the message body if no
• other policies prevent it.
• Normally, after a message has been sequenced and stored it is considered persistent and republishing, if applicable,
• would already have taken place. This parameter, when set to true, bypassed that by authorizing republishing
• even if messages are sequenced and stored.

• The default is ????? / jsRePublish RePublish;

  /* This Boolean variable allows direct access to individual messages which might increase performance.

• As an example, NATS imbeds a NoSQL implementation of type Key-Value (the type as conceptually described
• by Amazon’s Dynamo paper) in the form of KeyValue Stores, also called buckets. With this parameter set

• to true one can access a single KeyValue rather then a complete Bucket. */ bool AllowDirect;

  // Allow higher performance and unified direct access for mirrors as well.
  bool                    MirrorDirect;
  
  /* A Boolean variable that allows to fine tune the Discard member settings. If set to true the behavior selected 

• in Discard remains the same but is applied on a per subject basis rather than on a per stream basis.
• Discarding message per subject requires KeyValue alike semantics (see also AllowRollup). */ bool DiscardNewPerSubject;

} jsStreamConfig;