dogmatiq / dogma

A specification and API for building message-based applications in Go.
MIT License
32 stars 3 forks source link

Specify allowed lengths for all variable-length data types. #142

Open jmalloc opened 3 years ago

jmalloc commented 3 years ago

This goal of this change is to document reasonable limits to variable-length values passed around via Dogma interfaces to set some expectations for application implementors as to what is reasonable, and some minimum requirements for engines.

This arose out of a discussion about jmalloc/ax which uses a MessageDescription() method similarly to dogma.DescribableMessage. We had an issue where a long message description caused some message to be retried indefinitely because it produced another message with a description that overflowed the database schema's maximum column length (descriptions were persisted to aid humans browsing the event stream).

Off the top of my head I can think of the following variable-length values that would need to be addresses (all strings)...

jmalloc commented 3 years ago

We need to come up with lengths that we're comfortable adding to the spec. Just to get the ball rolling ...

message descriptions

I don't think there's any obvious length here, but given that they're intended to be written to a log we'd generally want them to fit in a "single line", however long that may be. Perhaps we could suggest that they SHOULD be a maximum of 120 characters. We'll probably truncate them when storing in the database, but I'm not sure if that's really relevant to the spec at all. We could say that the engine MAY truncate or compact the messages for display purposes.

identity keys

We already suggest that the keys be UUIDs, and I can't really think of a strong use case for using anything other than a UUID, so we could even consider mandating that keys MUST be a UUID. Short of that I'd probably make the limit here the same as for aggregate/process instance IDs just for consistency/ease.

identity names

These should probably be treated a bit like type names in an arbitrary language - long enough to be descriptive, but not so long as to become unwieldy. We might want to suggest that they SHOULD be <50 characters or so, but require engines to support more. In practice these could be truncated and not impact engine operation (since it's the key that is definitive) but I am disinclined to allow for such truncation since they are still an identifier.

aggregate & process instance IDs

We definitely want to at least allow for UUIDs, but also have the "compound" keys that we construct from multiple other UUIDs, so we want to allow for fairly long identifiers, however, the longer we allow the bigger and "slower" database indexes become in engines like Verity. I'm inclined to cap this one at something like 128 bytes - plenty long enough for a few UUIDs, etc, chained together, but still shorter than an average SQL database's VARCHAR column width.

danilvpetrov commented 3 years ago

I don't see any reasons not to specify those limits. Should those limits should be presented as recommendations for engine implementations or as hard requirements?

koden-km commented 3 years ago

Your suggested limits seem fine. I like the idea of recommended limits without them being hard limits. That allows users to still do what they need but guides them to the intended usage.

jmalloc commented 3 years ago

I think we probably want to have a hard minimum that engines should support, and then recommended maximums for application developers.

jmalloc commented 1 year ago