flux-framework / flux-core

core services for the Flux resource management framework
GNU Lesser General Public License v3.0
168 stars 50 forks source link

kvs: ensure large values exceeding maximum rejected consistently #6265

Open chu11 opened 2 months ago

chu11 commented 2 months ago

Following up #6264, it is not clear that the KVS ensures that all data is below the maximum allowed and that errors propogate correctly / consistently. For example:

int flux_kvs_txn_put (flux_kvs_txn_t *txn,
                      int flags,
                      const char *key,
                      const char *value);

internally flux_kvs_txn_put() calls strlen (value) which can return size_t which can be larger than int. It's not super clear what the fallout of such calls could be. That value is passed to treeobj_create_val() (takes an int), which then calls base64_encoded_length () (takes a size_t). I'm not sure what error would result here.

In

int flux_kvs_txn_pack (flux_kvs_txn_t *txn,
                       int flags,
                       const char *key,
                       const char *fmt,
                       ...);

It appears internally the resulting string after json_pack() is called is not checked for length either before being passed to treeobj_create_val().

Also need to audit that large values sent directly to the KVS module outside of the API are rejected accordingly.

garlick commented 2 months ago

Do we have a maximum value size other than the maximum content blob size?

chu11 commented 2 months ago

ahhh I had forgotten about the blob size limit.

AFAICT there is no KVS size limit, it needs to be defined, see #6266.