flux-framework / flux-core

core services for the Flux resource management framework
GNU Lesser General Public License v3.0
168 stars 50 forks source link

content: support RFC10 defined blob size limit #6457

Open chu11 opened 6 days ago

chu11 commented 6 days ago

in content/cache.c

/* Raise the max blob size value to 1GB so that large KVS values
 * (including KVS directories) can be supported while the KVS transitions
 * to the RFC 11 treeobj data representation.
 */
//static const uint32_t default_blob_size_limit = 1048576; /* RFC 10 */
static const uint32_t default_blob_size_limit = 1048576*1024;

this is mostly a reminder that we need to re-look at what would be necessary to make this work or if RFC10 should be updated with a larger limit. #1202 & #1206 are probably involved in the requirement to make it work.

as a side note, if the average sha has is 46 bytes in size, counting the quotes around it and the comma separating each one in a valref array, that's about a max of 21K blobref's in a valref array. We definitely got eventlogs with more than that.