search

haproxytech / dataplaneapi

HAProxy Data Plane API
https://www.haproxy.com/documentation/dataplaneapi/
Apache License 2.0
329 stars 76 forks source link

--update-map-files whitelist/blacklist to prevent very large or unwanted maps from syncing #300

Open fatchan opened 1 year ago

fatchan commented 1 year ago

Hi, I recently encountered an issue related to the update-map-files option.

I have been using the update-map-files to keep several small (<1MB) map files synced to disk. This works well. However, I recently added a much larger map file of ~30MB to HAProxy. Now, the dataplaneapi memory, cpu and disk utilization increased significantly because it frequently tries to:

  1. read the entire map file
  2. check for any changes
  3. sort it
  4. write to disk

I don't even need or want this map file to be synced, and it is never changed during runtime. So this is a huge waste of resources for me.

I have "solved" the issue in my case by patching dataplaneapi with a simple if statement to exclude the name of the large map file and to not call Sync() on it. But obviously, this is very inconvenient, and not at all flexible.

To fix this in a better way, I propose an option to blacklist or whitelist map files.

Thoughts?

Kind regards.

mjuraga commented 1 year ago

Hi @fatchan this is a nice use-case, and we'll add it to our roadmap for some future releases.