snowplow / snowplow-rdb-loader

Stores Snowplow enriched events in Redshift, Snowflake and Databricks
Other
31 stars 17 forks source link

Lower sensitivity of cats-effect responsiveness warning #1309

Closed istreeter closed 1 year ago

istreeter commented 1 year ago

Since upgrading to cats-effect 3 we get lots of warnings like:

2023-08-07T19:27:47.824Z [WARNING] Your app's responsiveness to a new asynchronous 
event (such as a new connection, an upstream response, or a timer) was in excess
of 100 milliseconds. Your CPU is probably starving. Consider increasing the 
granularity of your delays or adding more cedes. This may also be a sign that you
are unintentionally running blocking I/O operations (such as File or InetAddress)
without the blocking combinator.

The warning message is explained in detail in this documentation. Given the cpu requirements of these streaming applications, I believe it is safe and reasonable to decrease the sensitivity. The recommended way to do this is by increasing the value of cpuStarvationCheckInterval in the cats-effect runtime config.