unnecessary system load generated by the persistent redis instance

[alexskr]

currently, we use persistent redis instances to hold the following:

• annotator cache
• dictionary rebuild time stamp (needed for triggering mgrep restarts)
• ncbo_cron job queue
• google analytics data
• appliance uuid

Currently, redis is configured to persist data to disk with save 300 10 which means save to disk every 5 minutes if 10 keys were changed which is what we are observing. Persisting the annotator cache every 5 minutes even when no new ontologies are processed is less than ideal because it creates unnecessary high CPU/DISK/RAM usage.

My guess is that ncbo_cron job scheduler uses a locking mechanism for checking job queue, resulting in at least 10 writes / 5 minutes.

[syphax-bouazzouni]

My guess is that ncbo_cron job scheduler uses a locking mechanism for checking job queue, resulting in at least 10 writes / 5 minutes.

Maybe the cause is this job scheduled each 5minutes

This job calls the function queued_items which does the calls to Redis.

[alexskr]

yes, that is exactly what was the issue. We deployed 4th instance of redis and dedicated it to ncbo_cron.config.redis_host and LinkedData.config.ontology_analytics_redis_host which alleviated this issue.
I don't think this would be a significant issue for ontoportals with small number of ontologies. @syphax-bouazzouni are you experiencing this issue in your environment?

[syphax-bouazzouni]

@syphax-bouazzouni are you experiencing this issue in your environment?

No, not really just pass over this issue and became intrigued by it.