search

puppetlabs / puppet-enterprise_issues

This repository will allow public community members to file bug reports against Puppet Enterprise
1 stars 0 forks source link

Classifier API: Make background class update configureable #56

Open bastelfreak opened 1 week ago

bastelfreak commented 1 week ago

Use Case

By default, the PE Classifier asks the puppetserver for a list of environments + list of classes periodically in the background. There's also an API call to explicitly trigger an update: https://www.puppet.com/docs/pe/2023.8/update_classes_endpoint

When people use the API call, it is not required to run the background job as well. So I think it would be nice to disable it.

Describe the Solution You Would Like

Make background class update configureable / add opt-out option

Describe Alternatives You've Considered

A clear and concise description of any alternative solutions or features you've considered.

Additional Context

I would have tried to provide a patch instead, but the git repository is private and not accessible by partners.

steveax commented 1 day ago

In the PE Console Infrastructure group, set puppet_enterprise::profile::console::classifier_synchronization_period. If set to a negative number, it should disable the sync completely.

bastelfreak commented 1 day ago

so the default for classifier_synchronization_period is 600. which produces in /var/log/puppetlabs/console-services/console-services.log :

2024-10-04T09:23:41.835Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-8] INFO  [p.c.class-updater] Starting background class update
2024-10-04T09:23:41.863Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-8] INFO  [p.c.class-updater] Requesting environment list from "https://pe:8140/puppet/v3/environments".
2024-10-04T09:23:41.932Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-8] INFO  [p.c.class-updater] 200 response received for request for environments from "https://pe:8140/puppet/v3/environments"
2024-10-04T09:23:41.932Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-8] INFO  [p.c.class-updater] Requesting classes in production from https://pe:8140/puppet/v3/environment_classes?environment=production with etag 69c93a817958ae696f6e5bb6da4391dfb60ad1aafa1f186bb93e68b14afa3cdd
2024-10-04T09:23:45.022Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-8] INFO  [p.c.class-updater] 200 response received for request for classes in production.
2024-10-04T09:23:45.026Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-8] INFO  [p.c.class-updater] Requesting classes in peadm from https://pe:8140/puppet/v3/environment_classes?environment=peadm with etag ea45ff3c4966c568276595db9f6bfea2902bf4fec4a8cf57eae48b343c01ffab
2024-10-04T09:23:46.889Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-8] INFO  [p.c.class-updater] 304 response received for request for classes in peadm.
2024-10-04T09:23:46.890Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-8] INFO  [p.c.s.postgres] Beginning classes synchronization, locking tables
2024-10-04T09:23:46.891Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-8] INFO  [p.c.s.postgres] Found environments #{"development"} to delete
2024-10-04T09:23:47.025Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-8] INFO  [p.c.s.postgres] Found environment to delete, but it is empty so not deleting development
2024-10-04T09:23:47.026Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-8] INFO  [p.c.s.postgres] Completed classes synchronization, unlocking tables
2024-10-04T09:23:47.027Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-8] INFO  [p.c.class-updater] Synchronized 2 classes from the Puppet server in 5 seconds
2024-10-04T09:25:18.274Z [clojure-agent-send-off-pool-75] INFO  [p.c.class-updater] Requesting classes in production from https://pe:8140/puppet/v3/environment_classes?environment=production
2024-10-04T09:25:19.856Z [clojure-agent-send-off-pool-75] INFO  [p.c.class-updater] 200 response received for request for classes in production.
2024-10-04T09:30:38.895Z [clojure-agent-send-off-pool-86] INFO  [p.c.class-updater] Requesting environment list from "https://pe:8140/puppet/v3/environments".
2024-10-04T09:30:38.934Z [clojure-agent-send-off-pool-86] INFO  [p.c.class-updater] 200 response received for request for environments from "https://pe:8140/puppet/v3/environments"
2024-10-04T09:33:47.027Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-7] INFO  [p.c.class-updater] Starting background class update
2024-10-04T09:33:47.030Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-7] INFO  [p.c.class-updater] Requesting environment list from "https://pe:8140/puppet/v3/environments".
2024-10-04T09:33:47.058Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-7] INFO  [p.c.class-updater] 200 response received for request for environments from "https://pe:8140/puppet/v3/environments"
2024-10-04T09:33:47.058Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-7] INFO  [p.c.class-updater] Requesting classes in production from https://pe:8140/puppet/v3/environment_classes?environment=production with etag 5175449fa63e1732c0e9466048ef780ac91c05bccc2fb727b28422a20057bef5
2024-10-04T09:33:47.070Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-7] INFO  [p.c.class-updater] 304 response received for request for classes in production.
2024-10-04T09:33:47.070Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-7] INFO  [p.c.class-updater] Requesting classes in peadm from https://pe:8140/puppet/v3/environment_classes?environment=peadm with etag ea45ff3c4966c568276595db9f6bfea2902bf4fec4a8cf57eae48b343c01ffab
2024-10-04T09:33:47.079Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-7] INFO  [p.c.class-updater] 304 response received for request for classes in peadm.
2024-10-04T09:33:47.079Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-7] INFO  [p.c.s.postgres] Beginning classes synchronization, locking tables
2024-10-04T09:33:47.081Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-7] INFO  [p.c.s.postgres] Found environments #{"development"} to delete
2024-10-04T09:33:47.105Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-7] INFO  [p.c.s.postgres] Found environment to delete, but it is empty so not deleting development
2024-10-04T09:33:47.106Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-7] INFO  [p.c.s.postgres] Completed classes synchronization, unlocking tables
2024-10-04T09:33:47.106Z [87d78341-dbb8-427b-b8b0-3e9b706abc84_Worker-7] INFO  [p.c.class-updater] Synchronized 2 classes from the Puppet server in 0 seconds

After setting this in Hiera:

---
puppet_enterprise::profile::console::classifier_synchronization_period: 0
2024-10-04T09:41:46.489Z [async-dispatch-6] INFO  [p.p.routes] Starting pe-console-app
2024-10-04T09:45:02.758Z [clojure-agent-send-off-pool-130] INFO  [p.c.class-updater] Requesting environment list from "https://pe:8140/puppet/v3/environments".
2024-10-04T09:45:02.823Z [clojure-agent-send-off-pool-130] INFO  [p.c.class-updater] 200 response received for request for environments from "https://pe:8140/puppet/v3/environments"
2024-10-04T09:45:02.824Z [clojure-agent-send-off-pool-130] INFO  [p.c.class-updater] Requesting classes in production from https://pe:8140/puppet/v3/environment_classes?environment=production
2024-10-04T09:45:04.104Z [clojure-agent-send-off-pool-130] INFO  [p.c.class-updater] 200 response received for request for classes in production.
2024-10-04T09:50:38.888Z [clojure-agent-send-off-pool-145] INFO  [p.c.class-updater] Requesting environment list from "https://pe:8140/puppet/v3/environments".
2024-10-04T09:50:38.936Z [clojure-agent-send-off-pool-145] INFO  [p.c.class-updater] 200 response received for request for environments from "https://pe:8140/puppet/v3/environments"

I'm on PE 2021.7.9 and can confirm that setting the classifier_synchronization_period to 0 disables the background class import. @steveax do you know why the environment import is running / is that redundant when I'm already calling https://$(puppet config print server):4433/classifier-api/v1/update-classes?