palaciosjeremias
commented
2 years ago The following draft tasks were identified to perform the Epic:
-
Research.
- Integrity of the 4 values id, name, register_ip, internal_key (Finished)
- Investigate the key_polling module
-
Synchronization design
- Cluster.
- It's asynchronous, so an event-driven design doesn't apply.
- It must notify of eliminations and insertion of agents.
- Must co-exist with Remoted's synchronization mechanism.
- Remoted
- It must notify the removal and insertion of agents.
- It must avoid the regeneration of the entire key-store.
- Must coexist with the cluster synchronization mechanism.
- Proposal:
- A table of notifiers.
- A status column.
- When an agent is inserted, it is added, it is written with status NEW and it is written in the notifiers table who should know (cluster and remoted)
- When an agent is deleted, it is marked as DELETED and it is written in the notifiers table who should know about it (cluster and remoted).
- Both Remoted will be able to obtain agents by NEW and DELETED (I would make a single command) (keys_update), when the notifiers table is empty, the NEW are passed to SYNCED and the DELETED are deleted.
- In a start, WDB should NOT change the status of NEW and DELETED (cluster needs it)
- In addition, there must be a global_hash of the agent keys (it could be on the internal_key) so that the framework asks the integrity of the databases and resynchronizes everything if necessary. (cached)
- PROBLEM: For Core it is the same if we are master or if we are a single node manager. We should be able to differentiate this if we want to make a consistent list of providers.
- Graphics
- Review with Framework
- Review of unifying the agent and group synchronization commands.
- The groups command should not return agent groups in NEW or DELETED state (only for cluster, not remoted)
- Cluster.
-
Performance testing
- Perform baseline performance test.
- Insertion of 1k 10k and 100k simultaneous agents. (compare with pre-groups branch)
- Remoted in-memory keystore rearming with 100k agents
- Do performance test post changes.
- Perform baseline performance test.
-
wazuh_db commands:
- wdb_update_agent_name is only used for manager, but it allows to modify the name of an agent, should it be restricted?
- Deprecate WDB_STMT_GLOBAL_INSERT_AGENT for the new set_agents which, like groups, has to work with multiple agents
- Synchronization commands
- insert_agent with status logic
- update_agent_keys - Similar to sync_agent_groups_get (should return what is NEW and what is DELETED), update the states and (optionally) return the global_hash of the keys.
- integrity_check - receives the global_hash and compares it with yours
- get_global_hash (cached) (you must consider that the query is ordered by ID)
- get_keys (by id, by ip)
- Note: Consider that some of these actions usually require queries and auxiliary methods.
- get_agent_id (counterface of WDB_STMT_GLOBAL_SELECT_AGENT_NAME for monitord) (yes, I took it from the file...)
-
authd:
- Replace the initial reading of client.keys by the WDB query. (it is a common method to Remoted) (rename it too)
- Complete elimination of the thread writer.
- Make insertion in the DB as soon as we insert the agent (it is already in the groups branch)
- Delete as soon as we process the deletion.
-
Remoted:
- Add the logic so that in case of not having an agent in the keystor, it asks WDB
- Adapt the use of the get_keys command to initially load the key_store (already done in the Authd task)
- Adapt the periodic update that we do, by the update_keys command to wdb.
-
Tools:
- manage agents should deprecate the direct handling of client.keys. (The write almost everything is marked by the authd_running flag)
- manage agents must adapt the import mechanism k_import and k_bulkload read directly from the client.keys file.
- There are more places to adapt, NameExist() becomes obsolete. Etc.
-
wm_database:
- Deprecate periodic agent synchronization.
- Remove client.keys from inotify (this only leaves shared_groups in inotify, could imply refactoring/simplifying code)
- Implement a mechanism similar to groups, during start:
- In master: put client.keys in the DB and delete it after successful completion
- In worker: delete it directly (?)
-
Packages:
- The files that are deployed
-
Additional features:
- Recognize when a manager is single node or master. (method like is_worker that reads the cluster config)
- Manage_agents refactor (has tool methods mixed with those used by Authd)
- make it clustered
- Deprecate deprecated methods that handle client.keys
-
QA plans:
- Assembly of tests
- Execution of exploratory testing
After the requirements of https://github.com/wazuh/wazuh/issues/83, a complete migration of client.keys to Wazuh-db scope is required. For this migration, the following aspects must be considered: