Closed jesusgn90 closed 5 years ago
I am working on dev-new-agent-endpoint-426
branches (API and framework) for this development.
I created a new method into Agent
class for giving support to this new endpoint (I named it as GET agents/full_summary
temporally).
An example of this endpoint is the next:
# curl -u foo:bar "localhost:55000/agents/full_summary?pretty"
{
"error": 0,
"data": {
"unique_node_names": {
"items": [
{
"count": 2,
"node_name": "master"
},
{
"count": 1,
"node_name": "worker-1"
},
{
"count": 1,
"node_name": "worker-2"
}
],
"totalItems": 4
},
"groups": {
"items": [
{
"count": 3,
"name": "default",
"mergedSum": "ddda4e15b99efad1d3be7ae9d7ff14ff",
"configSum": "ab73af41699f13fdd81903b5f23d8d00"
},
{
"count": 0,
"name": "dmz",
"mergedSum": "dd77862c4a41ae1b3854d67143f3d3e4",
"configSum": "ab73af41699f13fdd81903b5f23d8d00"
},
{
"count": 0,
"name": "testsagentconf",
"mergedSum": "2acdb385658097abb9528aa5ec18c490",
"configSum": "297b4cea942e0b7d2d9c59f9433e3e97"
},
{
"count": 0,
"name": "testsagentconf2",
"mergedSum": "391ae29c1b0355c610f45bf133d5ea55",
"configSum": "297b4cea942e0b7d2d9c59f9433e3e97"
}
],
"totalItems": 4
},
"unique_agent_os": {
"items": [
{
"os": {
"name": "CentOS Linux",
"platform": "centos",
"version": "7.6"
},
"count": 3
},
{
"os": {
"name": "CentOS Linux",
"platform": "centos",
"version": "7"
},
"count": 1
}
],
"totalItems": 4
},
"summary": {
"Total": 4,
"Active": 4,
"Disconnected": 0,
"Never connected": 0,
"Pending": 0
},
"unique_agent_version": {
"items": [
{
"count": 1,
"version": "Wazuh v3.11.0"
},
{
"count": 2,
"version": "Wazuh v3.9.3"
},
{
"count": 1,
"version": "Wazuh v3.5.0"
}
],
"totalItems": 4
},
"last_registered_agent": {
"os": {
"arch": "x86_64",
"codename": "Core",
"major": "7",
"name": "CentOS Linux",
"platform": "centos",
"uname": "Linux |7e2159a46bff |5.2.5-200.fc30.x86_64 |#1 SMP Wed Jul 31 14:37:17 UTC 2019 |x86_64",
"version": "7"
},
"group": [
"default"
],
"node_name": "worker-2",
"manager": "636a34fa474a",
"configSum": "ab73af41699f13fdd81903b5f23d8d00",
"name": "7e2159a46bff",
"version": "Wazuh v3.5.0",
"registerIP": "172.30.0.7",
"id": "003",
"status": "Active",
"dateAdd": "2019-08-08 11:25:03",
"ip": "172.30.0.7",
"lastKeepAlive": "2019-08-08 11:59:02",
"mergedSum": "ddda4e15b99efad1d3be7ae9d7ff14ff"
}
}
}
mocha
and unit tests.I created a new controller (summary
) and I included in it this new call. This controller could be useful in the future to include aggregate information about cluster
, for example.
Below there is an example of the new endpoint:
# curl -u foo:bar localhost:55000/summary/agents?pretty
{
"error": 0,
"data": {
"nodes": {
"items": [
{
"count": 2,
"node_name": "master"
},
{
"count": 1,
"node_name": "worker-1"
},
{
"count": 1,
"node_name": "worker-2"
}
],
"totalItems": 4
},
"groups": {
"items": [
{
"count": 3,
"name": "default",
"mergedSum": "ddda4e15b99efad1d3be7ae9d7ff14ff",
"configSum": "ab73af41699f13fdd81903b5f23d8d00"
}
],
"totalItems": 1
},
"agent_os": {
"items": [
{
"os": {
"name": "CentOS Linux",
"platform": "centos",
"version": "7.6"
},
"count": 3
},
{
"os": {
"name": "CentOS Linux",
"platform": "centos",
"version": "7"
},
"count": 1
}
],
"totalItems": 4
},
"agent_status": {
"Total": 4,
"Active": 4,
"Disconnected": 0,
"Never connected": 0,
"Pending": 0
},
"agent_version": {
"items": [
{
"count": 1,
"version": "Wazuh v3.10.0"
},
{
"count": 2,
"version": "Wazuh v3.9.3"
},
{
"count": 1,
"version": "Wazuh v3.5.0"
}
],
"totalItems": 4
},
"last_registered_agent": {
"os": {
"arch": "x86_64",
"major": "7",
"minor": "6",
"name": "CentOS Linux",
"platform": "centos",
"uname": "Linux |ac993d3f801a |5.2.7-200.fc30.x86_64 |#1 SMP Thu Aug 8 05:35:29 UTC 2019 |x86_64",
"version": "7.6"
},
"registerIP": "172.25.0.6",
"node_name": "worker-1",
"version": "Wazuh v3.9.3",
"group": [
"default"
],
"dateAdd": "2019-08-14 10:56:58",
"manager": "ebb389fb8079",
"lastKeepAlive": "2019-08-14 11:12:20",
"configSum": "ab73af41699f13fdd81903b5f23d8d00",
"mergedSum": "ddda4e15b99efad1d3be7ae9d7ff14ff",
"id": "002",
"ip": "172.25.0.6",
"name": "ac993d3f801a",
"status": "Active"
}
}
}
# mocha test/test_summary.js
Summary
GET/summary/agents
✓ Request (358ms)
1 passing (364ms)
# mocha test/test_agents.js --timeout=10000
Agents
GET/agents
✓ Request (298ms)
✓ Pagination (320ms)
✓ Retrieve all elements with limit=0 (318ms)
✓ Sort (321ms)
✓ Wrong Sort (314ms)
✓ Search (322ms)
✓ Selector (315ms)
✓ Not allowed selector (323ms)
✓ Version (312ms)
✓ Os.platform (323ms)
✓ Os.version (314ms)
✓ ManagerHost (313ms)
✓ Filters: status (329ms)
✓ Filters: status 2 (321ms)
✓ Filters: Invalid filter
✓ Filters: Invalid filter - Extra field
✓ Filters: older_than (323ms)
✓ Filters: group (314ms)
✓ Select: single field (339ms)
✓ Select: multiple fields (313ms)
✓ Select: wrong field (319ms)
✓ Select: invalid character
✓ Filters: query (322ms)
GET/agents/summary
✓ Request (327ms)
GET/agents/summary/os
✓ Request (315ms)
GET/agents/outdated
✓ Request (304ms)
GET/agents/:agent_id
✓ Request (manager) (309ms)
✓ Request (agent) (335ms)
✓ Selector (329ms)
✓ Not allowed selector (395ms)
✓ Params: Bad agent id
✓ Errors: No agent (351ms)
✓ Select (361ms)
✓ Select: wrong field (349ms)
GET/agents/name/:agent_name
✓ Request (402ms)
✓ Wrong name (358ms)
✓ Selector (321ms)
✓ Not allowed selector (345ms)
GET/agents/:agent_id/key
✓ Request (358ms)
✓ Params: Bad agent id
✓ Errors: No key (376ms)
PUT/agents/groups/:group_id
✓ Request (365ms)
✓ Params: Bad group name
✓ Params: Group already exists (324ms)
PUT/agents/:agent_id/group/:group_id
✓ Request (303ms)
✓ Params: Bad agent name
✓ Params: Agent does not exist (329ms)
✓ Params: Replace parameter (319ms)
POST/agents/groups/:group_id/files/:file_name
✓ Request (360ms)
✓ ErrorOnBadGroup (337ms)
✓ ErrorOnEmptyConf
✓ OnlyAgentConfAllowed (347ms)
✓ InvalidConfDetected
✓ WrongConfDetected (335ms)
✓ TooBigXML
GET/agents/no_group
✓ Request (439ms)
✓ Pagination (350ms)
✓ Retrieve all elements with limit=0 (408ms)
✓ Sort (364ms)
✓ Search (346ms)
✓ Select (379ms)
✓ Wrong select (345ms)
✓ Filter: status (356ms)
GET/agents/groups
✓ Request (337ms)
✓ Retrieve all elements with limit=0 (335ms)
✓ Hash algorithm (319ms)
✓ Wrong Hash algorithm (364ms)
GET/agents/groups/:group_id
✓ Request (350ms)
✓ Params: Bad group name
✓ Retrieve all elements with limit=0 (323ms)
✓ Select (397ms)
✓ Filter: status (376ms)
GET/agents/groups/:group_id/configuration
✓ Request (328ms)
✓ Params: Bad group name
✓ Retrieve all elements with limit=0 (302ms)
GET/agents/groups/:group_id/files
✓ Request (361ms)
✓ Params: Bad group name
✓ Retrieve all elements with limit=0 (365ms)
✓ Hash algorithm (344ms)
✓ Wrong Hash algorithm (393ms)
GET/agents/groups/:group_id/files/:filename
✓ Request (404ms)
✓ UsingFormatAgentConfXML (397ms)
✓ UsingFormatAgentConfJSON (349ms)
✓ UsingFormatRootcheckXML (325ms)
✓ UsingFormatRootcheckJSON (363ms)
✓ Params: Bad group name
POST/agents/groups/:group_id/configuration
✓ Request (323ms)
✓ ErrorOnBadGroup (351ms)
✓ ErrorOnEmptyConf
✓ InvalidConfDetected
✓ WrongConfDetected (377ms)
✓ TooBigXML
DELETE/agents/:agent_id/group
✓ Request (339ms)
✓ Errors: ID is not present (341ms)
✓ Params: Bad agent id
DELETE/agents/:agent_id/group/:group_id
✓ Request (298ms)
✓ Errors: ID is not present (328ms)
✓ Errors: Group is not present (328ms)
✓ Params: Bad agent id
✓ Params: Bad group id (312ms)
DELETE/agents/groups/:group_id
✓ Request (341ms)
✓ Params: Bad group id
DELETE/agents
✓ Request
✓ Filter: older_than, status and ids (3907ms)
✓ Errors: Get deleted agent (378ms)
✓ Filter: older_than (355ms)
GET/agents/stats/distinct
✓ Request (316ms)
✓ Pagination (320ms)
✓ Retrieve all elements with limit=0 (341ms)
✓ Sort (361ms)
✓ Search (383ms)
✓ Select (350ms)
✓ Wrong select (371ms)
GET/agents/:agent/config/:component/:configuration
✓ Request-Agent-Client (383ms)
✓ Request-Agent-Buffer (326ms)
✓ Request-Agent-Labels (341ms)
✓ Request-Agent-Internal (340ms)
✓ Request-Agentless-Agentless (332ms)
✓ Request-Analysis-Global (343ms)
✓ Request-Analysis-Active-response (332ms)
✓ Request-Analysis-Alerts (341ms)
✓ Request-Analysis-Command (328ms)
✓ Request-Analysis-Internal (338ms)
✓ Request-Auth-Auth (319ms)
✓ Request-Com-Active-response (347ms)
✓ Request-Com-Internal (330ms)
✓ Request-Csyslog-Csyslog (331ms)
✓ Request-Integrator-Integration (347ms)
✓ Request-Logcollector-Localfile (328ms)
✓ Request-Logcollector-Socket (348ms)
✓ Request-Logcollector-Internal (392ms)
✓ Request-Mail-Global (364ms)
✓ Request-Mail-Alerts (341ms)
✓ Request-Mail-Internal (333ms)
✓ Request-Monitor-Internal (341ms)
✓ Request-Request-Remote (409ms)
✓ Request-Request-Internal (385ms)
✓ Request-Syscheck-Syscheck (390ms)
✓ Request-Syscheck-Rootcheck (467ms)
✓ Request-Syscheck-Internal (387ms)
✓ Request-Wmodules-Wmodules (410ms)
PUT/agents/restart
✓ Request (456ms)
PUT/agents/:agent_id/restart
✓ Request (376ms)
✓ Params: Bad agent id
✓ Request (348ms)
POST/agents/restart
✓ Request (348ms)
✓ Params: A good id and a bad one (405ms)
✓ Params: Bad agent id
✓ Request (449ms)
149 passing (52s)
% pytest test_agent.py -vv
======================================================================================== test session starts ========================================================================================
platform linux -- Python 3.7.4, pytest-4.3.0, py-1.8.0, pluggy-0.9.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /home/druizz/Git/wazuh/framework, inifile:
plugins: tavern-0.26.3
collected 52 items
test_agent.py::test_get_agents_overview_default PASSED [ 1%]
test_agent.py::test_get_agents_overview_select[select0-all-None-0] PASSED [ 3%]
test_agent.py::test_get_agents_overview_select[select1-all-None-1] PASSED [ 5%]
test_agent.py::test_get_agents_overview_select[select2-all-None-1] PASSED [ 7%]
test_agent.py::test_get_agents_overview_select[select3-Active,Pending-None-0] PASSED [ 9%]
test_agent.py::test_get_agents_overview_select[select4-Disconnected-None-1] PASSED [ 11%]
test_agent.py::test_get_agents_overview_select[select5-Disconnected-1s-1] PASSED [ 13%]
test_agent.py::test_get_agents_overview_select[select6-Disconnected-2h-0] PASSED [ 15%]
test_agent.py::test_get_agents_overview_select[select7-all-15m-2] PASSED [ 17%]
test_agent.py::test_get_agents_overview_select[select8-Active-15m-0] PASSED [ 19%]
test_agent.py::test_get_agents_overview_select[select9-Active,Pending-15m-1] PASSED [ 21%]
test_agent.py::test_get_agents_overview_select[select10-status10-15m-1] PASSED [ 23%]
test_agent.py::test_get_agents_overview_query[ip=172.17.0.201] PASSED [ 25%]
test_agent.py::test_get_agents_overview_query[ip=172.17.0.202] PASSED [ 26%]
test_agent.py::test_get_agents_overview_query[ip=172.17.0.202;registerIP=any] PASSED [ 28%]
test_agent.py::test_get_agents_overview_query[status=Disconnected;lastKeepAlive>34m] PASSED [ 30%]
test_agent.py::test_get_agents_overview_query[(status=Active,status=Pending);lastKeepAlive>5m] PASSED [ 32%]
test_agent.py::test_get_agents_overview_search[search0-3] PASSED [ 34%]
test_agent.py::test_get_agents_overview_search[search1-3] PASSED [ 36%]
test_agent.py::test_get_agents_overview_search[search2-1] PASSED [ 38%]
test_agent.py::test_get_agents_overview_search[search3-5] PASSED [ 40%]
test_agent.py::test_get_agents_overview_search[search4-2] PASSED [ 42%]
test_agent.py::test_get_agents_overview_status_olderthan[active-9m-1-None] PASSED [ 44%]
test_agent.py::test_get_agents_overview_status_olderthan[all-1s-5-None] PASSED [ 46%]
test_agent.py::test_get_agents_overview_status_olderthan[pending,neverconnected-30m-1-None] PASSED [ 48%]
test_agent.py::test_get_agents_overview_status_olderthan[55-30m-0-1729] PASSED [ 50%]
test_agent.py::test_get_config_error[100-logcollector-internal-1701] PASSED [ 51%]
test_agent.py::test_get_config_error[005-logcollector-internal-1740] PASSED [ 53%]
test_agent.py::test_get_config_error[002-logcollector-internal-1735] PASSED [ 55%]
test_agent.py::test_get_config_error[000-None-None-1307] PASSED [ 57%]
test_agent.py::test_get_config_error[000-random-random-1101] PASSED [ 59%]
test_agent.py::test_get_config_error[000-analysis-internal-1117] PASSED [ 61%]
test_agent.py::test_get_config_error[000-analysis-internal-1118] PASSED [ 63%]
test_agent.py::test_get_config_error[000-analysis-random-1116] PASSED [ 65%]
test_agent.py::test_get_config_error[000-analysis-internal-None] PASSED [ 67%]
test_agent.py::test_remove_manual[False] PASSED [ 69%]
test_agent.py::test_remove_manual[True] PASSED [ 71%]
test_agent.py::test_remove_manual_error[001-1746] PASSED [ 73%]
test_agent.py::test_remove_manual_error[100-1701] PASSED [ 75%]
test_agent.py::test_remove_manual_error[001-1600] PASSED [ 76%]
test_agent.py::test_remove_manual_error[001-1748] PASSED [ 78%]
test_agent.py::test_remove_manual_error[001-1747] PASSED [ 80%]
test_agent.py::test_get_available_versions[001] PASSED [ 82%]
test_agent.py::test_get_available_versions[002] PASSED [ 84%]
test_agent.py::test_upgrade[001] PASSED [ 86%]
test_agent.py::test_upgrade[002] PASSED [ 88%]
test_agent.py::test_get_wpk_file[001] PASSED [ 90%]
test_agent.py::test_get_wpk_file[002] PASSED [ 92%]
test_agent.py::test_send_wpk_file[001] PASSED [ 94%]
test_agent.py::test_send_wpk_file[002] PASSED [ 96%]
test_agent.py::test_get_outdated_agents PASSED [ 98%]
test_agent.py::test_get_full_summary PASSED [100%]
========================================================================================= warnings summary ==========================================================================================
/home/druizz/Git/wazuh/framework/wazuh/InputValidator.py:23
/home/druizz/Git/wazuh/framework/wazuh/InputValidator.py:23: DeprecationWarning: invalid escape sequence \w
"""
-- Docs: https://docs.pytest.org/en/latest/warnings.html
=============================================================================== 52 passed, 1 warnings in 1.59 seconds ===============================================================================
Solved in #429.
Hi team, from the @wazuh/frontend we are doing some API calls and some logic that might be done directly from the Wazuh API.
Current behavior
Here is what we need and so far we are obtaining it through 6 API calls:
Unique node names found in all agents
GET /agents/stats/distinct + { fields: 'node_name', select: 'node_name' }
Groups list
GET /agents/groups
Unique OS distributions found in all agents
GET /agents/stats/distinct + { fields: 'os.name,os.platform,os.version', select: 'os.name,os.platform,os.version' }
Unique Wazuh agent version found in all agents
GET /agents/stats/distinct + { fields: 'version', select: 'version' }
Unique Wazuh agent status found in all agents
GET /agents/summary
Last registered agent
GET /agents + { limit: 1, sort: '-dateAdd', q: 'id!=000' }
Hi team, from the @wazuh/frontend we are doing some API calls and some logic that might be done directly from the Wazuh API.
Desired behavior
Just one API call for all the information.
GET /newendpoint/for/agents
With this improvement, the Wazuh app won't make 6 API calls for just the agents preview page, instead, it would make just one API call.