Open adambaumeister opened 7 months ago
This issue seems to've started in 10.2 NGFW code
PANOS XML API literally returns multiple records for the same job ids after 10.2, which I believe is the root cause for this. See below response I tested on 10.2.9-h1.
<response status="success">
<result>
<job>
<tenq>2024/07/08 04:57:51</tenq>
<tdeq>04:57:51</tdeq>
<id>4</id>
<user/>
<type>Downld</type>
<status>FIN</status>
<queued>NO</queued>
<stoppable>no</stoppable>
<result>OK</result>
<tfin>2024/07/08 04:58:03</tfin>
<description/>
<positionInQ>0</positionInQ>
<progress>2024/07/08 04:58:03</progress>
<details>
<line>Successfully downloaded</line>
<line>Transferring a copy of image to HA peer</line>
<line>Preloading into software manager</line>
<line>Successfully loaded into software manager</line>
</details>
<warnings/>
</job>
<job>
<tenq>2024/07/03 01:02:03</tenq>
<tdeq>01:02:03</tdeq>
<id>3</id>
<user/>
<type>Downld</type>
<status>FIN</status>
<queued>NO</queued>
<stoppable>no</stoppable>
<result>OK</result>
<tfin>2024/07/03 01:02:05</tfin>
<description/>
<positionInQ>0</positionInQ>
<progress>2024/07/03 01:02:05</progress>
<details>
<line>File successfully downloaded </line>
</details>
<warnings/>
</job>
<job>
<tenq>2024/07/01 06:39:16</tenq>
<tdeq>06:39:29</tdeq>
<id>2</id>
<user/>
<type>AutoCom</type>
<status>FIN</status>
<queued>NO</queued>
<stoppable>no</stoppable>
<result>OK</result>
<tfin>2024/07/01 06:40:26</tfin>
<description/>
<positionInQ>0</positionInQ>
<progress>100</progress>
<details>
<line>Configuration committed successfully</line>
<line>Successfully committed last configuration</line>
</details>
<warnings/>
</job>
<job>
<tenq>2024/07/08 04:57:51</tenq>
<tdeq>04:57:51</tdeq>
<id>4</id>
<user/>
<type>Downld</type>
<status>FIN</status>
<queued>NO</queued>
<stoppable>no</stoppable>
<result>OK</result>
<tfin>2024/07/08 04:58:03</tfin>
<description/>
<positionInQ>0</positionInQ>
<progress>2024/07/08 04:58:03</progress>
<details>
<line>Successfully downloaded</line>
<line>Transferring a copy of image to HA peer</line>
<line>Preloading into software manager</line>
<line>Successfully loaded into software manager</line>
</details>
<warnings/>
</job>
<job>
<tenq>2024/07/03 01:02:03</tenq>
<tdeq>01:02:03</tdeq>
<id>3</id>
<user/>
<type>Downld</type>
<status>FIN</status>
<queued>NO</queued>
<stoppable>no</stoppable>
<result>OK</result>
<tfin>2024/07/03 01:02:05</tfin>
<description/>
<positionInQ>0</positionInQ>
<progress>2024/07/03 01:02:05</progress>
<details>
<line>File successfully downloaded </line>
</details>
<warnings/>
</job>
</result>
</response>
I did notice that the PAN job API was returning duplicate entries in the 10.2/11 codebase, but not before. And that seemed like the root cause. But, there's also something else going on in XSOAR.
The DT for the poller is - PANOS.JobStatus(val.status!=="FIN").id
. So even with duplicate job entries returned from PANOS, I would expect that DT should still give 0 results once the job is completed.
What I'm seeing in the context is more of a history, instead of just the last run.
For example, the last upgrade I did, the command result show the 2 "FIN" entries in the war room...
hostid | id | positionInQ | progress | result | status | stoppable | tenq | tfin | type |
---|---|---|---|---|---|---|---|---|---|
021201010640 | 37 | 0 | 2024/06/17 16:22:25 | OK | FIN | no | 2024/06/17 16:16:09 | 2024/06/17 16:22:25 | SWInstall |
021201010640 | 37 | 0 | 2024/06/17 16:22:25 | OK | FIN | no | 2024/06/17 16:16:09 | 2024/06/17 16:22:25 | SWInstall |
But the context shows 4 entries with different percentages....
{
"JobStatus": [
{
"description": null,
"hostid": "021201010640",
"id": 37,
"positionInQ": "0",
"progress": "0",
"result": "PEND",
"status": "ACT",
"stoppable": "no",
"tenq": "2024/06/17 16:16:09",
"tfin": null,
"type": "SWInstall",
"user": null
},
{
"description": null,
"hostid": "021201010640",
"id": 37,
"positionInQ": "0",
"progress": "37",
"result": "PEND",
"status": "ACT",
"stoppable": "no",
"tenq": "2024/06/17 16:16:09",
"tfin": null,
"type": "SWInstall",
"user": null
},
{
"description": null,
"hostid": "021201010640",
"id": 37,
"positionInQ": "0",
"progress": "30",
"result": "PEND",
"status": "ACT",
"stoppable": "no",
"tenq": "2024/06/17 16:16:09",
"tfin": null,
"type": "SWInstall",
"user": null
},
{
"description": null,
"hostid": "021201010640",
"id": 37,
"positionInQ": "0",
"progress": "2024/06/17 16:22:25",
"result": "OK",
"status": "FIN",
"stoppable": "no",
"tenq": "2024/06/17 16:16:09",
"tfin": "2024/06/17 16:22:25",
"type": "SWInstall",
"user": null
}
]
}
@GrantGabbert I did a PR on demisto/content to filter by job id using show jobs id
command. It resolves this issue as it returns a single entry when you query with a job id now. But your point on DT filtering remains the same. Maybe we can close this issue and investigate DT filtering separately.
So the issue was for generic polling failures, and that hasn't been resolved yet. So I think it would make the most sense to keep this issue open until that is resolved (or it's decided to not fix it).
Also, with the changes to the pan-os integration, are there also changes that need to be added in this repo to utilize new show job functionality?
All generic polling playbooks that are looking at jobs like download/install software fail on version 11 due to duplicate job ids in the context.