type=playbook - question for outputformatset usage

[Schlumpfbert]

Describe the bug

Output file generated by outputformatset seems buggy, sometimes 0 byte size or not-working commands

Expected behavior

I run the following command and playbook : Command: pan-os-php type=playbook json=pb_cleanup_unused-objects.json in=Export_Exped_2023-07-11.xml stagename=stages/stage- outputformatset=set-commands.txt out=cleanconfig.xml

Playbook: { "command": [ { "type": "address", "location": "location=any", "actions": "actions=exporttoexcel:addresses_unused.html", "filter": "filter=(object is.unused.recursive )", "shadow-ignoreinvalidaddressobjects": "shadow-ignoreinvalidaddressobjects" }, { "type": "service", "location": "location=any", "actions": "actions=exporttoexcel:services_unused.html", "filter": "filter=(object is.unused.recursive )" }, { "type": "tag", "location": "location=any", "actions": "actions=exporttoexcel:tags_unused.html", "filter": "filter=(object is.unused )" } ] }

I would expect that the file set-commands.txt contains all necessary set commands in the correct order to delete all unused objects (addresses, services, tags).

Current behavior

For some playbooks like the above the created set-commands.txt file is empty. For other playbooks, among correct commands, I also get commands in that file like this: delete Ton-Contractor_192.168.20.x delete Ton-Contractor_192.168.20.x delete Ton-Contractor_192.168.20.x

So not only does the command exist several times, it would obviously also not work without device group etc.

Possible solution

I think that the created set file was correct with previous pan-os-php versions.

Steps to reproduce

I ran the script several times with both develop and latest. docker pull swaschkut/pan-os-php:develop docker run -it -v /home/expedition/:/share swaschkut/pan-os-php:develop docker pull swaschkut/pan-os-php:latest docker run -it -v /home/expedition/:/share swaschkut/pan-os-php:latest

Screenshots

Context

Your Environment

Azure Ubuntu VM, running on our Expedition server

$ uname -a Linux servername 5.15.0-1038-azure #45~20.04.1-Ubuntu SMP Tue Apr 25 18:45:15 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

$ pan-os-php version

• PAN-OS-PHP version: 2.1.9 [UNIX]
• /tools/pan-os-php/utils
• PHP version: 8.1.2-1ubuntu2.13

[welcome-to-palo-alto-networks[bot]]

:tada: Thanks for opening your first issue here! Welcome to the community!

[swaschkut]

thanks for reaching out and sharing your findings with me. Please allow me to check this in the next coming two weeks

[swaschkut]

please allow me to re-ask:

for all three playbook task you are looking for unused objects to generate a HTML spreadsheet.

What is your expectation for the set commands, if you are not manipulating the config file at all? all these three HTML files I assume are available, and the set-command file, must be empty;

as it is not empty, the reason is that playbook file is adding this to existing file if available;

it is also empty as you defined NOT to delete any unused object, you only create spreadsheets;

please change your JSON playbook file to e.g. actions=delete so that the set-command.txt file is creating lines which you can send.

{ "command": [ { "type": "address", "location": "location=any", "actions": "actions=delete", "filter": "filter=(object is.unused.recursive )", "shadow-ignoreinvalidaddressobjects": "shadow-ignoreinvalidaddressobjects" }, { "type": "service", "location": "location=any", "actions": "actions=delete", "filter": "filter=(object is.unused.recursive )" }, { "type": "tag", "location": "location=any", "actions": "actions=delete", "filter": "filter=(object is.unused )" } ] }

please re-run your playbook command with a JSON playbook file, which is manipulating your configuration file

[Schlumpfbert]

OMG, you made my day, your playbook works, thank you so much! I totally misunderstood how that script works, sorry to waste your time. I was under the assumption that the set-commands file would always be created when using the outputformatset parameter.

If I may ask one last question, I wanted to use this for merging our duplicate address objects: { "type": "address-merger", "location": "location=any", "filter": "filter=!(object is.fqdn)", "DupAlgorithm": "DupAlgorithm=SameAddress", "exportcsv": "exportcsv=addresses_tomerge.html", "allowmergingwithupperlevel": "allowmergingwithupperlevel", "shadow-ignoreinvalidaddressobjects": "shadow-ignoreinvalidaddressobjects" }

Now I am unsure what would be the correct mechanism to create the set-commands file...? Is there maybe some website with examples?

[swaschkut]

you can also always create a separate set-command file for each task, so maybe this:

{ "type": "address-merger", "location": "location=any", "filter": "filter=!(object is.fqdn)", "DupAlgorithm": "DupAlgorithm=SameAddress", "exportcsv": "exportcsv=addresses_tomerge.html", "allowmergingwithupperlevel": "allowmergingwithupperlevel", "shadow-ignoreinvalidaddressobjects": "shadow-ignoreinvalidaddressobjects" "outputformatset": "outputformatset=addr-merge_set-command.txt" }

[Schlumpfbert]

Awesome, thank you very much for your quick help. :-)

[swaschkut]

yw; based on your GitHub name, I assume best greetings from Germany to Germany :-)

[Schlumpfbert]

Da hast du absolut recht. :-) Beste Grüße, Carsten

[Schlumpfbert]

Moin Sven,

falls du das hier noch lesen solltest, ich habe immer noch das Problem mit fehlerhaften set commands. In der generierten txt Datei stehen zwar viele korrekte Befehle, aber gerade am Ende z.B. auch sowas:

delete n-i-az-SoutheastAsia_10.152.0.0-16 delete n-i-az-SoutheastAsia_10.152.0.0-16 delete translated-address h-az-SEA-dmz-10.152.32.68 delete description "LG DE LAN Team server abc"

Es stehen also doppelte Einträge drin und auch commands, die nicht funktionieren können.

Ich habe diesen Befehl und folgendes Playbook verwendet:

pan-os-php type=playbook json=pb_cleanup_merges.json in=Export_Exped_2023-07-14.xml stagename=stages/stage- out=cleanconfig.xml

{ "command": [ { "type": "address-merger", "location": "location=any", "filter": "filter=!(object is.fqdn)", "DupAlgorithm": "DupAlgorithm=SameAddress", "exportcsv": "exportcsv=addresses_tomerge.html", "allowmergingwithupperlevel": "allowmergingwithupperlevel", "shadow-ignoreinvalidaddressobjects": "shadow-ignoreinvalidaddressobjects", "outputformatset": "outputformatset=set-command_merge_addr-nofqdn.txt" } ] }

[swaschkut]

Hallo Michael, bitte schreibe mich kurz via swaschkut [at] paloaltonetworks.com

vieles lässt sich ueber nicht oeffentliche Wege einfacher klären :-)