fortinet-ansible-dev / ansible-galaxy-fortianalyzer-collection

GNU General Public License v3.0
0 stars 1 forks source link

Support for reports (scheduling etc) #5

Open cr0cdev opened 8 months ago

cr0cdev commented 8 months ago

Hi! Is there any plan to add reporting API endpoints support as well? In the documentation there is no information that this collection supports scheduling reports etc. FortiAnalyzer API documentation (https://fndn.fortinet.net/index.php?/fortiapi/175-fortianalyzer/3832/175/report/) states that these endpoints can be used to configure ADOM specific reports scheduling:

/report/run
/report/adom/<adom-name>/config/schedule
etc...

Thank you!

dux-fortinet commented 8 months ago

Hi cr0cdev,

The format of report APIs is quite different from other APIs. I will write new logic to support those report APIs manually.

I can support the following APIs in the next Fortianalyzer Ansible version

/report/run
/report/adom/<adom-name>/config/chart/*
/report/adom/<adom-name>/config/dataset/*
/report/adom/<adom-name>/config/layout/*
/report/adom/<adom-name>/config/layout-folder/*
/report/adom/<adom-name>/config/macro/*
/report/adom/<adom-name>/config/output/*
/report/adom/<adom-name>/config/schedule/*

(/report/adom//config/schedule/* means all APIs starting with /report/adom//config/schedule/, including /report/adom//config/schedule//address-filter)

Please let me know if there are other APIs you want to use.

Currently, you can use the module "faz_generic" (https://ansible-galaxy-fortianalyzer-docs.readthedocs.io/en/latest/faz_generic.html) to call those APIs.

FortiAnalyzer Ansible is expected to be updated every 2 months. And since we just released a version 2 weeks ago, the next release will be in May. Please let me know if you need it urgently.

Best, Xinwei

cr0cdev commented 8 months ago

Hi, Xinwei

Thanks for adding those. Could you also add these so it is possible to initiate reports downloading via Ansible.

/report/adom/<adom-name>/reports/state
/report/adom/<adom-name>/data/id

It would be also good to add the templates to the automation as well in which case we could import/export/list them via automation.

/report/adom/<adom-name>/template/*

Regarding /report/run API path, could you also include /report/adom/<adom-name>/run/<tid> in which case we will be able to get the template run task information. I think we could use faz_fact for this as well, but I haven't tested it as I understand it takes the task from system task list.

Thanks!

dux-fortinet commented 8 months ago

Hi @cr0cdev.

Sure, will do.

I just checked those report APIs in the FNDN. Those APIs are different than other APIs. (faz_generic doesn't support those APIs because those APIs have "jsonrpc": "2.0" in their request body, and faz_generic doesn't support it now. I will fix it in the next version.)

I will try to support those (get) method APIs in faz_fact. But since those APIs are in different formats, if I can't add those APIs to faz_fact, I will create a module to support those APIs.

The APIs we already supported, like/cli/global/system/admin/profile, have get, add, set, update, delete methods. So I can use one module to get/update/delete the corresponding resource. Yet those report APIs only have part methods. In this case, I will create different modules for different methods. (/report/reports/data (delete) and /report/reports/data (get) will be in different modules) and each module only supports one API-method pair.

In this case, the behavior of those report modules is more like a wrapper of the corresponding report API: each module reads YML format playbook, transfers it to JSON format, and sends it to API.

Those APIs will be added in the next version.

Thanks, Xinwei

cr0cdev commented 8 months ago

Thanks Dux!

cr0cdev commented 7 months ago

@dux-fortinet do you have a ETA on when new FAZ collection that supports this will be released?

dux-fortinet commented 6 months ago

Hi @cr0cdev,

Sorry for the late update. We are working on other projects so this update may not be so timely. New FAZ collection will be released 2 weeks later. I'm expecting to release it before May 17th. (If everything goes well, we can release it before May 10th if possible)

Thanks, Dux

dux-fortinet commented 6 months ago

This is the follow-up: The development is finished. All report APIs will be supported in the next version. FortiAnalyzer Ansible is in the QA process. The new version is expected to be released this Friday or early next week. Thanks for your patience.

dux-fortinet commented 6 months ago

FortiAnalzyer Ansible 1.5.0 has been release. All report APIs are supported in this version.

For the APIs that only get information, and don't change data in FortiAnalyzer, they are supported by faz_fact. For other APIs, you can find the relationship between APIs and modules on this page.

Following are some faz_fact examples.

Download generated report. /report/reports/data/<tid> (get)

- name: Download generated report
  hosts: fortianalyzers
  connection: httpapi
  vars:
    ansible_httpapi_use_ssl: true
    ansible_httpapi_validate_certs: false
    ansible_httpapi_port: 443
    adom_name: "root"
    task_id: "6d99e7b0-0bf0-11ef-85b2-000c293bfd45"
  tasks:
    - name: report_reports_data
      fortinet.fortianalyzer.faz_fact:
        facts:
          selector: "report_reports_data"
          params:
            adom: "{{adom_name}}"
            tid: "{{task_id}}"
            # data-type: "text"
            format: "PDF" # HTML, PDF, XML, CSV
      register: response
    # - name: Display data
    #   debug:
    #     var: response.meta.response_data.data

    # Zipped and Base64 encoded data by default. 
    # Plain text for XML and CSV when specified (data-type: "text") in request.
    - name: Base64 decode
      set_fact:
        decode_content: "{{ response.meta.response_data.data | b64decode }}"
    - name: Save content
      copy:
        content: "{{ decode_content }}"
        dest: "./{{task_id}}.zip"

List reports. /report/reports/state (get)

- name: List reports
  hosts: fortianalyzers
  connection: httpapi
  vars:
    ansible_httpapi_use_ssl: true
    ansible_httpapi_validate_certs: false
    ansible_httpapi_port: 443
    adom_name: "root"
  tasks:
    - name: report_reports_state
      fortinet.fortianalyzer.faz_fact:
        facts:
          selector: "report_reports_state"
          params:
            adom: "{{adom_name}}"
            state: "generated" # or "pending-running"
            time-range: # It is required only if "state" is "generated".
              start: "2024-04-17T20:00:00-07:00"
              end: "2024-05-17T20:00:00-07:00"
            # Following params are optional
            # sort-by:
            #   - filed: ""
            #   - order: "asc"
            # timezone: "string"
            # title: "string"
      register: response
    - name: Display response
      debug:
        var: response

Export report templates. /report/template/export (get)

- name: Export report template
  hosts: fortianalyzers
  connection: httpapi
  vars:
    ansible_httpapi_use_ssl: true
    ansible_httpapi_validate_certs: false
    ansible_httpapi_port: 443
    adom_name: "root"
  tasks:
    - name: report_template_export
      fortinet.fortianalyzer.faz_fact:
        facts:
          selector: "report_template_export"
          params:
            adom: "{{adom_name}}"
            # Following params are optional
            # dev-type: "fgt"
            # language: "en"
            # title: "my_template" # Template title, filter for exporting report template.
      register: response
    - name: Base64 decode
      set_fact:
        decode_content: "{{ response.meta.response_data.data | b64decode }}"
    - name: Save content
      copy:
        content: "{{ decode_content }}"
        dest: "./template.tar.gz"

List report templates for a specified language. /report/template/list (get)

- name: Get template list
  hosts: fortianalyzers
  connection: httpapi
  vars:
    ansible_httpapi_use_ssl: true
    ansible_httpapi_validate_certs: false
    ansible_httpapi_port: 443
    adom_name: "root"
  tasks:
    - name: report_template_list
      fortinet.fortianalyzer.faz_fact:
        facts:
          selector: "report_template_list"
          params:
            adom: "{{adom_name}}"
            # Following params are optional
            # dev-type: "fgt"
            # language: "en"
      register: response
    - name: Display response
      debug:
        var: response
cr0cdev commented 6 months ago

Thanks @dux-fortinet you're a life saver with this :)

dux-fortinet commented 6 months ago

You're welcome. If you have any questions about report modules, please feel free to let me know.

cr0cdev commented 6 months ago

You're welcome. If you have any questions about report modules, please feel free to let me know.

I have a question regarding importing report templates. I tried to import a template, but it seems like the API endpoint is missing ADOM parameter.

- name: Import weekly report
  fortinet.fortianalyzer.faz_report_template_import:
    report_template_import:
      data: "{{ lookup('file', './templates/reports/template.tar.gz') | b64encode }}"
      dev_type: "fgt"
    enable_log: true
fatal: [demolabfaz01]: FAILED! => {
    "changed": true,
    "invocation": {
        "module_args": {
            "access_token": null,
            "bypass_validation": false,
            "enable_log": true,
            "forticloud_access_token": null,
            "log_path": "/tmp/fortianalyzer.ansible.log",
            "rc_failed": null,
            "rc_succeeded": null,
            "report_template_import": {
                "data": "H4sIAAwwT2YAA+1abXPaOBDmc37Fju/uWwyyAZtQwkyaXprc9CadJk3uOswwwpaNWltSJUFwOv3vNzKGAOGlvZB02vB8IWiX3dW+aS3nLR6dEhwSWdEkFQnWpLR1IIT8eh1KCDl+Hc1+juH6dXBqnlv1XN/3HUBO3XHqJRht35T7GCiNZQmh+PONc9BAK/k20cdbgennT4IqglTTlBw6vuPVvLpX98ue36g2XOS6e40GvDl7efTu+PTs6s/yCGsty4oEA0l1VlYkoWwwOgyv6x97bj3F1/Xkw8mBc54eUJxefQxPDrJzdnX74Z+/Ivz6QH+4OkDn7O+bo6M934GL49OzN/+ukjlgAWcRZSTsDpq895EEuiubA0Vkt89T0tVNhUp7P9p5vwAmVV95RB2b6t/Uy3z9V13HLUH9EW2a4pnX/zT+b+8dBKrcw4p4tQfrQAh5tdqa/l/1J/Gv+24NkIscb9f/nwS7Dv+8Ma3/rVf9HTbV/0z/H9e/47kIlWB1sW0Rz7z+1/Z/U6Fb0LGx/y/O/y5C1dqu/z8FdvP/88aS/r+lqr/Dhvr3ao63MP97yPN2/f8pkJdZDOpzYksiuNSQ4IwP9B4AAAmpBiePEUI1N18zUERDL7YDnnAJ1m8nOax5Mg8zsFohHYLSWUIOO9YNDXW/6SD0R8dqt/pOu6UEZlNylHCsmwmJ9AuIONO2orek6SIx6ljta0I+JRlcDNIUywze5aa2OhUjod3qVO5Jy21rxpJks9IcJEYvQOAwpCy2ja58qWO1X2GN4fcvphUKIikPv85ID+mw3dK4l5AVm9Fmu+2Wlu2WDqc8QyI1DXBi44TGrKm5MLwRvg36WGpIP5HMSMq3Zhdbsy8ljiIa2C8xC3Ml9hvKSMcCTXUutWNBvn7YsRr1sfGtTmUi1fytww1mvBhLaLrI+MMIEIvBuAuAN1YR4dsUB5KvMptrnNwZPTUq/017xpdinTKnkSsrfADvFY7J/G/Hu+tUjKs7lcLvnUoenO+I0eYwcGEfCTHj90su4EiIhAZYU86gl8EFGRJzbM2ExEdrQvKtWgOsZ1250oRjrEnM5az+6uqUeLDTHpjYXNiXfUmwVgtbGq9+qxMfbsUrnmLKFoy4Jj1FNTFhLdLvp/DqGQtoSJhWdi+zZ/JxsrUpfUW+ug9w9f3mDu6koRRNc3tBO+apkDylioT2KVdzSTRDA0Ob2V/tcVPpvSJyIZHM0vIsWmPLiiyaP1ODotzBuijm5QW6kWhTFnE7pEokOIOQKiNono0PiRQ4JnYPB59iyQcstGmKYwLWF5WpLk3jrsC6/7VTCUnUzfm7vdgshyQaumXBYmuFSMKMPjswFU1sc5zCeGk9v2G0xwfvev6Ic02k3YuL4UNLzJTAkjC9yiRNRno6q4zHs5WsJo5gfQlJhAeJ/jrPFxIVSCry1mu9lUQRpgFDT1ISgSpGk2KKwj0+0KDHObAPOm9x+4CF2AfzGLMPtKjMfQhm0rfPlQbMQlAcOCsvGjC0dSYInHCp6Wus572Un6PrNpozRDilSQbWuSAMLjBTS3jMYQyOu0CY9/zSsa+fXyFsYqIhKabNwuNoCZ3f0HCBQFmQDEJik1TozM7TXS1LF6rsyePMMnKCWTzI052wecvMRGinWMaUgTdH4ZKaRWduUUiuSaDJ0pydbBDHYJkI42AhQyWN+yu0qWFgJzRY2H6RnZeTrdmwdCy+02JQDPjjwMxRYDrh31ueajTJFkss+jRYyVTQwYpMVjKiu2bsnm8SEzAy0nOLhIXLzB0n2/8yN6WM5hX97cpzgvnyo5/Gnh5r7/9oGqst6PiO9/9e1fHz+7/a7v7vSbBw/1dr1Pxy3fE910c1b3f/98tjyf2fqfqt/jvAd7z/H9e/4zk1Z/f+f4cddtjhMfEfjfRZ/QAqAAA=",
                "dev-type": "fgt",
                "dev_type": "fgt"
            },
            "version_check": false
        }
    },
    "meta": {
        "request_url": "/report/template/import",
        "response_data": [],
        "response_error": {
            "code": -32603,
            "message": "Internal error: Report template import failure"
        },
        "response_message": "",
        "system_information": {
            "Admin Domain Configuration": "Enabled",
            "BIOS version": "04000002",
            "Branch Point": "2487",
            "Build": "2487",
            "Current Time": "Thu May 23 15:11:04 EEST 2024",
            "Daylight Time Saving": "Yes",
            "Disk Usage": "Free 454.08GB, Total 491.08GB",
            "FIPS Mode": "Disabled",
            "Hostname": "FAZ",
            "License Status": "Valid",
            "Major": 7,
            "Max Number of Admin Domains": 10000,
            "Minor": 4,
            "Patch": 3,
            "Platform Full Name": "FortiAnalyzer-VM64",
            "Platform Type": "FAZVM64",
            "Release Version Information": " (GA)",
            "Serial Number": "FAZ-",
            "TZ": "Europe/Helsinki",
            "Time Zone": "(GMT+2:00) Helsinki, Riga,Tallinn.",
            "Version": "v7.4.3-build2487 240514 (GA)",
            "x86-64 Applications": "Yes"
        }
    },
    "rc": -32603
}

I also tried to import a template that I downloaded directly from FAZ (.dat) and it still failed to import it.

Can you also show me a example of scheduling task fortinet.fortianalyzer.faz_report_config_schedule with schedule-valid-start and schedule-valid-end defined?

Thanks!

Thanks!

dux-fortinet commented 6 months ago

Hi @cr0cdev,

The report template APIs accept "tar/base64" format data, so (.dat) may not work. I decoded and unzipped your data ("data": "H4sIAAwwT2YAA+1abXPaOBD...). I found your have one extra folder 'template', and FAZ may not parse it correctly.

./template/templates.imgs
./template/templates.base64
./template/templates.conf

Please use tar czvf templates.tar.gz * under the folder template, so templates.tar.gz contains:

./templates.imgs
./templates.base64 # Some templates may have this file and some may not
./templates.conf

Further more, API endpoints /report/template/import (add) and /report/template/delete (add) may not be used to add and delete template. I tried those APIs and checked templates in GUI, and those APIs can't add/delete template. We are consulting with FAZ dev team to figure out the usage of those two APIs.

I'm able to add a template in FAZ 7.4.2 by using faz_report_template_install

- name: Install report template
  hosts: fortianalyzers
  connection: httpapi
  vars:
    ansible_httpapi_use_ssl: true
    ansible_httpapi_validate_certs: false
    ansible_httpapi_port: 443
  tasks:
    - name: Install weekly report
      fortinet.fortianalyzer.faz_report_template_install:
        adom: "root"
        report_template_install:
          data: "{{ lookup('file', '../templates_fixed.tar.gz') | b64encode }}"
          language: 'en'

One more thing, for FAZ 7.4.2, I could install your template successfully by both GUI (Reports->Report Denfinitions->Templates->More->Install Template Pack) and Ansible methods (mentioned above). For FAZ 7.4.3 I can't install your template via either method (I can install my test templates generated by FAZ 7.4.2). Maybe some data in your template is not compatible with FAZ 7.4.3

And for fortinet.fortianalyzer.faz_report_config_schedule, it seems there are differences between the schema in FNDN and the real schema that FAZ accept. Please use bypass_validation: true in the playbook so Ansible won't check the type of schedule-valid-start and schedule-valid-end. Sorry for this mismatch, we will fix it in the next release. schedule-valid-start and schedule-valid-end accept ["00:00", "2023/05/25"] or "2024/05/25 16:09:53"

I'm running out of time today, I haven't figured out how to configure all parameters in fortinet.fortianalyzer.faz_report_config_schedule. Following is just an illustration of how to set schedule-valid-start and schedule-valid-end.

- name: Schedule
  hosts: fortianalyzers
  connection: httpapi
  vars:
    ansible_httpapi_use_ssl: true
    ansible_httpapi_validate_certs: false
    ansible_httpapi_port: 443
  tasks:
    - name: Report schedule
      fortinet.fortianalyzer.faz_report_config_schedule:
        bypass_validation: true
        adom: root
        state: present
        report_config_schedule:
          name: "9"
          description: "test"
          devices:
            - devices-name: "All_Device"
          schedule-valid-start: ["00:00", "2023/05/25"] # accept format 1
          # schedule-valid-start: "2024/05/25 16:09:53" # accept format 2
          schedule-valid-end: null # never end

          # other parameters...
          # report-layout: 
          # - is-global: 0
          #   layout-id: 9

PS: It seems you need to config faz_report_config_report faz_report_config_layout and set report-layout in faz_report_config_schedule correctly, otherwise, the GUI 'Reports->Advanced Settings->Report Calendar' may not display correctly.

cr0cdev commented 6 months ago

Thanks @dux-fortinet for excellent explanation. I owe you a :beer: :smile: ! Am I missing something or is faz_report_config_report missing from the modules? I do not see it under modules folder either.

dux-fortinet commented 6 months ago

Thanks @dux-fortinet for excellent explanation. I owe you a 🍺 😄 ! Am I missing something or is faz_report_config_report missing from the modules? I do not see it under modules folder either.

Sorry, typo. faz_report_config_layout. It seems you need to configure faz_report_config_layout and set report-layout in faz_report_config_schedule correctly. Otherwise, the GUI may have some display bugs. I'm not sure if this affects the functionality of the schedule, I'll investigate it.

cr0cdev commented 6 months ago

Thanks @dux-fortinet I've tried to modify predefined reports schedule using the module and sadly it does not allow me to do it. Without defining language:

failed: [fortianalyzer01] (item={'name': 'Daily Summary Report', 'disp_name': 'Daily Summary Report', 'desc': 'Testing', 'state': 'present', 'status': 'enable', 'layout_id': 1000060042, 'output_format': 'pdf', 'schedule_type': 'every-n-days', 'time_period': 'yesterday', 'week_start': 'mon'}) => {
    "ansible_loop_var": "item",
    "changed": true,
    "invocation": {
        "module_args": {
            "access_token": null,
            "adom": "ansible-test-1",
            "bypass_validation": true,
            "enable_log": false,
            "forticloud_access_token": null,
            "log_path": "/tmp/fortianalyzer.ansible.log",
            "proposed_method": null,
            "rc_failed": null,
            "rc_succeeded": null,
            "report_config_schedule": {
                "description": "Testing",
                "name": "Daily Summary Report",
                "output_format": "pdf",
                "report_layout": [
                    {
                        "layout_id": "1000060042"
                    }
                ],
                "schedule_type": "every-n-days",
                "status": "enable",
                "time_period": "yesterday",
                "week_start": "mon"
            },
            "state": "present",
            "version_check": false
        }
    },
    "item": {
        "desc": "Testing",
        "disp_name": "Daily Summary Report",
        "layout_id": 1000060042,
        "name": "Daily Summary Report",
        "output_format": "pdf",
        "schedule_type": "every-n-days",
        "state": "present",
        "status": "enable",
        "time_period": "yesterday",
        "week_start": "mon"
    },
    "meta": {
        "request_url": "/report/adom/ansible-test-1/config/schedule",
        "response_data": {
            "status": {
                "code": -10131,
                "message": "datasrc invalid. object: sql-report schedule.1000060042:language. detail: en. solution: data not exist"
            }
        },
        "response_message": "",
        "system_information": {
            "Admin Domain Configuration": "Enabled",
            "BIOS version": "04000002",
            "Branch Point": "1460",
            "Build": "1460",
            "Current Time": "Mon May 27 09:38:11 EEST 2024",
            "Daylight Time Saving": "Yes",
            "Disk Usage": "Free 135.32GB, Total 146.64GB",
            "FIPS Mode": "Disabled",
            "Hostname": "FAZ",
            "License Status": "Valid",
            "Major": 7,
            "Max Number of Admin Domains": 2,
            "Minor": 2,
            "Patch": 4,
            "Platform Full Name": "FortiAnalyzer-VM64-HV",
            "Platform Type": "FAZVM64-HV",
            "Release Version Information": " (GA)",
            "Serial Number": "FAZ-",
            "TZ": "Europe/Helsinki",
            "Time Zone": "(GMT+2:00) Helsinki, Riga,Tallinn.",
            "Version": "v7.2.4-build1460 230926 (GA)",
            "x86-64 Applications": "Yes"
        }
    },
    "rc": -10131
}

With defining language:

`failed: [fortianalyzer01] (item={'name': 'Daily Summary Report', 'disp_name': 'Daily Summary Report', 'desc': 'Testing', 'state': 'present', 'status': 'enable', 'layout_id': 1000060042, 'output_format': 'pdf', 'schedule_type': 'every-n-days', 'time_period': 'yesterday', 'week_start': 'mon', 'language': 'en'}) => {
    "ansible_loop_var": "item",
    "changed": true,
    "invocation": {
        "module_args": {
            "access_token": null,
            "adom": "ansible-test-1",
            "bypass_validation": true,
            "enable_log": false,
            "forticloud_access_token": null,
            "log_path": "/tmp/fortianalyzer.ansible.log",
            "proposed_method": null,
            "rc_failed": null,
            "rc_succeeded": null,
            "report_config_schedule": {
                "description": "Testing",
                "name": "Daily Summary Report",
                "output_format": "pdf",
                "report_layout": [
                    {
                        "layout_id": "1000060042"
                    }
                ],
                "schedule_type": "every-n-days",
                "status": "enable",
                "time_period": "yesterday",
                "week_start": "mon"
            },
            "state": "present",
            "version_check": false
        }
    },
    "item": {
        "desc": "Testing",
        "disp_name": "Daily Summary Report",
        "language": "en",
        "layout_id": 1000060042,
        "name": "Daily Summary Report",
        "output_format": "pdf",
        "schedule_type": "every-n-days",
        "state": "present",
        "status": "enable",
        "time_period": "yesterday",
        "week_start": "mon"
    },
    "meta": {
        "request_url": "/report/adom/ansible-test-1/config/schedule",
        "response_data": {
            "status": {
                "code": -10131,
                "message": "datasrc invalid. object: sql-report schedule.1000060042:language. detail: en. solution: data not exist"
            }
        },
        "response_message": "",
        "system_information": {
            "Admin Domain Configuration": "Enabled",
            "BIOS version": "04000002",
            "Branch Point": "1460",
            "Build": "1460",
            "Current Time": "Mon May 27 09:38:50 EEST 2024",
            "Daylight Time Saving": "Yes",
            "Disk Usage": "Free 135.32GB, Total 146.64GB",
            "FIPS Mode": "Disabled",
            "Hostname": "FAZ",
            "License Status": "Valid",
            "Major": 7,
            "Max Number of Admin Domains": 2,
            "Minor": 2,
            "Patch": 4,
            "Platform Full Name": "FortiAnalyzer-VM64-HV",
            "Platform Type": "FAZVM64-HV",
            "Release Version Information": " (GA)",
            "Serial Number": "FAZ-",
            "TZ": "Europe/Helsinki",
            "Time Zone": "(GMT+2:00) Helsinki, Riga,Tallinn.",
            "Version": "v7.2.4-build1460 230926 (GA)",
            "x86-64 Applications": "Yes"
        }
    },
    "rc": -10131
}

Not exactly sure if I need to create a brand new layout with the same settings as predefined report and then schedule the new report (which wouldn't be a ideal solution).

dux-fortinet commented 6 months ago

Hi @cr0cdev,

I thought you wanted to create a brand new schedule (which is a little bit hard and complex and I'm still investing in it). If you just want to modify the existing schedule, it is much easier.

You don't need to specify report_layout under report_config_schedule if you just want to update the existing schedule. You just need to specify the parameter you want to update, and the rest of the parameters will be unchanged.

Everything in your previous playbook is correct except the "name". The name for "report_config_schedule" is a digital ID rather than a string. The ID of "Daily Summary Report" is "10042". (Not "1000060042". I tried to modify "1000060042", and "Daily Summary Report" in GUI 'Reports->Report Definitions->All Reports->Daily Summary Report' didn't change. )

Following playbook works well in my test environment:

    - name: Report schedule
      fortinet.fortianalyzer.faz_report_config_schedule:
        bypass_validation: true
        adom: "{{adom_name}}"
        state: present
        report_config_schedule:
          name: "10042"
          description: "Daily Summary Report" # Changing the description does not affect the data in the GUI
          output_format: "pdf" # Maybe no need for FAZ 7.x
          schedule_type: "every-n-days"
          status: "enable"
          time_period: "yesterday"
          week_start: "mon"
          schedule-valid-start: ["11:55", "2023/05/23"]
          schedule-valid-end: null # never end
          # more parameters....

Thanks, Dux

cr0cdev commented 6 months ago

Thanks Dux for the explanation. To be more clear I need to modify existing schedules and I do need to create new ones as well. Another question I have is how can I find the proper digital ID of certain report? I've tried searching in the GUI, but without any luck and faz_fact doesn't seem to have a selector to list all reports with their ID's.

dux-fortinet commented 6 months ago

Sorry for faz_fact missing this selector, we will support it in the next version.

Please try following playbook to get target ID.

- name: Find existing schedule ID
  hosts: fortianalyzers
  connection: httpapi
  vars:
    ansible_httpapi_use_ssl: true
    ansible_httpapi_validate_certs: false
    ansible_httpapi_port: 443
    adom_name: root
    target_description: "Daily Summary Report"
  tasks:
    - name: Get all schedule data
      fortinet.fortianalyzer.faz_generic:
        enable_log: true
        method: "get"
        params:
          - url: "/report/adom/{{adom_name}}/config/schedule/"
      register: schdule_data

    # - name: Save schdule_data to JSON file
    #   ansible.builtin.copy:
    #     content: "{{ schdule_data }}"
    #     dest: ./schdule_data.json

    - name: Find dict with description "{{target_description}}"
      set_fact:
        target_dict: "{{ item }}"
      when: item['is-template'] == 0 and item.description is defined and item.description == "{{target_description}}"
      with_items: "{{ schdule_data.meta.response_data.data }}"

    - name: Display the target name
      debug:
        var: target_dict.name
      when: target_dict is defined
dux-fortinet commented 6 months ago

Hi @cr0cdev ,

Here is an example of how to create a new report based on a template and set its schedule. It seems JSON API doesn't have API related to creating a report based on the template, I used GUI API to finish this task. The following example also shows you how to use Ansible to act like a browser and call GUI API (login and get cookies, send GET request, send POST request) and it works well in my test environment.

(Update: this script works with FAZ 7.4.X.) (For FAZ 7.2.X, please check the script posted on June 4th )

- name: Ansible send post/get request
  hosts: fortianalyzers
  gather_facts: no
  connection: httpapi
  vars:
    ansible_httpapi_use_ssl: true
    ansible_httpapi_validate_certs: false
    ansible_httpapi_port: 443
    target_template_name: "Template - 360 Protection Report"
    report_name: "ansible"
    adom_name: root
  tasks:
    - name: Login via GUI API
      uri:
        url: "https://{{ansible_host}}/cgi-bin/module/flatui_auth"
        method: POST
        body: '{"url": "/gui/userauth","method": "login","params": {"secretkey": "{{ansible_password}}","username": "{{ansible_user}}"}}'
        body_format: json
        validate_certs: false # FAZ certificate is self-signed
        headers:
          Content-Type: "application/json"
      register: login_response
      delegate_to: localhost

    - name: Use GET method to find target template
      uri:
        url: "https://{{ansible_host}}/p/report/template/list/all/"
        method: GET
        headers:
          Cookie: "{{ login_response.cookies_string }}"
        validate_certs: false
      register: get_response
      delegate_to: localhost
    # - name: Print get_response
    #   debug:
    #     var: get_response
    - name: Find dictionary with title "{{target_template_name}}"
      set_fact:
        target_template: "{{ get_response.json | selectattr('title', 'equalto', target_template_name) | list | first}}"
    - name: Print the target template
      debug:
        var: target_template

    # target_template["layout-id"] is your template ID
    - name: Create a new report based on a template
      uri:
        url: "https://{{ansible_host}}/p/report/layout/create/"
        method: POST
        body: '{"title":"{{report_name}}","folders":[99999],"create-from":"template","layout-id":{{target_template["layout-id"]}},"is-global":0}'
        body_format: json
        validate_certs: false
        headers:
          Content-Type: "application/json"
          Cookie: "{{ login_response.cookies_string }}"
          Xsrf-Token: "{{ login_response.cookies.HTTP_CSRF_TOKEN }}"
          Referer: "https://{{ansible_host}}"
      register: create_report_response
      delegate_to: localhost
    - name: Print create_report_response
      debug:
        var: create_report_response

    # create_report_response.json["layout-id"] is your schedule id
    - name: Config schedule
      fortinet.fortianalyzer.faz_report_config_schedule:
        enable_log: true
        bypass_validation: true
        adom: "{{adom_name}}"
        state: present
        report_config_schedule:
          name: "{{create_report_response.json['layout-id']}}"
          schedule_type: "every-n-days"
          status: "enable"
          time_period: "yesterday"
          week_start: "mon"
          schedule-valid-start: ["17:44", "2023/05/31"]
          schedule-valid-end: null # never end
      register: schedule_response
    - name: Print schedule_response
      debug:
        var: schedule_response

Best, Dux

cr0cdev commented 5 months ago

Thanks @dux-fortinet for the example :) I will test it out.

cr0cdev commented 5 months ago

@dux-fortinet did you ever get 403 when the playbook tried to create a new report from the template? I'm trying exactly the same code with same template. Logging in, listing reports etc work, but once it tried to create a new report from the template we get a 403 error. Not exactly sure what is going on since the same headers (cookies) work fine on other tasks. I also did try to create the report from template manually via GUI and double checked all the parameters, they are all the same. This is the result when trying to create the report from template:

fatal: [fortianalyzer01 -> localhost]: FAILED! => {
    "changed": false,
    "connection": "Upgrade, close",
    "content_language": "en",
    "content_security_policy": "default-src 'self'; frame-ancestors 'none'; object-src 'none'; script-src 'self'; style-src 'self' 'unsafe-inline';",
    "content_type": "text/html",
    "date": "Mon, 03 Jun 2024 08:11:13 GMT",
    "elapsed": 0,
    "invocation": {
        "module_args": {
            "attributes": null,
            "body": {
                "create-from": "template",
                "folders": [
                    99999
                ],
                "is-global": 0,
                "layout-id": 1000060034,
                "title": "ansible"
            },
            "body_format": "json",
            "ca_path": null,
            "ciphers": null,
            "client_cert": null,
            "client_key": null,
            "creates": null,
            "decompress": true,
            "dest": null,
            "follow_redirects": "safe",
            "force": false,
            "force_basic_auth": false,
            "group": null,
            "headers": {
                "Content-Type": "application/json",
                "Cookie": "CURRENT_SESSION=OQCGr9sV1GsP17aTpsdoFUt/G8ekmDpMdUZ1UKKxYx1v5PGOCrzbZIW4+ioQfSDnGuK+WLNI46kvdTNhRLE89Q==; auth_state=; remoteauth=; selectadom=1; HTTP_CSRF_TOKEN=VImKFyqpbseQzXI4nFSJ5vcbtKfFZ3I",
                "Referer": "https://XXXXX",
                "Xsrf-Token": "VImKFyqpbseQzXI4nFSJ5vcbtKfFZ3I"
            },
            "http_agent": "ansible-httpget",
            "method": "POST",
            "mode": null,
            "owner": null,
            "remote_src": false,
            "removes": null,
            "return_content": false,
            "selevel": null,
            "serole": null,
            "setype": null,
            "seuser": null,
            "src": null,
            "status_code": [
                200
            ],
            "timeout": 30,
            "unix_socket": null,
            "unredirected_headers": [],
            "unsafe_writes": false,
            "url": "https://XXXX/p/report/layout/create/",
            "url_password": null,
            "url_username": null,
            "use_gssapi": false,
            "use_netrc": true,
            "use_proxy": true,
            "validate_certs": false
        }
    },
    "msg": "Status code was 403 and not [200]: HTTP Error 403: Forbidden",
    "redirected": false,
    "referrer_policy": "strict-origin-when-cross-origin",
    "status": 403,
    "strict_transport_security": "max-age=63072000",
    "transfer_encoding": "chunked",
    "upgrade": "h2",
    "url": "https://XXXX/p/report/layout/create/",
    "vary": "Accept-Encoding",
    "x_content_type_options": "nosniff",
    "x_frame_options": "SAMEORIGIN",
    "x_ua_compatible": "IE=Edge",
    "x_xss_protection": "1; mode=block"
}

I'm running my test environment on FAZ 7.2.4.

Thanks!

dux-fortinet commented 5 months ago

Hi @cr0cdev,

My test environment is also 7.2.4. If I ran my above-mentioned script and changed target_template_name to "Template - Weekly Summary Report", I didn't get this error. I have asked my colleague to look into it.

HTTP 403 code means the server received and understood your request, yet it refused to do it. It usually means something wrong with authorization. For example, I will receive HTTP 403 if my Referer does not start with "https://{{ansible_host}}".

We will let you know when my colleague has any updates.

Best, Dux

cr0cdev commented 5 months ago

Hi @dux-fortinet The interesting part is that authorization works for login, listing templates and getting template ID, but when it tries to create a report from the template it fails. It seems like the user is not allowed to create the report, but the user itself has superuser access which should allow the user to create the report from template. If I try to create a report from GUI then it works fine. Until your colleague answers, I will see if there is any issues with the headers that are being sent.

Update: Added return_content: true to the report creation task and it seems that CSRF cookie is not sent correctly.

<body>\n<div id=\"summary\">\n <h1>Forbidden <span>(403)</span></h1>\n <p>CSRF verification failed. Request aborted.</p>
\n\n\n <p>You are seeing this message because this site requires a CSRF cookie when submitting forms. This
cookie is required for security reasons, to ensure that your browser is not being hijacked by third parties.
</p>\n <p>If you have configured your browser to disable cookies, please re-enable them, at least for this site,
or for “same-origin” requests.</p>\n\n</div>\n\n<div id=\"explanation\">\n <p><small>More information is
available with DEBUG=True.</small></p>\n</div>\n\n</body>\n

I will dig around a bit to see if I can find out why it doesn't want to use the CSRF cookie that is being sent, as the cookie itself is correct.

Thanks!

dux-fortinet commented 5 months ago

Hi @cr0cdev,

I finally figured it out. Previously, my environment was 7.4.2, not 7.2.4, sorry for the confusion. The authorization mechanism that FAZ GUI API used is different between FAZ 7.4.x and FAZ 7.2.x.

For FAZ 7.4.x, you can use my previously mentioned playbook. For FAZ 7.2.x, after you log in, you need to get csrf_token and specify it in every POST request manually.

Following playbook works well in my test environments (tested on FAZ 7.2.4 and FAZ 7.2.5)

- name: Ansible send post/get request
  hosts: fortianalyzers
  gather_facts: no
  connection: httpapi
  vars:
    ansible_httpapi_use_ssl: true
    ansible_httpapi_validate_certs: false
    ansible_httpapi_port: 443
    target_template_name: "Template - 360 Security Report"
    report_name: "faz_72"
    adom_name: root
  tasks:
    - name: Login via GUI API
      uri:
        url: "https://{{ansible_host}}/cgi-bin/module/flatui_auth"
        method: POST
        body: '{"url": "/gui/userauth","method": "login","params": {"secretkey": "{{ansible_password}}","username": "{{ansible_user}}"}}'
        body_format: json
        validate_certs: false # FAZ certificate is self-signed
        headers:
          Content-Type: "application/json"
      register: login_response
      delegate_to: localhost

    - name: Get csrf token
      uri:
        url: "https://{{ansible_host}}/p/login/"
        method: GET
        validate_certs: false
        return_content: true
        headers:
          Cookie: "{{ login_response.cookies_string }}"
      register: csrf_response
      delegate_to: localhost

    - name: Use GET method to find target template
      uri:
        url: "https://{{ansible_host}}/p/report/template/list/all/"
        method: GET
        headers:
          Cookie: "{{ login_response.cookies_string }}"
        validate_certs: false
        return_content: true
      register: get_response
      delegate_to: localhost
    - name: Print the get_response
      debug:
        var: get_response

    - name: Find dictionary with title "{{target_template_name}}"
      set_fact:
        target_template: "{{ get_response.json | selectattr('title', 'equalto', target_template_name) | list | first}}"
    - name: Print the target template
      debug:
        var: target_template

    # target_template["layout-id"] is your template ID
    - name: Create a new report based on a template
      uri:
        url: "https://{{ansible_host}}/p/report/layout/create/"
        method: POST
        body: '{"title":"{{report_name}}","folders":[99999],"create-from":"template","layout-id":{{target_template["layout-id"]}},"is-global":0}'
        body_format: json
        validate_certs: false
        return_content: true
        headers:
          Content-Type: "application/json"
          Cookie: "{{ login_response.cookies_string }} {{ csrf_response.cookies_string }}" # need {{ csrf_response.cookies_string }} in FAZ 7.2.X
          # Xsrf-Token: "{{ login_response.cookies.HTTP_CSRF_TOKEN }}" # Used in FAZ 7.4.x
          X-Csrftoken: "{{ csrf_response.cookies.csrftoken }}" # Used in FAZ 7.2.X
          Referer: "https://{{ansible_host}}"
      register: create_report_response
      delegate_to: localhost
    - name: Print create_report_response
      debug:
        var: create_report_response

    # create_report_response.json["layout-id"] is your schedule id
    - name: Config schedule
      fortinet.fortianalyzer.faz_report_config_schedule:
        enable_log: true
        bypass_validation: true
        adom: "{{adom_name}}"
        state: present
        report_config_schedule:
          name: "{{create_report_response.json['layout-id']}}"
          schedule_type: "every-n-days"
          status: "enable"
          time_period: "yesterday"
          week_start: "mon"
          schedule-valid-start: ["18:55", "2023/05/31"]
          schedule-valid-end: null # never end
      register: schedule_response
    - name: Print schedule_response
      debug:
        var: schedule_response

Best, Dux

cr0cdev commented 5 months ago

Thanks for looking into it as well @dux-fortinet. Now the example code works. I will try to add loops into it etc and also share the end playbook in here for everybody to use.

Update: Unfortunately it seems that the template is always created under root ADOM and if you want to create a report for a different ADOM it does not work. It seems that the GUI API endpoint does not have a parameter to define under which ADOM to create the report.

cr0cdev commented 2 months ago

Update: I'm talking to FAZ CSE regarding this issue and we are trying to figure out how to do this so once we have a solution I will share the info here as well.