Need help to connect to Grafana cloud

[franklinis]

Hey Guys,

First of all, it's a great job and I am trying to get this working, but with no luck so far...!

This is my first time using Prometheus and Grafana. Your help is really appreciated :)

My Situation:

1. I have setup the fortigate exporter and prometheus on my linux pc

2. As mentioned in the guide, I used [http://localhost:9710/probe, but that didn't work and was with the error "404 Not found". Post changing to metrics, job is now alright.

3. I have also established the connection to grafana cloud and this prometheus. But I don't see anywhere except the job name

I would be happy to provide any additional info.

I really appreciate your help again, Thanks in advance :)

[bluecmd]

Hi, have a look at https://github.com/bluecmd/fortigate_exporter?tab=readme-ov-file#prometheus-configuration. When you scrape /metrics you are scraping the internal metrics of the exporter, not your device.

[franklinis]

@bluecmd Thanks for the quick reply. That was my previous config and it always the same error as below.

[bluecmd]

Have you added https://192.168.1.99 to your Fortigate Exporter configuration?

[franklinis]

Yes, I have added it in fortigate-key.yaml file

[franklinis]

If you think, I might done something totally wrong. It would be great to have your detailed guidance. Because I tried my best following your guide in git and also in reddit

[bluecmd]

Is it added exactly as you've written the target? E.g. have you put the port in the config but not the Prometheus target? They have to match exactly. If you have, please post your Prometheus config and exporter config but redact any sensitive data.

[franklinis]

I believe so ;)

Below is the configuration of my forti-exporter, prometheus and grafana agent :

fortigate-key.yaml:

"https://192.168.1.99":
  token: xxxxxxxxxxxxxxxxxxxxxxx
  # If you have a smaller fortigate unit you might want
  # to exclude sensors as they do not have any
  probes:
    exclude:
      # - System/SensorInfo

prometheus.yaml:

# my global config
global:
  scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
    - static_configs:
        - targets:
          # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'fortigate_exporter'
    metrics_path: /probe
    static_configs:
      - targets:
        - https://192.168.1.99
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
        # Drop the https:// and port (if specified) for the 'instance=' label
        regex: '(?:.+)(?::\/\/)([^:]*).*'
      - target_label: __address__
        replacement: 'localhost:9710'

remote_write:
  - url: https://prometheus-prod-xx-prod-xx-xxxx-x.grafana.net/api/prom/push
    basic_auth:
      username: xxxxxx
      password: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Grafana-agent,yaml:

integrations:
  prometheus_remote_write:
  - basic_auth:
      password: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
      username: xxxxxxx
    url: https://prometheus-prod-xx-prod-xx-xxxx-x.grafana.net/api/prom/push
  agent:
    enabled: true
    relabel_configs:
    - action: replace
      source_labels:
      - agent_hostname
      target_label: instance
    - action: replace
      target_label: job
      replacement: "integrations/agent-check"
    metric_relabel_configs:
    - action: keep
      regex: (prometheus_target_sync_length_seconds_sum|prometheus_target_scrapes_.*|prometheus_target_interval.*|prometheus_sd_discovered_targets|agent_build.*|agent_wal_samples_appended_total|process_start_time_seconds)
      source_labels:
      - __name__
  # Add here any snippet that belongs to the `integrations` section.
  # For a correct indentation, paste snippets copied from Grafana Cloud at the beginning of the line.
logs:
  configs:
  - clients:
    - basic_auth:
        password: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        username: xxxxxxxxxxxxx
      url: https://logs-prod-xxxxx.grafana.net/loki/api/v1/push
    name: integrations
    positions:
      filename: /tmp/positions.yaml
    scrape_configs:
      # Add here any snippet that belongs to the `logs.configs.scrape_configs` section.
      # For a correct indentation, paste snippets copied from Grafana Cloud at the beginning of the line.
metrics:
  configs:
  - name: integrations
    remote_write:
    - basic_auth:
        password: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        username: xxxxxx
      url: https://prometheus-prod-xx-prod-xx-xxxx-x.grafana.net/api/prom/push
    scrape_configs:
      # Add here any snippet that belongs to the `metrics.configs.scrape_configs` section.
      # For a correct indentation, paste snippets copied from Grafana Cloud at the beginning of the line.
  global:
    scrape_interval: 60s
  wal_directory: /tmp/grafana-agent-wal

[bluecmd]

Can you post the logs from the exporter?

[franklinis]

Can you help me to understand how I can get the logs ? I am beginner in this area

[bluecmd]

How are you running the exporter? Through docker? Systemd? Screen?

[franklinis]

I not using docker and I am not sure what you exactly you mean by running the exporter. So far, I have just restarted the process of prometheus and grafana by using the below commands

sudo systemctl restart prometheus and sudo systemctl restart grafana-agent.service

And I can confirm that both are running

[bluecmd]

Well, you have installed this exporter somehow, right? I mean, it doesn't come installed and all the installation is manual - so you or somebody needs to download the software and install it. That somebody also needs to make sure the exporter is told where the fortigate-key.yaml file is.

[franklinis]

Yes, I did the installation. I downloaded the binary and modified the fortigate-key.yaml. and then added the details of the job to prometheus configuration.

Can you tell me how I can say the exporter that this is where the fortigate-key.yaml file is ? I am doing it on ubuntu.

I tried running the command _

sudo ./fortigate_exporter -auth-file ~/fortigate-key.yaml

_ , but it ends with the error, sudo: ./fortigate_exporter: command not found. I am running this from the folder where the binaries are placed.

[bluecmd]

Sorry, don't have time to hold hands on basic Linux administration. I'll leave the issue open for others to chime in if they want to.

[franklinis]

Hey @bluecmd

I managed to run the exporter on my linux, now it's up and running. Maybe can you help me now ?

I can see the job data in Grafana cloud dashboard for prometheus. Maybe can you help me now ?

However, in the dashboard I couldn't get any data even if I enter the variable data as the IP address of the firewall or as http://localhost:9710. Can you help me here ?

In the logs, I have the below error

[franklinis]

Update :

I got it running with -insecure. Now I am stuck with next error as below :(

However, If I visit https://192.168.1.99/api/v2/monitor/system/status/?access_token=xxxxxxxxxx

It gives output as well as the curl as `$ curl -vvv -k -H "Authorization: Bearer xxxxxxxxxxxxxxxxxxxxx" "https://192.168.1.99/api/v2/cmdb/firewall/address/"'