Atlassian Jira (cloud): Can't collect audit logs

[Myasnik2000]

We try to set up Atlassian Jira integration for Atlassian Cloud and filled all required fields. Also run a stand alone elastic-agent in k8s and configure it like suggested integration but we don't get any logs. Pls can anyone help? I use this configuration code

apiVersion: v1
kind: ConfigMap
metadata:
  name: agent-node-datastreams
  namespace: elk
  labels:
    k8s-app: elastic-agent-standalone
data:
  agent.yml: |-
    outputs:
      default:
        type: elasticsearch
        hosts: [******"]
        username: "***"
        password: "***"
    output_permissions:
      default:
        _elastic_agent_monitoring:
          indices:
            - names:
                - logs-elastic_agent.apm_server-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-elastic_agent.apm_server-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - logs-elastic_agent.auditbeat-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-elastic_agent.auditbeat-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - logs-elastic_agent.cloudbeat-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-elastic_agent.cloudbeat-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - logs-elastic_agent-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-elastic_agent.elastic_agent-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-elastic_agent.endpoint_security-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - logs-elastic_agent.endpoint_security-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - logs-elastic_agent.filebeat-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-elastic_agent.filebeat-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - logs-elastic_agent.fleet_server-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-elastic_agent.fleet_server-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - logs-elastic_agent.heartbeat-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-elastic_agent.heartbeat-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - logs-elastic_agent.metricbeat-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-elastic_agent.metricbeat-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - logs-elastic_agent.osquerybeat-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-elastic_agent.osquerybeat-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - logs-elastic_agent.packetbeat-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-elastic_agent.packetbeat-default
              privileges:
                - auto_configure
                - create_doc
        _elastic_agent_checks:
          cluster:
            - monitor
        5f0b6da2-41e0-46c9-b0f0-0eed5c53b251:
          indices:
            - names:
                - logs-system.auth-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - logs-system.syslog-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - logs-system.application-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - logs-system.security-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - logs-system.system-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-system.cpu-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-system.diskio-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-system.filesystem-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-system.fsstat-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-system.load-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-system.memory-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-system.network-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-system.process-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-system.process.summary-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-system.socket_summary-default
              privileges:
                - auto_configure
                - create_doc
            - names:
                - metrics-system.uptime-default
              privileges:
                - auto_configure
                - create_doc
        e36d91c7-56fd-4fd5-a33d-0fca2f61e0c1:
          indices:
            - names:
                - logs-atlassian_jira.audit-default
              privileges:
                - auto_configure
                - create_doc
    agent:
      download:
        source_uri: 'https://artifacts.elastic.co/downloads/'
      monitoring:
        enabled: true
        use_output: default
        namespace: default
        logs: true
        metrics: true
    inputs:
      - id: logfile-system-5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
        name: system-1
        revision: 1
        type: logfile
        use_output: default
        meta:
          package:
            name: system
            version: 1.20.4
        data_stream:
          namespace: default
        package_policy_id: 5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
        streams:
          - id: logfile-system.auth-5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
            data_stream:
              dataset: system.auth
              type: logs
            ignore_older: 72h
            paths:
              - /var/log/auth.log*
              - /var/log/secure*
            exclude_files:
              - .gz$
            multiline:
              pattern: ^\s
              match: after
            tags:
              - system-auth
            processors:
              - add_locale: null
          - id: logfile-system.syslog-5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
            data_stream:
              dataset: system.syslog
              type: logs
            paths:
              - /var/log/messages*
              - /var/log/syslog*
            exclude_files:
              - .gz$
            multiline:
              pattern: ^\s
              match: after
            processors:
              - add_locale: null
            ignore_older: 72h
      - id: winlog-system-5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
        name: system-1
        revision: 1
        type: winlog
        use_output: default
        meta:
          package:
            name: system
            version: 1.20.4
        data_stream:
          namespace: default
        package_policy_id: 5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
        streams:
          - id: winlog-system.application-5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
            name: Application
            data_stream:
              dataset: system.application
              type: logs
            condition: '${host.platform} == ''windows'''
            ignore_older: 72h
          - id: winlog-system.security-5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
            name: Security
            data_stream:
              dataset: system.security
              type: logs
            condition: '${host.platform} == ''windows'''
            ignore_older: 72h
          - id: winlog-system.system-5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
            name: System
            data_stream:
              dataset: system.system
              type: logs
            condition: '${host.platform} == ''windows'''
            ignore_older: 72h
      - id: system/metrics-system-5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
        name: system-1
        revision: 1
        type: system/metrics
        use_output: default
        meta:
          package:
            name: system
            version: 1.20.4
        data_stream:
          namespace: default
        package_policy_id: 5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
        streams:
          - id: system/metrics-system.cpu-5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
            data_stream:
              dataset: system.cpu
              type: metrics
            metricsets:
              - cpu
            cpu.metrics:
              - percentages
              - normalized_percentages
            period: 10s
          - id: system/metrics-system.diskio-5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
            data_stream:
              dataset: system.diskio
              type: metrics
            metricsets:
              - diskio
            diskio.include_devices: null
            period: 10s
          - id: system/metrics-system.filesystem-5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
            data_stream:
              dataset: system.filesystem
              type: metrics
            metricsets:
              - filesystem
            period: 1m
            processors:
              - drop_event.when.regexp:
                  system.filesystem.mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)
          - id: system/metrics-system.fsstat-5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
            data_stream:
              dataset: system.fsstat
              type: metrics
            metricsets:
              - fsstat
            period: 1m
            processors:
              - drop_event.when.regexp:
                  system.fsstat.mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)
          - id: system/metrics-system.load-5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
            data_stream:
              dataset: system.load
              type: metrics
            metricsets:
              - load
            condition: '${host.platform} != ''windows'''
            period: 10s
          - id: system/metrics-system.memory-5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
            data_stream:
              dataset: system.memory
              type: metrics
            metricsets:
              - memory
            period: 10s
          - id: system/metrics-system.network-5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
            data_stream:
              dataset: system.network
              type: metrics
            metricsets:
              - network
            period: 10s
            network.interfaces: null
          - id: system/metrics-system.process-5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
            data_stream:
              dataset: system.process
              type: metrics
            metricsets:
              - process
            period: 10s
            process.include_top_n.by_cpu: 5
            process.include_top_n.by_memory: 5
            process.cmdline.cache.enabled: true
            process.cgroups.enabled: false
            process.include_cpu_ticks: false
            processes:
              - .*
          - id: >-
              system/metrics-system.process.summary-5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
            data_stream:
              dataset: system.process.summary
              type: metrics
            metricsets:
              - process_summary
            period: 10s
          - id: >-
              system/metrics-system.socket_summary-5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
            data_stream:
              dataset: system.socket_summary
              type: metrics
            metricsets:
              - socket_summary
            period: 10s
          - id: system/metrics-system.uptime-5f0b6da2-41e0-46c9-b0f0-0eed5c53b251
            data_stream:
              dataset: system.uptime
              type: metrics
            metricsets:
              - uptime
            period: 10s
      - id: httpjson-audit-e36d91c7-56fd-4fd5-a33d-0fca2f61e0c1
        name: atlassian_jira-logs
        revision: 1
        type: httpjson
        use_output: default
        meta:
          package:
            name: atlassian_jira
            version: 1.6.1
        data_stream:
          namespace: default
        package_policy_id: e36d91c7-56fd-4fd5-a33d-0fca2f61e0c1
        streams:
          - id: httpjson-atlassian_jira.audit-e36d91c7-56fd-4fd5-a33d-0fca2f61e0c1
            data_stream:
              dataset: atlassian_jira.audit
              type: logs
            config_version: '2'
            interval: 2m
            request.method: GET
            request.url: 'https://****.atlassian.net/rest/api/3/auditing/record'
            request.timeout: 60s
            auth.basic.user: ******
            auth.basic.password: ********
            request.transforms:
              - set:
                  target: url.params.limit
                  value: 1000
              - set:
                  target: url.params.from
                  value: '[[.cursor.last_timestamp]]'
                  default: >-
                    [[formatDate (now (parseDuration "-3m"))
                    "2006-01-02T15:04:05.999"]]
              - set:
                  target: url.params.to
                  value: '[[formatDate (now) "2006-01-02T15:04:05.999"]]'
              - set:
                  target: url.params.offset
                  value: '0'
            response.split:
              target: body.records
              ignore_empty_value: true
            response.pagination:
              - set:
                  target: url.value
                  value: >-
                    [[sprintf
                    "%s/rest/api/3/auditing/record?from=%s&to=%s&offset=%d&limit=%s"
                    "https://****.atlassian.net" (.last_response.url.params.Get
                    "from") (.last_response.url.params.Get "to") (add (toInt
                    .last_response.body.offset) (toInt "1000")) "1000"]]
                  fail_on_template_error: true
            cursor:
              last_timestamp:
                value: '[[.first_event.created]]'
            tags:
              - forwarded
              - jira-audit
            publisher_pipeline.disable_host: true
            fields_under_root: true
            fields:
              _config:
                atlassian_cloud: true
---
# For more information refer https://www.elastic.co/guide/en/fleet/current/running-on-kubernetes-standalone.html
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: elastic-agent-standalone
  namespace: elk
  labels:
    app: elastic-agent-standalone
spec:
  selector:
    matchLabels:
      app: elastic-agent-standalone
  template:
    metadata:
      labels:
        app: elastic-agent-standalone
    spec:
      # Tolerations are needed to run Elastic Agent on Kubernetes control-plane nodes.
      # Agents running on control-plane nodes collect metrics from the control plane components (scheduler, controller manager) of Kubernetes
      tolerations:
        - key: node-role.kubernetes.io/control-plane
          effect: NoSchedule
        - key: node-role.kubernetes.io/master
          effect: NoSchedule
      serviceAccountName: elastic-agent-standalone
      hostNetwork: true
      dnsPolicy: ClusterFirstWithHostNet
      # Uncomment if using hints feature
      #initContainers:
      #  - name: k8s-templates-downloader
      #    image: busybox:1.28
      #    command: ['sh']
      #    args:
      #      - -c
      #      - >-
      #        mkdir -p /etc/elastic-agent/inputs.d &&
      #        wget -O - https://github.com/elastic/elastic-agent/archive/8.5.tar.gz | tar xz -C /etc/elastic-agent/inputs.d --strip=5 "elastic-agent-main/deploy/kubernetes/elastic-agent-standalone/templates.d"
      #    volumeMounts:
      #      - name: external-inputs
      #        mountPath: /etc/elastic-agent/inputs.d
      containers:
        - name: elastic-agent-standalone
          image: docker.elastic.co/beats/elastic-agent:8.5.3
          args: [
            "-c", "/etc/elastic-agent/agent.yml",
            "-e",
          ]
          env:
            # The basic authentication username used to connect to Elasticsearch
            # This user needs the privileges required to publish events to Elasticsearch.
            - name: ES_USERNAME
              value: "*****"
            # The basic authentication password used to connect to Elasticsearch
            - name: ES_PASSWORD
              value: "*****"
            # The Elasticsearch host to communicate with
            - name: ES_HOST
              value: "******"
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: STATE_PATH
              value: "/etc/elastic-agent"
          securityContext:
            runAsUser: 0
          resources:
            limits:
              memory: 700Mi
            requests:
              cpu: 100m
              memory: 400Mi
          volumeMounts:
            - name: datastreams
              mountPath: /etc/elastic-agent/agent.yml
              readOnly: true
              subPath: agent.yml
            # Uncomment if using hints feature
            #- name: external-inputs
            #  mountPath: /etc/elastic-agent/inputs.d
            - name: proc
              mountPath: /hostfs/proc
              readOnly: true
            - name: cgroup
              mountPath: /hostfs/sys/fs/cgroup
              readOnly: true
            - name: varlibdockercontainers
              mountPath: /var/lib/docker/containers
              readOnly: true
            - name: varlog
              mountPath: /var/log
              readOnly: true
            - name: etc-full
              mountPath: /hostfs/etc
              readOnly: true
            - name: var-lib
              mountPath: /hostfs/var/lib
              readOnly: true
      volumes:
        - name: datastreams
          configMap:
            defaultMode: 0640
            name: agent-node-datastreams
        # Uncomment if using hints feature
        #- name: external-inputs
        #  emptyDir: {}
        - name: proc
          hostPath:
            path: /proc
        - name: cgroup
          hostPath:
            path: /sys/fs/cgroup
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers
        - name: varlog
          hostPath:
            path: /var/log
        # The following volumes are needed for Cloud Security Posture integration (cloudbeat)
        # If you are not using this integration, then these volumes and the corresponding
        # mounts can be removed.
        - name: etc-full
          hostPath:
            path: /etc
        - name: var-lib
          hostPath:
            path: /var/lib
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: elastic-agent-standalone
subjects:
  - kind: ServiceAccount
    name: elastic-agent-standalone
    namespace: elk
roleRef:
  kind: ClusterRole
  name: elastic-agent-standalone
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  namespace: elk
  name: elastic-agent-standalone
subjects:
  - kind: ServiceAccount
    name: elastic-agent-standalone
    namespace: elk
roleRef:
  kind: Role
  name: elastic-agent-standalone
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: elastic-agent-standalone-kubeadm-config
  namespace: elk
subjects:
  - kind: ServiceAccount
    name: elastic-agent-standalone
    namespace: elk
roleRef:
  kind: Role
  name: elastic-agent-standalone-kubeadm-config
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: elastic-agent-standalone
  labels:
    k8s-app: elastic-agent-standalone
rules:
  - apiGroups: [""]
    resources:
      - nodes
      - namespaces
      - events
      - pods
      - services
      - configmaps
      # Needed for cloudbeat
      - serviceaccounts
      - persistentvolumes
      - persistentvolumeclaims
    verbs: ["get", "list", "watch"]
  # Enable this rule only if planing to use kubernetes_secrets provider
  #- apiGroups: [""]
  #  resources:
  #  - secrets
  #  verbs: ["get"]
  - apiGroups: ["extensions"]
    resources:
      - replicasets
    verbs: ["get", "list", "watch"]
  - apiGroups: ["apps"]
    resources:
      - statefulsets
      - deployments
      - replicasets
      - daemonsets
    verbs: ["get", "list", "watch"]
  - apiGroups: ["batch"]
    resources:
      - jobs
      - cronjobs
    verbs: ["get", "list", "watch"]
  - apiGroups:
      - ""
    resources:
      - nodes/stats
    verbs:
      - get
  # Needed for apiserver
  - nonResourceURLs:
      - "/metrics"
    verbs:
      - get
  # Needed for cloudbeat
  - apiGroups: ["rbac.authorization.k8s.io"]
    resources:
      - clusterrolebindings
      - clusterroles
      - rolebindings
      - roles
    verbs: ["get", "list", "watch"]
  # Needed for cloudbeat
  - apiGroups: ["policy"]
    resources:
      - podsecuritypolicies
    verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: elastic-agent-standalone
  # Should be the namespace where elastic-agent is running
  namespace: elk
  labels:
    k8s-app: elastic-agent-standalone
rules:
  - apiGroups:
      - coordination.k8s.io
    resources:
      - leases
    verbs: ["get", "create", "update"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: elastic-agent-standalone-kubeadm-config
  namespace: elk
  labels:
    k8s-app: elastic-agent-standalone
rules:
  - apiGroups: [""]
    resources:
      - configmaps
    resourceNames:
      - kubeadm-config
    verbs: ["get"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: elastic-agent-standalone
  namespace: elk
  labels:
    k8s-app: elastic-agent-standalone
---

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[botelastic[bot]]

Hi! We just realized that we haven't looked into this issue in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!