search

OpenCTI-Platform / client-python

OpenCTI Python Client
https://www.opencti.io
Apache License 2.0
113 stars 128 forks source link

Missing reference to handle creation #175

Closed mwatermolen closed 3 years ago

mwatermolen commented 3 years ago

Missing reference to handle creation [Loglevel: ERROR]

Receiving only the error message Missing reference to handle creation.

Please ref https://github.com/OpenCTI-Platform/opencti/issues/1087 THIS IS NOT A DISK SPACE ISSUE image

Environment

  1. OS (where OpenCTI server runs): Ubuntu -> Kubernetes(RKE) -> Docker
  2. OpenCTI version: opencti/platform:4.5.2
  3. pycti: 4.5.3
  4. Other environment details:
    • Running In kubernetes -RKE
    • WebUI works and all appears fuctional
    • Objects CAN be added via web UI

Reproducible Steps

Steps to create the smallest reproducible scenario:

  1. Edit Kubernetes deployment below and replace <REDACTED> fields
  2. execute stix_cyber_observable.create
  3. Use a file payload or IPV4
  4. Receive log message [ERROR]Missing reference to handle creation
  5. Verify no object created in platform

Expected Output

Expected an Observable to be created

Actual Output

Only an error level message Missing reference to handle creation.

There are no failed API calls

image

Posted data

image

Console Output

image

Additional information

Code responsible

image

IOC Code

    def addIOC(self, name, IOC, Type, org, source=[], Description="", CreateIndicator=False, data="", TLP=None):
        # Define an OpenCTI compatible date
        date = datetime.now().strftime("%Y-%m-%dT%H:%M:%SZ")

        # Get STIX Pattern
        STIXPattern = self.getStixPattern(IOC, Type)

        # Get labels to tag
        labels = self.getlabels(source)

        IOCx = None
        # Generate Stix Obserable Payload
        observableData = self.getStixObject(IOC,Type,date,data)

        # Use the client to create an observable in OpenCTI
        sentry_sdk.set_context("stix_cyber_observable",
            {
                "observableData":json.dumps(observableData),
                "createdBy": self.REACTOR, 
                "objectMarking": [TLP],
                "objectLabel": labels
            }
        )
        IOCx = self.API.stix_cyber_observable.create(
                observableData=observableData,
                update=True,
                objectMarking=[TLP],
                objectLabel=labels,
                createdBy=self.REACTOR,
        )

Stix Object creation

def getStixObject(self, IOC, Type, date, data=None):
        if Type.lower() == "sha256":
            observableData = {
                "type": "file",
                "atime": date,
                "ctime": date,
                "hashes":{
                    "SHA-256": IOC
                }
            }
       return observableData
Example STIX Object

image

Opencti Kubernetes yaml

apiVersion: v1
items:
  - apiVersion: v1
    kind: Service
    metadata:
      labels:
        io.kompose.service: minio
      name: minio
    spec:
      ports:
        - name: "9000"
          port: 9000
          targetPort: 9000
      selector:
        io.kompose.service: minio
    status:
      loadBalancer: {}
  - apiVersion: v1
    kind: Service
    metadata:

      labels:
        io.kompose.service: redis
      name: redis
    spec:
      ports:
        - name: "6379"
          port: 6379
          targetPort: 6379
      selector:
        io.kompose.service: redis
    status:
      loadBalancer: {}
  - apiVersion: v1
    kind: Service
    metadata:

      labels:
        io.kompose.service: rabbitmq
      name: rabbitmq
    spec:
      ports:
        - name: "5672"
          port: 5672
          targetPort: 5672
      selector:
        io.kompose.service: rabbitmq
    status:
      loadBalancer: {}
  - apiVersion: v1
    kind: Service
    metadata:

      labels:
        io.kompose.service: opencti
      name: opencti
    spec:
      ports:
        - name: "8080"
          port: 8080
          targetPort: 8080
      selector:
        io.kompose.service: opencti
    status:
      loadBalancer: {}
  - apiVersion: apps/v1
    kind: Deployment
    metadata:

      labels:
        io.kompose.service: connector-export-file-csv
      name: connector-export-file-csv
    spec:
      replicas: 1
      selector:
        matchLabels:
          io.kompose.service: connector-export-file-csv
      strategy: {}
      template:
        metadata:

          labels:
            io.kompose.service: connector-export-file-csv
        spec:
          containers:
            - env:
                - name: CONNECTOR_CONFIDENCE_LEVEL
                  value: "15"
                - name: CONNECTOR_ID
                  value: 1071cae3-8497-4368-9258-40f5ec776344
                - name: CONNECTOR_LOG_LEVEL
                  value: info
                - name: CONNECTOR_NAME
                  value: ExportFileCsv
                - name: CONNECTOR_SCOPE
                  value: text/csv
                - name: CONNECTOR_TYPE
                  value: INTERNAL_EXPORT_FILE
                - name: OPENCTI_TOKEN
                  value: <REDACTED>
                - name: OPENCTI_URL
                  value: http://opencti:8080
              image: opencti/connector-export-file-csv:4.5.0
              name: connector-export-file-csv
              resources: {}
          restartPolicy: Always
    status: {}
  - apiVersion: apps/v1
    kind: Deployment
    metadata:

      labels:
        io.kompose.service: connector-export-file-stix
      name: connector-export-file-stix
    spec:
      replicas: 1
      selector:
        matchLabels:
          io.kompose.service: connector-export-file-stix
      strategy: {}
      template:
        metadata:

          labels:
            io.kompose.service: connector-export-file-stix
        spec:
          containers:
            - env:
                - name: CONNECTOR_CONFIDENCE_LEVEL
                  value: "15"
                - name: CONNECTOR_ID
                  value: 124ef481-d883-4542-86c8-40bd7863b074
                - name: CONNECTOR_LOG_LEVEL
                  value: info
                - name: CONNECTOR_NAME
                  value: ExportFileStix2
                - name: CONNECTOR_SCOPE
                  value: application/json
                - name: CONNECTOR_TYPE
                  value: INTERNAL_EXPORT_FILE
                - name: OPENCTI_TOKEN
                  value: <REDACTED>
                - name: OPENCTI_URL
                  value: http://opencti:8080
              image: opencti/connector-export-file-stix:4.5.0
              name: connector-export-file-stix
              resources: {}
          restartPolicy: Always
    status: {}
  - apiVersion: apps/v1
    kind: Deployment
    metadata:

      labels:
        io.kompose.service: connector-history
      name: connector-history
    spec:
      replicas: 1
      selector:
        matchLabels:
          io.kompose.service: connector-history
      strategy: {}
      template:
        metadata:

          labels:
            io.kompose.service: connector-history
        spec:
          containers:
            - env:
                - name: CONNECTOR_CONFIDENCE_LEVEL
                  value: "15"
                - name: CONNECTOR_ID
                  value: ef0a8a82-67a0-4def-a007-98e2431e4544
                - name: CONNECTOR_LOG_LEVEL
                  value: info
                - name: CONNECTOR_NAME
                  value: History
                - name: CONNECTOR_SCOPE
                  value: history
                - name: CONNECTOR_TYPE
                  value: STREAM
                - name: OPENCTI_TOKEN
                  value: <REDACTED>
                - name: OPENCTI_URL
                  value: http://opencti:8080
              image: opencti/connector-history:4.5.0
              name: connector-history
              resources: {}
          restartPolicy: Always
    status: {}
  - apiVersion: apps/v1
    kind: Deployment
    metadata:

      labels:
        io.kompose.service: connector-import-file-pdf-observables
      name: connector-import-file-pdf-observables
    spec:
      replicas: 1
      selector:
        matchLabels:
          io.kompose.service: connector-import-file-pdf-observables
      strategy: {}
      template:
        metadata:

          labels:
            io.kompose.service: connector-import-file-pdf-observables
        spec:
          containers:
            - env:
                - name: CONNECTOR_AUTO
                  value: "false"
                - name: CONNECTOR_CONFIDENCE_LEVEL
                  value: "15"
                - name: CONNECTOR_ID
                  value: 8915a83d-6127-499a-984f-2c3c297a31ec
                - name: CONNECTOR_LOG_LEVEL
                  value: info
                - name: CONNECTOR_NAME
                  value: ImportFilePdfObservables
                - name: CONNECTOR_ONLY_CONTEXTUAL
                  value: "true"
                - name: CONNECTOR_SCOPE
                  value: application/pdf
                - name: CONNECTOR_TYPE
                  value: INTERNAL_IMPORT_FILE
                - name: OPENCTI_TOKEN
                  value: <REDACTED>
                - name: OPENCTI_URL
                  value: http://opencti:8080
                - name: PDF_OBSERVABLES_CREATE_INDICATOR
                  value: "False"
              image: opencti/connector-import-file-pdf-observables:4.5.0
              name: connector-import-file-pdf-observables
              resources: {}
          restartPolicy: Always
    status: {}
  - apiVersion: apps/v1
    kind: Deployment
    metadata:

      labels:
        io.kompose.service: connector-import-file-stix
      name: connector-import-file-stix
    spec:
      replicas: 1
      selector:
        matchLabels:
          io.kompose.service: connector-import-file-stix
      strategy: {}
      template:
        metadata:

          labels:
            io.kompose.service: connector-import-file-stix
        spec:
          containers:
            - env:
                - name: CONNECTOR_AUTO
                  value: "false"
                - name: CONNECTOR_CONFIDENCE_LEVEL
                  value: "15"
                - name: CONNECTOR_ID
                  value: 25d40f5d-082d-4cfe-9a18-40892987e286
                - name: CONNECTOR_LOG_LEVEL
                  value: info
                - name: CONNECTOR_NAME
                  value: ImportFileStix
                - name: CONNECTOR_SCOPE
                  value: application/json,text/xml
                - name: CONNECTOR_TYPE
                  value: INTERNAL_IMPORT_FILE
                - name: OPENCTI_TOKEN
                  value: <REDACTED>
                - name: OPENCTI_URL
                  value: http://opencti:8080
              image: opencti/connector-import-file-stix:4.5.0
              name: connector-import-file-stix
              resources: {}
          restartPolicy: Always
    status: {}
  - apiVersion: apps/v1
    kind: Deployment
    metadata:

      labels:
        io.kompose.service: minio
      name: minio
    spec:
      replicas: 1
      selector:
        matchLabels:
          io.kompose.service: minio
      strategy:
        type: Recreate
      template:
        metadata:

          labels:
            io.kompose.service: minio
        spec:
          containers:
            - args:
                - server
                - /data
              env:
                - name: MINIO_ACCESS_KEY
                  value: <REDACTED>
                - name: MINIO_SECRET_KEY
                  value: <REDACTED>
              image: minio/minio:RELEASE.2021-04-22T15-44-28Z
              livenessProbe:
                exec:
                  command:
                    - curl
                    - -f
                    - http://localhost:9000/minio/health/live
                failureThreshold: 3
                periodSeconds: 30
                timeoutSeconds: 20
              name: minio
              ports:
                - containerPort: 9000
              resources: {}
              volumeMounts:
                - mountPath: /data
                  name: s3data
          restartPolicy: Always
          volumes:
            - name: s3data
              persistentVolumeClaim:
                claimName: s3data
    status: {}
  - apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:

      labels:
        io.kompose.service: s3data
      name: s3data
    spec:
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 100Mi
    status: {}
  - apiVersion: apps/v1
    kind: Deployment
    metadata:

      labels:
        io.kompose.service: opencti
      name: opencti
    spec:
      replicas: 1
      selector:
        matchLabels:
          io.kompose.service: opencti
      strategy: {}
      template:
        metadata:

          labels:
            io.kompose.service: opencti
        spec:
          containers:
            - env:
                - name: APP__ADMIN__EMAIL
                  value: securityOps@infosec-ops.com
                - name: APP__ADMIN__PASSWORD
                  value: <REDACTED>
                - name: APP__ADMIN__TOKEN
                  value: <REDACTED>
                - name: APP__APP_LOGS__LOGS_LEVEL
                  value: info
                - name: APP__PORT
                  value: "8080"
                - name: ELASTICSEARCH__URL
                  value: http://elasticsearch.elk.svc.cluster.local:9200
                - name: MINIO__ACCESS_KEY
                  value: <REDACTED>
                - name: MINIO__ENDPOINT
                  value: minio
                - name: MINIO__PORT
                  value: "9000"
                - name: MINIO__SECRET_KEY
                  value: <REDACTED>
                - name: MINIO__USE_SSL
                  value: "false"
                - name: NODE_OPTIONS
                  value: --max-old-space-size=8096
                - name: PROVIDERS__LOCAL__STRATEGY
                  value: LocalStrategy
                - name: RABBITMQ__HOSTNAME
                  value: rabbitmq.opencti.svc.cluster.local
                - name: RABBITMQ__MANAGEMENT_SSL
                  value: "false"
                - name: RABBITMQ__PASSWORD
                  value: <REDACTED>
                - name: RABBITMQ__PORT
                  value: "5672"
                - name: RABBITMQ__PORT_MANAGEMENT
                  value: "15672"
                - name: RABBITMQ__USERNAME
                  value: opencti
                - name: REDIS__HOSTNAME
                  value: redis.opencti.svc.cluster.local
                - name: REDIS__PORT
                  value: "6379"
              image: opencti/platform:4.5.2
              name: opencti
              ports:
                - containerPort: 8080
              resources: {}
          restartPolicy: Always
    status: {}
  - apiVersion: apps/v1
    kind: Deployment
    metadata:

      labels:
        io.kompose.service: rabbitmq
      name: rabbitmq
    spec:
      replicas: 1
      selector:
        matchLabels:
          io.kompose.service: rabbitmq
      strategy:
        type: Recreate
      template:
        metadata:

          labels:
            io.kompose.service: rabbitmq
        spec:
          containers:
            - env:
                - name: RABBITMQ_DEFAULT_PASS
                  value: <REDACTED>
                - name: RABBITMQ_DEFAULT_USER
                  value: opencti
              image: rabbitmq:3.8-management
              name: rabbitmq
              resources: {}
              volumeMounts:
                - mountPath: /var/lib/rabbitmq
                  name: amqpdata
          restartPolicy: Always
          volumes:
            - name: amqpdata
              persistentVolumeClaim:
                claimName: amqpdata
    status: {}
  - apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:

      labels:
        io.kompose.service: amqpdata
      name: amqpdata
    spec:
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 800Mi
    status: {}
  - apiVersion: apps/v1
    kind: Deployment
    metadata:

      labels:
        io.kompose.service: redis
      name: redis
    spec:
      replicas: 1
      selector:
        matchLabels:
          io.kompose.service: redis
      strategy:
        type: Recreate
      template:
        metadata:

          labels:
            io.kompose.service: redis
        spec:
          containers:
            - image: redis:6.2.2
              name: redis
              resources: {}
              volumeMounts:
                - mountPath: /data
                  name: redisdata
          restartPolicy: Always
          volumes:
            - name: redisdata
              persistentVolumeClaim:
                claimName: redisdata
    status: {}
  - apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:

      labels:
        io.kompose.service: redisdata
      name: redisdata
    spec:
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 100Mi
    status: {}
  - apiVersion: apps/v1
    kind: Deployment
    metadata:

      labels:
        io.kompose.service: worker
      name: worker
    spec:
      replicas: 3
      selector:
        matchLabels:
          io.kompose.service: worker
      strategy: {}
      template:
        metadata:

          labels:
            io.kompose.service: worker
        spec:
          containers:
            - env:
                - name: OPENCTI_TOKEN
                  value: <REDACTED>
                - name: OPENCTI_URL
                  value: http://opencti:8080
                - name: WORKER_LOG_LEVEL
                  value: info
              image: opencti/worker:4.5.0
              name: worker
              resources: {}
          restartPolicy: Always
    status: {}
kind: List
metadata: {}
mwatermolen commented 3 years ago

Turns out it was the objectMarking element being null... closing 🤦