OpenCTI-Platform / opencti

Open Cyber Threat Intelligence Platform
https://opencti.io
Other
6.23k stars 922 forks source link

CSV Feed displays no data #8465

Open J1mp3r opened 1 week ago

J1mp3r commented 1 week ago

Description

Hello Community, Hello Filigran Team,

I am trying to create different csv feeds for different indicators. I want to create multiple feeds for: IPv4 addresses IPv6 addresses Domains URLs and files.

However, I have noticed that when a new CSV feed is created, only indicators that were created or updated after the feed was created are added to the feed. So as soon as the feed is subsequently customised in any way, all indicators disappear from the feed again. I would expect the CSV feeds to behave similarly to the TAXII feeds. In that the CSV feed also refers to retro data that corresponds to the configurations of the CSV feed. As a restriction, I have given the feed a specific label and a score. I also realised that indicators can only be used collectively and not individually (IPv4, IPv6,... separately) for the CSV feed. We are currently solving this by only providing observables with the given label in the feed. However, the problem with observables is that they are not updated by the decay rules.

Environment

  1. OS Rocky Linux 9
  2. OpenCTI version: 6.2.18
  3. OpenCTI client: frontend
  4. Other environment details: running on docker with the official docker compose file.

Reproducible Steps

Steps to create the smallest reproducible scenario:

  1. Steps to create the smallest reproducible scenario:
  2. Create a public CSV Feed
  3. Set filter to Label Test and observable ipv4 adresses
  4. Create csv feed with 1 one colum (eg. colum: A Field: value)
  5. Udate or Create a existing observable with the Label Test.
  6. Open The CSV Feed to see the observable
  7. Update the roling Time of the CSV Feed by one minute
  8. Open the CSV Feed again

Expected Output

A CSV feed based on this filter that displays data continuously and retrogradely.

Actual Output

No Data inside the CSV Feed. New observables must be created or updated to be shown in the feed.

Screenshots (optional)

365688765-cc4ee936-e07a-4849-9fc7-f544838be3fe

nino-filigran commented 1 week ago

@J1mp3r this behavior has been built on purpose. The goal of the CSV feed (in data sharing) is to send snapshots, no history. Therefore, if you increase your rolling time to let's say an hours, you'll see all the entities matching your filters being sent. But if you click on your csv feed after let's say 1h30, the previous data would not be shown, because as mentionned above, we're only sending snapshots.

Why is that a problem for you? Why do you need the history?

J1mp3r commented 1 week ago

@nino-filigran I implemented a TAXII feed a few months ago, which I filled with IOCs. Now I wanted to feed the data into a firewall for automatic blocking, but the firewall does not accept TAXII or json. It only accepts plain text or CSV. That's why I tried it this way. I thought the CSV feed behaves the same as the TAXII feed. Is there a way to implement this function as a button or something similar so that the CSV feed behaves in the same way as the TAXII feed?