OpenCTI-Platform / opencti

Open Cyber Threat Intelligence Platform
https://opencti.io
Other
6.4k stars 946 forks source link

RSS Feed is not fetching details for Red Packet Security #8968

Open Mr-AnyThink opened 4 days ago

Mr-AnyThink commented 4 days ago

Description

RSS Feed is not fetching details for Red Packet Security. I suspect it is due Cloudflare, but I can access it from system

Environment

  1. OS => Ubuntu 22.04.5 LTS
  2. OpenCTI version: 6.3.10

Reproducible Steps

Configure RSS Feed https://www.redpacketsecurity.com/feed/

nino-filigran commented 4 days ago

Potentially similar to: https://github.com/OpenCTI-Platform/opencti/issues/8736

Mr-AnyThink commented 4 days ago

@nino-filigran , I check for daily darkweb from given issue and it is working from me. I noticed there is no cloudflare verification for the feed https://dailydarkweb.net/feed/

Mr-AnyThink commented 4 days ago

Security Week is not working. Is it possible to share details where I can see logs so that I can provide it for more insights?

Mr-AnyThink commented 4 days ago

I used "curl -I https://www.redpacketsecurity.com/feed/" and below is the output,

HTTP/2 403 date: Tue, 12 Nov 2024 09:04:10 GMT content-type: text/html; charset=UTF-8 accept-ch: Sec-CH-UA-Bitness, Sec-CH-UA-Arch, Sec-CH-UA-Full-Version, Sec-CH-UA-Mobile, Sec-CH-UA-Model, Sec-CH-UA-Platform-Version, Sec-CH-UA-Full-Version-List, Sec-CH-UA-Platform, Sec-CH-UA, UA-Bitness, UA-Arch, UA-Full-Version, UA-Mobile, UA-Model, UA-Platform-Version, UA-Platform, UA critical-ch: Sec-CH-UA-Bitness, Sec-CH-UA-Arch, Sec-CH-UA-Full-Version, Sec-CH-UA-Mobile, Sec-CH-UA-Model, Sec-CH-UA-Platform-Version, Sec-CH-UA-Full-Version-List, Sec-CH-UA-Platform, Sec-CH-UA, UA-Bitness, UA-Arch, UA-Full-Version, UA-Mobile, UA-Model, UA-Platform-Version, UA-Platform, UA cross-origin-embedder-policy: require-corp cross-origin-opener-policy: same-origin cross-origin-resource-policy: same-origin origin-agent-cluster: ?1 permissions-policy: accelerometer=(),autoplay=(),browsing-topics=(),camera=(),clipboard-read=(),clipboard-write=(),geolocation=(),gyroscope=(),hid=(),interest-cohort=(),magnetometer=(),microphone=(),payment=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),sync-xhr=(),usb=() x-content-options: nosniff cf-mitigated: challenge cf-chl-out: a+Vvb8fdTwlWLxeabAJRyw/3eFEOXBu3SBjGc/b9SC8pEXQZR7AuuuJjd1ayjbiCtUNWd7+N6eD67fOOT601TZRmTD8jGILMulUfNhJi0EvTDvtQOsaS7Lf8j7ugIKqIO5ARqyPrMoRb5c7OXAkRCA==$J/KoPynY0ZSgJCYqR8vw7g== cache-control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 expires: Thu, 01 Jan 1970 00:00:01 GMT report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=2qcJsX7nfRpxKd3N%2FmvCcf0iqUymhd8WLV7RiJ8MmwvZ7TTyAsQXXnBBHyCQ3JiafsjT4NHftEyftYNofYwXfODHqKaMWvbMBzK5jjn4%2B67ShNizDCl0IkueIm3%2BZMVqepPUjTC902UhxK0%3D"}],"group":"cf-nel","max_age":604800} nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800} expect-ct: max-age=86400, enforce referrer-policy: same-origin x-content-type-options: nosniff x-frame-options: SAMEORIGIN x-xss-protection: 1; mode=block server: cloudflare cf-ray: 8e155a6079f83aec-BOM alt-svc: h3=":443"; ma=86400 server-timing: cfL4;desc="?proto=TCP&rtt=7910&sent=5&recv=6&lost=0&retrans=0&sent_bytes=3453&recv_bytes=838&delivery_rate=337315&cwnd=247&unsent_bytes=0&cid=4933a8e7f547be5b&ts=36&x=0"

The 403 response and the "cf-mitigated: challenge" header indicate that Cloudflare is blocking requests to the RSS feed. This challenge is likely being triggered by Cloudflare’s security settings on Red Packet Security’s website, which could be due to traffic coming from automated tools or unfamiliar IP addresses.

romain-filigran commented 3 days ago

I have not been able to reproduce the problem with redpacketsecurity. The RSS feed is correctly ingested. But I confirm that is not possible to ingest SecurityWeek Feed but I think that the problem is not related to OpenCTI as it's not also possible to ingest it through Google FeedBurner

Mr-AnyThink commented 3 days ago

Is there anything that I need to change? I have many other RSS feeds which are working fine, but for redpacketsecurity, it is causing issue. I am getting 403 error. I removed and added again but facing same issue