Internal Server Error on Feed

[timnolte]

I don't see any option to turn on logging or debugging to get more details. I'm trying to pull feeds from CVE and it's not working. It works just fine via wget/curl/browser.

Checking...CVE Security Vulnerability Feed (https://www.cvedetails.com/rss-feed/v1?feedId=###&check=........) Failed to fetch: https://www.cvedetails.com/rss-feed/v1?feedId=###&check=....... (returned internal server error)

[timnolte]

Wanted to supply a sample of the RSS feed in the event that it's some sort of parsing issue, though I suspect an Internal Server Error might imply that this doesn't support query parameters in RSS feeds.

Feed XML

```xml https://www.cvedetails.com Security vulnerability feeds by https://www.cvedetails.com en-us 60 The Directorist – WordPress Business Directory Plugin with Classified Ads Listings plugin for WordPress is vulnerable to unauthorized modification of data due to a missing capability check on the 'setup_wizard' function in all versions up to, and including, 7.8.4. This makes it possible for unauthenticated attackers to recreate default pages and enable or disable monetization and change map provider. (CVSS:5.3) (Last Update:2024-02-29 13:49:29) https://www.cvedetails.com/cve/CVE-2024-1322/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2024-1322/?utm_source=rssfeed Thu, 29 Feb 2024 01:43:48 +0000 Cross-Site Request Forgery (CSRF) vulnerability in Duplicator Duplicator – WordPress Migration & Backup Plugin.This issue affects Duplicator – WordPress Migration & Backup Plugin: from n/a through 1.5.7. (CVSS:6.5) (EPSS:0.04%) (Last Update:2024-02-29 13:49:47) https://www.cvedetails.com/cve/CVE-2023-51681/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2023-51681/?utm_source=rssfeed Wed, 28 Feb 2024 17:15:07 +0000 LiteSpeed OpenLiteSpeed before 1.7.18 does not strictly validate HTTP request headers. (CVSS:7.5) (EPSS:0.05%) (Last Update:2023-08-22 01:16:07) https://www.cvedetails.com/cve/CVE-2023-40518/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2023-40518/?utm_source=rssfeed Mon, 14 Aug 2023 22:15:14 +0000 Exposure of Sensitive Information to an Unauthorized Actor in WordPress from 6.3 through 6.3.1, from 6.2 through 6.2.2, from 6.1 through 6.13, from 6.0 through 6.0.5, from 5.9 through 5.9.7, from 5.8 through 5.8.7, from 5.7 through 5.7.9, from 5.6 through 5.6.11, from 5.5 through 5.5.12, from 5.4 through 5.4.13, from 5.3 through 5.3.15, from 5.2 through 5.2.18, from 5.1 through 5.1.16, from 5.0 through 5.0.19, from 4.9 through 4.9.23, from 4.8 through 4.8.22, from 4.7 through 4.7.26, from 4.6 through 4.6.26, from 4.5 through 4.5.29, from 4.4 through 4.4.30, from 4.3 through 4.3.31, from 4.2 through 4.2.35, from 4.1 through 4.1.38. (CVSS:4.3) (EPSS:0.25%) (Last Update:2024-02-16 17:16:08) https://www.cvedetails.com/cve/CVE-2023-39999/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2023-39999/?utm_source=rssfeed Fri, 13 Oct 2023 12:15:10 +0000 Auth. Stored (contributor+) Cross-Site Scripting (XSS) vulnerability in WordPress core 6.3 through 6.3.1, from 6.2 through 6.2.2, from 6.1 through 6.1.3, from 6.0 through 6.0.5, from 5.9 through 5.9.7 and Gutenberg plugin <= 16.8.0 versions. (CVSS:6.5) (EPSS:0.14%) (Last Update:2023-10-16 17:04:07) https://www.cvedetails.com/cve/CVE-2023-38000/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2023-38000/?utm_source=rssfeed Fri, 13 Oct 2023 10:15:10 +0000 WordPress through 6.1.1 depends on unpredictable client visits to cause wp-cron.php execution and the resulting security updates, and the source code describes "the scenario where a site may not receive enough visits to execute scheduled tasks in a timely manner," but neither the installation guide nor the security guide mentions this default behavior, or alerts the user about security risks on installations with very few visits. (CVSS:5.3) (EPSS:0.08%) (Last Update:2023-02-02 16:42:58) https://www.cvedetails.com/cve/CVE-2023-22622/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2023-22622/?utm_source=rssfeed Thu, 05 Jan 2023 02:15:08 +0000 WordPress does not properly restrict which user fields are searchable via the REST API, allowing unauthenticated attackers to discern the email addresses of users who have published public posts on an affected website via an Oracle style attack (CVSS:5.3) (EPSS:0.09%) (Last Update:2023-11-20 23:15:07) https://www.cvedetails.com/cve/CVE-2023-5561/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2023-5561/?utm_source=rssfeed Mon, 16 Oct 2023 20:15:18 +0000 A vulnerability was found in MariaDB. An OpenVAS port scan on ports 3306 and 4567 allows a malicious remote client to cause a denial of service. (CVSS:7.5) (EPSS:0.12%) (Last Update:2023-12-22 19:25:32) https://www.cvedetails.com/cve/CVE-2023-5157/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2023-5157/?utm_source=rssfeed Wed, 27 Sep 2023 15:19:42 +0000 The LiteSpeed Cache plugin for WordPress is vulnerable to Stored Cross-Site Scripting via the 'esi' shortcode in versions up to, and including, 5.6 due to insufficient input sanitization and output escaping on user supplied attributes. This makes it possible for authenticated attackers with contributor-level and above permissions to inject arbitrary web scripts in pages that will execute whenever a user accesses an injected page. (CVSS:6.4) (EPSS:0.05%) (Last Update:2024-01-17 21:50:15) https://www.cvedetails.com/cve/CVE-2023-4372/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2023-4372/?utm_source=rssfeed Thu, 11 Jan 2024 09:15:47 +0000 In PHP version 8.0.* before 8.0.30, 8.1.* before 8.1.22, and 8.2.* before 8.2.8, when loading phar file, while reading PHAR directory entries, insufficient length checking may lead to a stack buffer overflow, leading potentially to memory corruption or RCE. (CVSS:9.8) (EPSS:0.08%) (Last Update:2023-10-27 18:58:24) https://www.cvedetails.com/cve/CVE-2023-3824/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2023-3824/?utm_source=rssfeed Fri, 11 Aug 2023 06:15:11 +0000 In PHP versions 8.0.* before 8.0.30, 8.1.* before 8.1.22, and 8.2.* before 8.2.8 various XML functions rely on libxml global state to track configuration variables, like whether external entities are loaded. This state is assumed to be unchanged unless the user explicitly changes it by calling appropriate function. However, since the state is process-global, other modules - such as ImageMagick - may also use this library within the same process, and change that global state for their internal purposes, and leave it in a state where external entities loading is enabled. This can lead to the situation where external XML is parsed with external entities loaded, which can lead to disclosure of any local files accessible to PHP. This vulnerable state may persist in the same process across many requests, until the process is shut down. (CVSS:8.6) (EPSS:0.06%) (Last Update:2023-10-27 18:58:56) https://www.cvedetails.com/cve/CVE-2023-3823/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2023-3823/?utm_source=rssfeed Fri, 11 Aug 2023 06:15:09 +0000 In PHP versions 8.0.* before 8.0.29, 8.1.* before 8.1.20, 8.2.* before 8.2.7 when using SOAP HTTP Digest Authentication, random value generator was not checked for failure, and was using narrower range of values than it should have. In case of random generator failure, it could lead to a disclosure of 31 bits of uninitialized memory from the client to the server, and it also made easier to a malicious server to guess the client's nonce. (CVSS:4.3) (EPSS:0.05%) (Last Update:2023-08-01 16:38:09) https://www.cvedetails.com/cve/CVE-2023-3247/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2023-3247/?utm_source=rssfeed Sat, 22 Jul 2023 05:15:37 +0000 WordPress Core is vulnerable to Directory Traversal in versions up to, and including, 6.2, via the ‘wp_lang’ parameter. This allows unauthenticated attackers to access and load arbitrary translation files. In cases where an attacker is able to upload a crafted translation file onto the site, such as via an upload form, this could be also used to perform a Cross-Site Scripting attack. (CVSS:6.1) (EPSS:0.25%) (Last Update:2023-06-21 01:15:09) https://www.cvedetails.com/cve/CVE-2023-2745/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2023-2745/?utm_source=rssfeed Wed, 17 May 2023 09:15:10 +0000 In PHP 8.0.X before 8.0.28, 8.1.X before 8.1.16 and 8.2.X before 8.2.3, excessive number of parts in HTTP form upload can cause high resource consumption and excessive number of log entries. This can cause denial of service on the affected server by exhausting CPU resources or disk space. (CVSS:7.5) (EPSS:0.06%) (Last Update:2023-05-17 20:15:10) https://www.cvedetails.com/cve/CVE-2023-0662/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2023-0662/?utm_source=rssfeed Thu, 16 Feb 2023 07:15:11 +0000 In PHP 8.0.X before 8.0.28, 8.1.X before 8.1.16 and 8.2.X before 8.2.3, core path resolution function allocate buffer one byte too small. When resolving paths with lengths close to system MAXPATHLEN setting, this may lead to the byte after the allocated buffer being overwritten with NUL value, which might lead to unauthorized data access or modification. (CVSS:8.1) (EPSS:0.16%) (Last Update:2023-05-17 20:15:09) https://www.cvedetails.com/cve/CVE-2023-0568/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2023-0568/?utm_source=rssfeed Thu, 16 Feb 2023 07:15:10 +0000 In PHP 8.0.X before 8.0.28, 8.1.X before 8.1.16 and 8.2.X before 8.2.3, password_verify() function may accept some invalid Blowfish hashes as valid. If such invalid hash ever ends up in the password database, it may lead to an application allowing any password for this entry as valid. (CVSS:7.7) (EPSS:0.05%) (Last Update:2023-03-10 17:32:29) https://www.cvedetails.com/cve/CVE-2023-0567/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2023-0567/?utm_source=rssfeed Wed, 01 Mar 2023 08:15:12 +0000 MariaDB Server before 10.3.34 thru 10.9.3 is vulnerable to Denial of Service. It is possible for function spider_db_mbase::print_warnings to dereference a null pointer. (CVSS:6.5) (EPSS:0.11%) (Last Update:2023-06-16 04:15:12) https://www.cvedetails.com/cve/CVE-2022-47015/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2022-47015/?utm_source=rssfeed Fri, 20 Jan 2023 19:15:17 +0000 Cross-Site Request Forgery (CSRF) vulnerability in LiteSpeed Technologies LiteSpeed Cache plugin <= 5.3 versions. (CVSS:8.8) (EPSS:0.06%) (Last Update:2023-05-31 19:08:57) https://www.cvedetails.com/cve/CVE-2022-46800/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2022-46800/?utm_source=rssfeed Thu, 25 May 2023 09:15:11 +0000 Improper authentication vulnerability in WordPress versions prior to 6.0.3 allows a remote unauthenticated attacker to obtain the email address of the user who posted a blog using the WordPress Post by Email Feature. The developer also provides new patched releases for all versions since 3.7. (CVSS:5.3) (EPSS:0.16%) (Last Update:2023-02-03 16:58:26) https://www.cvedetails.com/cve/CVE-2022-43504/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2022-43504/?utm_source=rssfeed Mon, 05 Dec 2022 04:15:11 +0000 Cross-site scripting vulnerability in WordPress versions prior to 6.0.3 allows a remote unauthenticated attacker to inject an arbitrary script. The developer also provides new patched releases for all versions since 3.7. (CVSS:6.1) (EPSS:0.16%) (Last Update:2023-02-03 16:58:22) https://www.cvedetails.com/cve/CVE-2022-43500/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2022-43500/?utm_source=rssfeed Mon, 05 Dec 2022 04:15:11 +0000 Cross-site scripting vulnerability in WordPress versions prior to 6.0.3 allows a remote unauthenticated attacker to inject an arbitrary script. The developer also provides new patched releases for all versions since 3.7. (CVSS:6.1) (EPSS:0.16%) (Last Update:2023-02-03 16:58:19) https://www.cvedetails.com/cve/CVE-2022-43497/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2022-43497/?utm_source=rssfeed Mon, 05 Dec 2022 04:15:10 +0000 In MariaDB before 10.9.2, compress_write in extra/mariabackup/ds_compress.cc does not release data_mutex upon a stream write failure, which allows local users to trigger a deadlock. (CVSS:5.5) (EPSS:0.04%) (Last Update:2022-12-08 03:44:59) https://www.cvedetails.com/cve/CVE-2022-38791/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2022-38791/?utm_source=rssfeed Sat, 27 Aug 2022 20:15:08 +0000 The Keccak XKCP SHA-3 reference implementation before fdc6fef has an integer overflow and resultant buffer overflow that allows attackers to execute arbitrary code or eliminate expected cryptographic properties. This occurs in the sponge function interface. (CVSS:9.8) (EPSS:1.04%) (Last Update:2023-05-03 11:15:12) https://www.cvedetails.com/cve/CVE-2022-37454/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2022-37454/?utm_source=rssfeed Fri, 21 Oct 2022 06:15:09 +0000 MariaDB v10.7 was discovered to contain an use-after-poison in in __interceptor_memset at /libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc. (CVSS:7.5) (EPSS:0.22%) (Last Update:2022-12-07 18:13:29) https://www.cvedetails.com/cve/CVE-2022-32091/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2022-32091/?utm_source=rssfeed Fri, 01 Jul 2022 20:15:09 +0000 MariaDB v10.5 to v10.7 was discovered to contain a segmentation fault via the component st_select_lex_unit::exclude_level. (CVSS:7.5) (EPSS:0.27%) (Last Update:2022-12-07 17:07:42) https://www.cvedetails.com/cve/CVE-2022-32089/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2022-32089/?utm_source=rssfeed Fri, 01 Jul 2022 20:15:08 +0000 MariaDB v10.2 to v10.7 was discovered to contain a segmentation fault via the component Exec_time_tracker::get_loops/Filesort_tracker::report_use/filesort. (CVSS:7.5) (EPSS:0.10%) (Last Update:2022-10-26 22:27:06) https://www.cvedetails.com/cve/CVE-2022-32088/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2022-32088/?utm_source=rssfeed Fri, 01 Jul 2022 20:15:08 +0000 MariaDB v10.2 to v10.7 was discovered to contain a segmentation fault via the component Item_args::walk_args. (CVSS:7.5) (EPSS:0.10%) (Last Update:2022-10-26 22:26:46) https://www.cvedetails.com/cve/CVE-2022-32087/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2022-32087/?utm_source=rssfeed Fri, 01 Jul 2022 20:15:08 +0000 MariaDB v10.4 to v10.8 was discovered to contain a segmentation fault via the component Item_field::fix_outer_field. (CVSS:7.5) (EPSS:0.09%) (Last Update:2022-10-25 19:49:02) https://www.cvedetails.com/cve/CVE-2022-32086/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2022-32086/?utm_source=rssfeed Fri, 01 Jul 2022 20:15:08 +0000 MariaDB v10.2 to v10.7 was discovered to contain a segmentation fault via the component Item_func_in::cleanup/Item::cleanup_processor. (CVSS:7.5) (EPSS:0.10%) (Last Update:2022-10-26 22:26:03) https://www.cvedetails.com/cve/CVE-2022-32085/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2022-32085/?utm_source=rssfeed Fri, 01 Jul 2022 20:15:08 +0000 MariaDB v10.2 to v10.7 was discovered to contain a segmentation fault via the component sub_select. (CVSS:7.5) (EPSS:0.22%) (Last Update:2022-12-07 18:13:39) https://www.cvedetails.com/cve/CVE-2022-32084/?utm_source=rssfeed https://www.cvedetails.com/cve/CVE-2022-32084/?utm_source=rssfeed Fri, 01 Jul 2022 20:15:08 +0000 ```

[VerifiedJoseph]

The feed sample is valid. The issue is most likely with cvedetails.com.

I've improved the failed to fetch error message. it now includes the HTTP status code (e.g 404 Not Found). See https://github.com/VerifiedJoseph/vigilant/commit/112800526b7fce8074a8fb8410ff6a3c95cd8913

[timnolte]

What kind of headers are being sent when requesting feeds. Because the feed URL works perfectly fine when I use my browser or wget.

[timnolte]

Is there a way to get more debugging output logging? ~I have a feeling that the way the code us written is that it is dropping all query parameters from feed URL and that is why it's failing. Has this been tested using feeds with query parameters?~

Actually, I was doing some testing with this RSS testing feed which uses parameters and it was working.

https://lorem-rss.herokuapp.com/feed?unit=minute&interval=5

[timnolte]

If I run

wget "https://www.cvedetails.com/rss-feed/v1?feedId=###&check=........." -O rss2.xml

I get a valid result.

Resolving www.cvedetails.com (www.cvedetails.com)... 104.18.32.86, 172.64.155.170, 2606:4700:4400::6812:2056, ...
Connecting to www.cvedetails.com (www.cvedetails.com)|104.18.32.86|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/rss+xml]
Saving to: ‘rss2.xml’

[timnolte]

Doing some searching I'm thinking the issue is that Vigilant isn't actually sending a User-Agent at all perhaps. I found some notes about something like this here: https://developers.exlibrisgroup.com/forums/topic/rss-feed-returning-500-internal-server-error/#post-36468

I ran the RSS feed through the W3C Feed Validation Service and the only thing it flagged was:

This feed is valid, but interoperability with the widest range of feed readers could be improved by implementing the following recommendations. line 6, column 163: Self reference doesn't match document location

[timnolte]

Ran wget with some additional debugging output if it will help. It looks like wget does set a User Agent.

wget Debugging Output

```shell Setting --output-document (outputdocument) to rss.xml DEBUG output created by Wget 1.21.2 on linux-gnu. Reading HSTS entries from /root/.wget-hsts URI encoding = ‘UTF-8’ --2024-03-02 19:34:18-- https://www.cvedetails.com/rss-feed/v1?feedId=###&check=..... Resolving www.cvedetails.com (www.cvedetails.com)... 104.18.32.86, 172.64.155.170, 2606:4700:4400::ac40:9baa, ... Caching www.cvedetails.com => 104.18.32.86 172.64.155.170 2606:4700:4400::ac40:9baa 2606:4700:4400::6812:2056 Connecting to www.cvedetails.com (www.cvedetails.com)|104.18.32.86|:443... connected. Created socket 4. Releasing 0x0000aaaac24d5780 (new refcount 1). Initiating SSL handshake. Handshake successful; connected socket 4 to SSL handle 0x0000aaaac24d6d90 certificate: subject: CN=sni.cloudflaressl.com,O=Cloudflare\\, Inc.,L=San Francisco,ST=California,C=US issuer: CN=Cloudflare Inc ECC CA-3,O=Cloudflare\\, Inc.,C=US X509 certificate successfully verified and matches host www.cvedetails.com ---request begin--- GET /rss-feed/v1?feedId=###&check=..... HTTP/1.1 Host: www.cvedetails.com User-Agent: Wget/1.21.2 Accept: */* Accept-Encoding: identity Connection: Keep-Alive ---request end--- HTTP request sent, awaiting response... ---response begin--- HTTP/1.1 200 OK Date: Sat, 02 Mar 2024 19:34:19 GMT Content-Type: application/rss+xml; charset=utf-8 Transfer-Encoding: chunked Connection: keep-alive Vary: Accept-Encoding CF-Cache-Status: DYNAMIC Strict-Transport-Security: max-age=31536000; includeSubDomains; preload X-Content-Type-Options: nosniff Server: cloudflare CF-RAY: 85e3d2cf8cce7ffa-IAD ---response end--- 200 OK Registered socket 4 for persistent reuse. Parsed Strict-Transport-Security max-age = 31536000, includeSubDomains = true Updated HSTS host: www.cvedetails.com:443 (max-age: 31536000, includeSubdomains: true) URI content encoding = ‘utf-8’ Length: unspecified [application/rss+xml] Saving to: ‘rss.xml’ ```

[VerifiedJoseph]

https://github.com/VerifiedJoseph/vigilant/pull/109 should, hopefully, address this.

[timnolte]

@VerifiedJoseph I just pull the latest changes from main and manually ran a test and can confirm that this has fixed the issue.