DIYgod / RSSHub

🧡 Everything is RSSible
https://docs.rsshub.app
MIT License
33.06k stars 7.36k forks source link

Some SoundCloud artists doesn't work #4926

Closed He3chai2 closed 2 years ago

He3chai2 commented 4 years ago

Involved route

https://docs.rsshub.app/en/multimedia.html#soundcloud-tracks

What is expected?

Get RSS feed for soundcloud tracks of specified artist

What is actually happening?

Error - Self-depoyed env:

Looks like something went wrong
Error: this route is empty, please check the original site or create an issue
    at module.exports (/app/lib/middleware/parameter.js:19:19)
    at process._tickCallback (internal/process/next_tick.js:68:7)

Error - rsshub.app

Looks like something went wrong
Error: this route is empty, please check the original site or create an issue
    at module.exports (/data/wwwroot/do02.rsshub.app/lib/middleware/parameter.js:19:19)
    at runMicrotasks ()
    at processTicksAndRejections (internal/process/task_queues.js:94:5)
    at async module.exports (/data/wwwroot/do02.rsshub.app/lib/middleware/anti-hotlink.js:33:5)
    at async module.exports (/data/wwwroot/do02.rsshub.app/lib/middleware/template.js:15:5)
    at async /data/wwwroot/do02.rsshub.app/lib/middleware/api-response-handler.js:143:9
    at async module.exports (/data/wwwroot/do02.rsshub.app/lib/middleware/api-template.js:2:5)
    at async module.exports (/data/wwwroot/do02.rsshub.app/lib/middleware/utf8.js:4:5)
    at async module.exports (/data/wwwroot/do02.rsshub.app/lib/middleware/debug.js:14:5)
    at async grant (/data/wwwroot/do02.rsshub.app/lib/middleware/access-control.js:20:13)
    at async module.exports (/data/wwwroot/do02.rsshub.app/lib/middleware/access-control.js:45:24)
    at async module.exports (/data/wwwroot/do02.rsshub.app/lib/middleware/header.js:20:5)
    at async module.exports (/data/wwwroot/do02.rsshub.app/lib/middleware/onerror.js:20:9)

Self-deployed information

Env Value
OS CentOS 7.8
Node version v10.21.0
if Docker, version 1.13.1
if Docker, container build date 6/4/2020 at 11:23 pm

Additional info (logs errors etc)

# Not Working examples: https://rsshub.app/soundcloud/tracks/upsahlmusic https://rsshub.app/soundcloud/tracks/vena-cava

# Working example: https://rsshub.app/soundcloud/tracks/halvsan

hoilc commented 4 years ago

Soundcloud doesn't return tracks of specified artist when JavaScript is disabled.

image

image

He3chai2 commented 4 years ago

I currently generate RSS feeds for 2187 SoundCloud artists with Five Filter's Feed Generator. Now I was about to migrate them to RSSHub. I see 289 RSS feeds with different problems for each:

$ curl -s "https://api.soundcloud.com/users/1207292/tracks?client_id=95f22ed54a5c297b1c41f72d713623ef" | jq '.[] | .permalink_url' "https://soundcloud.com/virtual-riot/virtual-riot-bossfight-afterparty" "https://soundcloud.com/virtual-riot/insides" "https://soundcloud.com/virtual-riot/iteration-interlude" "https://soundcloud.com/virtual-riot/goat" "https://soundcloud.com/virtual-riot/virtual-riot-hysteria" "https://soundcloud.com/virtual-riot/flutter-feat-madi" "https://soundcloud.com/virtual-riot/virtual-riot-self-checkout-demo" "https://soundcloud.com/virtual-riot/4-producers-1-sample-pn35a-virtual-riot-remix-free-download" "https://soundcloud.com/virtual-riot/zedd-clarity-sad-virtual-riot-remix-free-christmas-download" "https://soundcloud.com/virtual-riot/serum-presets-for-preset-junkies-demo-track-1"

or without client_id

https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/users/1207292



* And a lot of them didn't serve track list without JavaScript
This seems to be new behavior from SoundCloud. For example [this SC artists](https://soundcloud.com/stickybuds/tracks) uploaded a new track 8 days ago. I have this track in my feed reader (items will be cached forever, if seen once). If I now check the feed url directly, I see no items anymore.  
Maybe the API could help here too?
HenryQW commented 4 years ago

Looks like API is the way to go.

NeverBehave commented 2 years ago

@TonyRL This route has some strange behavior:

$ curl -v https://rsshub.app/soundcloud/tracks/halvsan
*   Trying 2606:4700:20::681a:d57:443...
* Connected to rsshub.app (2606:4700:20::681a:d57) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (OUT), TLS handshake, Client hello (1):
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-AES256-GCM-SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=San Francisco; O=Cloudflare, Inc.; CN=sni.cloudflaressl.com
*  start date: Jun 27 00:00:00 2021 GMT
*  expire date: Jun 26 23:59:59 2022 GMT
*  subjectAltName: host "rsshub.app" matched cert's "rsshub.app"
*  issuer: C=US; O=Cloudflare, Inc.; CN=Cloudflare Inc ECC CA-3
*  SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fcf8100bc00)
> GET /soundcloud/tracks/halvsan HTTP/2
> Host: rsshub.app
> user-agent: curl/7.79.1
> accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 256)!
< HTTP/2 302
< date: Sun, 22 May 2022 01:55:30 GMT
< location: https://google.com/404
< cache-control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
< expires: Thu, 01 Jan 1970 00:00:01 GMT
< expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=kTfuXP9sNj05MhCoQgv6Bq6Q3mZmdSCHaxDDES0x%2FxgspGGVGsNTjLrewyn6eqKBtUxxbXvMPE6x3s0CY8DpWLOuvqCv4z0X8m3eEM9SC2DA5cpcwVGTHPNVFDkQslnP4w%2BtAcvUUVg%3D"}],"group":"cf-nel","max_age":604800}
< nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< server: cloudflare
< cf-ray: 70f1ee1089487da7-LAX
<
* Connection #0 to host rsshub.app left intact

I would suggest disable this route for now.

TonyRL commented 2 years ago

I would suggest disable this route for now.

Cross checked with self-hosted from Heroku, Vercel and my home. All of them can provide a RSS feed. Looks like there’s something wrong with demo instance.

NeverBehave commented 2 years ago

I would suggest disable this route for now.

Cross checked with self-hosted from Heroku, Vercel and my home. All of them can provide a RSS feed. Looks like there’s something wrong with demo instance.

If it is working then we shall close the issue, adding a tag for future reference

TonyRL commented 2 years ago

This is caused by some weird settings on Cloudflare and has been fixed now.