[FR]: overcome cloudflare js handshake protection

Brief description of the feature request

some feeds, guarded by cloudflare, return a 403 response AND html page with javascript and then after some check the rss. Maybe there is a way to overcome that - liferea somehow does that, but quiterss and rssguard fail to fetch such feeds

curl -v [mydealz.de/rss…hot](https://www.mydealz.de/rss/hot) [8:28:49]
* Trying 104.18.201.116:443...
* TCP_NODELAY set
* Connected to [mydealz.de](http://www.mydealz.de/) (104.18.201.116) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=[mydealz.de](http://mydealz.de/)
* start date: Sep 25 06:12:53 2024 GMT
* expire date: Dec 24 06:12:52 2024 GMT
* subjectAltName: host "[mydealz.de](http://www.mydealz.de/)" matched cert's "*.[mydealz.de](http://mydealz.de/)"
* issuer: C=US; O=Google Trust Services; CN=WE1
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0xaaab048e1e90)
> GET /rss/hot HTTP/2
> Host: [mydealz.de](http://www.mydealz.de/)
> user-agent: curl/7.68.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 403

< server: cloudflare

this is the html returned by cloudflare

<!DOCTYPE html><html lang="en-US"><head><title>Just a moment...</title><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta http-equiv="X-UA-Compatible" content="IE=Edge"><meta name="robots" content="noindex,nofollow"><meta name="viewport" content="width=device-width,initial-scale=1"><style>*{box-sizing:border-box;margin:0;padding:0}html{line-height:1.15;-webkit-text-size-adjust:100%;color:#313131;font-family:system-ui,-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Helvetica Neue,Arial,Noto Sans,sans-serif,Apple Color Emoji,Segoe UI Emoji,Segoe UI Symbol,Noto Color Emoji}body{display:flex;flex-direction:column;height:100vh;min-height:100vh}.main-content{margin:8rem auto;max-width:60rem;padding-left:1.5rem}@media (width <= 720px){.main-content{margin-top:4rem}}.h2{font-size:1.5rem;font-weight:500;line-height:2.25rem}@media (width <= 720px){.h2{font-size:1.25rem;line-height:1.5rem}}#challenge-error-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSI+PHBhdGggZmlsbD0iI0IyMEYwMyIgZD0iTTE2IDNhMTMgMTMgMCAxIDAgMTMgMTNBMTMuMDE1IDEzLjAxNSAwIDAgMCAxNiAzbTAgMjRhMTEgMTEgMCAxIDEgMTEtMTEgMTEuMDEgMTEuMDEgMCAwIDEtMTEgMTEiLz48cGF0aCBmaWxsPSIjQjIwRjAzIiBkPSJNMTcuMDM4IDE4LjYxNUgxNC44N0wxNC41NjMgOS41aDIuNzgzem0tMS4wODQgMS40MjdxLjY2IDAgMS4wNTcuMzg4LjQwNy4zODkuNDA3Ljk5NCAwIC41OTYtLjQwNy45ODQtLjM5Ny4zOS0xLjA1Ny4zODktLjY1IDAtMS4wNTYtLjM4OS0uMzk4LS4zODktLjM5OC0uOTg0IDAtLjU5Ny4zOTgtLjk4NS40MDYtLjM5NyAxLjA1Ni0uMzk3Ii8+PC9zdmc+);background-repeat:no-repeat;background-size:contain;padding-left:34px}@media (prefers-color-scheme:dark){body{background-color:#222;color:#d9d9d9}}</style><meta http-equiv="refresh" content="390"></head><body class="no-js"><div class="main-wrapper" role="main"><div class="main-content"><noscript><div class="h2"><span id="challenge-error-text">Enable JavaScript and cookies to continue</span></div></noscript></div></div><script>(function(){window._cf_chl_opt={cvId: '3',cZone: "www.mydealz.de",cType: 'managed',cRay: '8da16a55fc50dbe0',cH: 'oZw5rz_QV__1wMQ7Nt8oAvuq9cOfFPAk406m_eCcnVc-1730186555-1.2.1.1-5rj5di_K5TzPwA2VxNsyiounpkz1kfa88JzJ2B_GzMsJCVqSoNOR1zk6tNqWgToc',cUPMDTk: "\/rss\/hot?__cf_chl_tk=ChQueCchzyXcc9xld3cBm8Xu6H6vx.nAUFx8Mj9hBd0-1730186555-1.0.1.1-D2LNW1W63BFkNA2gPpnY8_puVZ3F0.n5g.jkWz4NPjE",cFPWv: 'g',cITimeS: '1730186555',cTTimeMs: '1000',cMTimeMs: '390000',cTplC: 0,cTplV: 5,cTplB: 'cf',cK: "",fa: "\/rss\/hot?__cf_chl_f_tk=ChQueCchzyXcc9xld3cBm8Xu6H6vx.nAUFx8Mj9hBd0-1730186555-1.0.1.1-D2LNW1W63BFkNA2gPpnY8_puVZ3F0.n5g.jkWz4NPjE",md: "lT9240DY9AjqWA4OVhaUEQXIMoOi1_VXWqnGHcpu.xc-1730186555-1.2.1.1-yutBacz0HMf9sn2S8l3ykhzYqoGmrSpNNC8sVvBluHZpmh.XB3fQfeAsXUQrtIUar5X3Sgddh1oysvdubYJOJL.nNk8okoRKXyvZpaqquMdxlz2pREg6ANzWTTH01xXhxVH0LI1NsWSbKxjkVVTxVikO8h2zugJCwP0EWNd9OBpcb9oyItFGDlkxjomac94z8hK_6_p9v6O1Eu3vIOm5YeRol9PpgTzPqAMwMRu5mar35P6.r7CovSlYE49wifpITKVTo.E0wKU4WHgCVqCoPg__RuC0UMiuTbmnLZD174eJr9NTWOzOjQqWh.n.zypk_cbYpaq1GwGpI1qCdVgPa.aJiuiYJkfF1sJ5SvBjP5EFvqxNv7b2Um0Xla5ps9JcgiKaqdyKawoTxjlfKfbwb1JqrkmRb5XQZC3Ln7nqCK_y9nTjpwIp_a1j3prA3WqOZCf8bctCKMC01F6A6OVn7kP03yIVog2BrXRrUmXVoR_G_Ydh6sdt93lLPlifjwK1Cn.tykzPeAUxEw2ip8g7AdCfxIWzJ0vHP77Zkh8mHVwVqXNoQ5C1LFHTQuYtxpZeKZOJGGAtyuTpqyGeSsoMKoyzQKRVmto.KVZqEI2TusRtZKoOOuuqPjzhs.5i.CpvNazzNVJ5mURelj5mLt6WZhkMNqNUN3_QpuP.RKRfNw7AScQF3mBpRi9DvMgA9I7Hnl5whvrIhBJTYnaUBnjlzeW_cu9qN3YF5k4eBAHJguENtPJmMOh67nFx0BLMTQ46twhQ9S8m7_Ae1id5Ucl3AIyyv9Y1L9.hCdoEv._vqytM9FCbdyVtdDqDE9ya0pyz2tchP66okV5HUObaTNNvggsVfcPTY4ocZ4zlRtTdt8jhiIU3YLtr.FakuUFg3Vl5um.IBmzUHxEdY_JD5dsMwwM7p7K.pvTKqCGcN0EhEKoyy7m906UxMGg33Vld5d__MH5NWLyzRQrvslm4MATk8dg.ggjjJiyPVbXyUdzXZ0w6KZjXDYMdJl_PFZD.7W.RmsJkXV0l7A5op9KRTdR5Ru2gCn9ViM5vk_X47tTJ7_A.hzXOhhSfzJHB8oFAsHns3rxkJChU3u0r7p2suZ.Gqo93EJYkM9.sIlIQEKY0VyZdIaqbNF_8W3Ln2YJE0TteSVSY0xWqfA7FD9J5AWkDvzHl6bjvc.VE1DTgEQwtrbTP1LcIwep3SpFZFLe525eMtRAT8vPbt_NwQAzih840Xd.K.W7fUzk8L.54rhem2v1BnpDL27e_y5Pg4lBsCLOkQk.7Fsa3j9Ahz3S.zpjROdRqK9k9FtayMGgboJfC_9AbVa3mua5Xip2ULjoJMfAid0fLAajjwgNXQhXageheAe8GsW7yzo48iGft2JWGB_X1UMpOaamJxbxRtQkmdryFs3GtZffZkG4.bbCzRiBjB73IqRfC8YGELiEIeT9JlCbe00PNUwrQJojcwC4dZ70NdmOcBw8DXqCb6AYh.tTbqKKIrgsLGQLOSKLsUkBHnUYaMw6cxyRz.sA5kFN8or1AQNZaFLFNZAmGhy8f.eV7YIK8JMW_q3mINJpyGacvnb4oUfo6Vt.kvdxATTAwXEHF428VfBHrvmbEE5AVRYFNspia0TNLM_vu9kWJWlVS67VRl8AIzaqjTb0xaIXcsurD7_NlDkVoYjr8K3M9ajEkYfennqv5ks8ZQi7jnoovWZUo9PpAfXw9qtadZIUmWdscBAdLT92EISzYjfkasRsXep3oC8vbT5vPjleQpUyUDnepjjXczBJ75OS1xemKN.A4zSiKbZMWbBfQpm7NPE2XmxM5p000vyr2lpwYI3Waqc2.YHt1aNCXpoufSWNwFTkEhb_RrXTeD3DlefNhrPupBGtq3SPp_rHAKRWhCt23J2q_xIe_06UIuR7o_6GjL73d_97JszhNjGqAiGxhDimDR9EH6fLYAUu1DvIfY3H3WTLAY6W2eDWNVs5JQ1Jql9SjLTiTJjtdxCQiyMMzFUMwctdXig2OCk0kRzNX3xrvyP4wiycPvoHMNgfg5jKY2cs26nW6PwBRgs27gr4zr7fNtVJrElGctOTdYzUNx55gWYE5wl0DuPjBZz.johZ91G6WLvfXkL0zsW5DtptnIVXBuQQzH0WNgHrpT2plzcwsddsgSzXnDN_B39.mluCiA0EXArvUBjJv1Kr2OSvYoo5mjg",mdrd: "BpV8SxFpqmWzX4tvh9QlYAmT2PAsHvNtD3SZ9W0Yw9c-1730186555-1.2.1.1-zWm3RvyTIUNRa7mmTFPKY_NybztNgaDbRERlWPsZ7B5Eri8yNYE7IBorwiBfd_whxGpP4r7l84JtVFSgwGZgM4lizVQ1K.Ch9ZGh8T4az5Q6c_rBYuHlbxO0qoZzROcQE3Vot6pzpXhG2EKbFcSH4SwYcWwuYJaWvIyWQPLy_1rZEoXcV4TTsyTydxMiTjKENdz23pP.HMCBqC7SF8p0E5prpQ7n8qXCAApE1HALm_sB621yhh7uz8MjLSARiVDfB_wsOdWPM3HrYmTQAFEckr.rbt2IFCfFQGbAK5VePkFCoGN37qMft53rjIwWs9Dg._lT0Xrbzwc9mYzp5McOGoxmsIgnFCocssXPhvWiJu957fdYyVRccxd5mwa0bANQSQm2V3F0M8oBttBAGgX7_S_BkjwP4MoJqCrXIMHwAELYEoUH2mhAxLlS55steGRIfB9BCqUYis9lCD3U73UnLoOMVI8ZNVTnRX940Ov21rrKiKMvqObmyMIIuWeaPy.0dL3AES9rBnk5IjQ_P8DbtpCCpY0y73ky40qYPcsTdo2JWx6X9P96hPw0r4HVI0O_vquzj_fIXhr206TAK47KTSrbyaVcABPOuZ4NcUyY1oTj5qAmBCHc6NZwdmr.fAwfujLIiLEjpKU5.bcUmm.Kf2JvhEr_9b4ipZHpQfTKnyqwuUIRdhDXwf0liTSTkMeUqp_olZ_7CF6.ITpY1EAY_Z6iIZX6hHDgzf_WfLZXock4QRAHV0zeAc0PfNizLAfUocnnElb0ZgxUmfUz9kDtN4spT9wlUMcqnH1O_Jruss3Wl0bE7KiEYMQimqnfTWrg4bwt06zsJCa0xhemBDjxuvdj4U2_fo6S6RxVPPIxNOpuSXRgHV3KYKDgloGgyiicH4Bj_FFOZCkaReSjf0aPi1dyv3RU8kxrSDCvWq2UF0Hdele0H.3XTb0cMvRfuiZ43vIZ8BxCwzFN4wk1UlxMgjxscdr2IOEwUYuCxuRvgDk8T5H94QxuALvw9ufZIJsnupf8zI.DZXwwV6gGSzsb468.XmcMDwqFnQFIbHlQxA2tiDDN8n86IOc7yAt1ruo5TF_tQ8NCaFeucQ4l6ycIGNYNZmjN7BXn1D8TBKmFE_mYDvuxB9gYmRvbobnuhUChBAAPaobKQDRxtdz3Uj46Fk_y_mjJe0H9s1QpQ_93fH1pgz3vd5B4tst76wQtedhrDG0pEDm8kyXGyggDHM4VRXgZ9zdHjqIJQvvsLdCmhGkUXIzyf9S.mJ5LH5KrTLqjqtvGXwiN00Z1Qw1j4fwfAVWolAYPDFL9wCOagjGZibw75Q1t5jD9Ox05G_t4w4jdSjqBqvnKth8albcuAyjTtkzZC4kU4dQ9W8xT2E68gHiskfyaSjvH8rLKQgzK3uuD3yOdJ4KjZVQ4et0Sqb.BNDmh5cQfCXFVJzJRFdpg60RtGaZi0a6QF9bnrgN5QNmPXIZQrBXIRuBH3JOM7baQKQiN0FUXRUSMLAOWthdfY7zN46fpLef73CtRZqU4DWvbz3Nh0a1q155WS_YhwuJ_W8iYKIlzYlUkZgkvj1SzDLb.44YLpy811EPYxnkC8zCivBEkSBDxEio9kYFOCuX4V8FhJZDSmICLfqSGzkby6YnPnm3OOWxNidaj9YvdTrwwTRNqsX3T02K216YW_FBb9rn6fnQdYWpy0jzazLZUX9mRvGlCzd3ycfGRbXe2vP6N8xIwLQmXqE682RQkQoVvRJHLiqLTDYWJ7sSpY.pFDnoFVkOF_Zec_teKwM8y_yov_Yxx_7DhZNcDM3N_Qu5c8sZjxWyLX1tjGyrSOSrC_OJyUtP6.UC5DA2EEDETl_y2DGmn3rrGfIJwfGiKUTU74dhUxjMh7V32_FkkBNDJLcvVxE_QzqkZdeK4bWDNyA0iQQwyXRAfZbqBUIMgQVkNlhY5UnMs6mvKGEN3rUHM85h46.UsEEVx.K8_v._QWv0JD8ClW_TIHIyqh0AJ4Zn_3lL_aXGeajg5R80aJsAuS05p2WYNZa9ASZBL9gzbTRsLYUKZahvBDbHcq4WqVnSOJ.4ee2xeLs5z7d6AHSxpkjvBKFRD8AYlorxTE0PJI_P.NbmPcEs.brfjMa8bEJU117D5F.yY.aWIhrmJxXmIamxQ7oMMm26_oUg61TEynX44TCO7E97Of3EkFaBDFp26h.IlrCXsMMg0PEHgaFGRpL6Bji4e8axsKWzh99Wn94i7cxys.q9RHpVOCsx4y8KTsglYcOtpfN0CxvknX4jYues"};var cpo = document.createElement('script');cpo.src = '/cdn-cgi/challenge-platform/h/g/orchestrate/chl_page/v1?ray=8da16a55fc50dbe0';window._cf_chl_opt.cOgUHash = location.hash === '' && location.href.indexOf('#') !== -1 ? '#' : location.hash;window._cf_chl_opt.cOgUQuery = location.search === '' && location.href.slice(0, location.href.length - window._cf_chl_opt.cOgUHash.length).indexOf('?') !== -1 ? '?' : location.search;if (window.history && window.history.replaceState) {var ogU = location.pathname + window._cf_chl_opt.cOgUQuery + window._cf_chl_opt.cOgUHash;history.replaceState(null, null, "\/rss\/hot?__cf_chl_rt_tk=ChQueCchzyXcc9xld3cBm8Xu6H6vx.nAUFx8Mj9hBd0-1730186555-1.0.1.1-D2LNW1W63BFkNA2gPpnY8_puVZ3F0.n5g.jkWz4NPjE" + window._cf_chl_opt.cOgUHash);cpo.onload = function() {history.replaceState(null, null, ogU);}}document.getElementsByTagName('head')[0].appendChild(cpo);}());</script></body></html>

martinrotter / rssguard

[FR]: overcome cloudflare js handshake protection #1527

Brief description of the feature request