Open ryparker opened 3 months ago
Related to issue #203
Good morning. This appears to be a rate limiting issue hitting the main Congress.gov website. We are investigating possible solutions. In the meantime, the GovInfo API might be of good use to you: https://api.govinfo.gov/docs/
This problem also occurs when collecting congressional daily record texts (e.g. https://www.congress.gov/118/crec/2024/03/21/170/50/modified/CREC-2024-03-21-pt1-PgD289.htm
).
Hi @ryparker - have you noticed these issues recently? Thanks!
Just ran some tests and hit it after about 7 minutes, and about 708 requests.
[congressgov-client] error: <<< Request failed for 'www.congress.gov/118/bills/sres706/BILLS-118sres706ats.xml' with error
Response: 429 (Too Many Requests)
<!DOCTYPE html><html lang="en-US"><head><title>Just a moment...</title><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta http-equiv="X-UA-Compatible" content="IE=Edge"><meta name="robots" content="noindex,nofollow"><meta name="viewport" content="width=device-width,initial-scale=1"><style>*{box-sizing:border-box;margin:0;padding:0}html{line-height:1.15;-webkit-text-size-adjust:100%;color:#313131}button,html{font-family:system-ui,-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Helvetica Neue,Arial,Noto Sans,sans-serif,Apple Color Emoji,Segoe UI Emoji,Segoe UI Symbol,Noto Color Emoji}@media (prefers-color-scheme:dark){body{background-color:#222;color:#d9d9d9}body a{color:#fff}body a:hover{color:#ee730a;text-decoration:underline}body .lds-ring div{border-color:#999 transparent transparent}body .font-red{color:#b20f03}body .pow-button{background-color:#4693ff;color:#1d1d1d}body #challenge-success-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSIgdmlld0JveD0iMCAwIDI2IDI2Ij48cGF0aCBmaWxsPSIjZDlkOWQ5IiBkPSJNMTMgMGExMyAxMyAwIDEgMCAwIDI2IDEzIDEzIDAgMCAwIDAtMjZtMCAyNGExMSAxMSAwIDEgMSAwLTIyIDExIDExIDAgMCAxIDAgMjIiLz48cGF0aCBmaWxsPSIjZDlkOWQ5IiBkPSJtMTAuOTU1IDE2LjA1NS0zLjk1LTQuMTI1LTEuNDQ1IDEuMzg1IDUuMzcgNS42MSA5LjQ5NS05LjYtMS40Mi0xLjQwNXoiLz48L3N2Zz4=)}body #challenge-error-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSI+PHBhdGggZmlsbD0iI0IyMEYwMyIgZD0iTTE2IDNhMTMgMTMgMCAxIDAgMTMgMTNBMTMuMDE1IDEzLjAxNSAwIDAgMCAxNiAzbTAgMjRhMTEgMTEgMCAxIDEgMTEtMTEgMTEuMDEgMTEuMDEgMCAwIDEtMTEgMTEiLz48cGF0aCBmaWxsPSIjQjIwRjAzIiBkPSJNMTcuMDM4IDE4LjYxNUgxNC44N0wxNC41NjMgOS41aDIuNzgzem0tMS4wODQgMS40MjdxLjY2IDAgMS4wNTcuMzg4LjQwNy4zODkuNDA3Ljk5NCAwIC41OTYtLjQwNy45ODQtLjM5Ny4zOS0xLjA1Ny4zODktLjY1IDAtMS4wNTYtLjM4OS0uMzk4LS4zODktLjM5OC0uOTg0IDAtLjU5Ny4zOTgtLjk4NS40MDYtLjM5NyAxLjA1Ni0uMzk3Ii8+PC9zdmc+)}}body{display:flex;flex-direction:column;min-height:100vh}body.no-js .loading-spinner{visibility:hidden}body.no-js .challenge-running{display:none}body.dark{background-color:#222;color:#d9d9d9}body.dark a{color:#fff}body.dark a:hover{color:#ee730a;text-decoration:underline}body.dark .lds-ring div{border-color:#999 transparent transparent}body.dark .font-red{color:#b20f03}body.dark .pow-button{background-color:#4693ff;color:#1d1d1d}body.dark #challenge-success-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSIgdmlld0JveD0iMCAwIDI2IDI2Ij48cGF0aCBmaWxsPSIjZDlkOWQ5IiBkPSJNMTMgMGExMyAxMyAwIDEgMCAwIDI2IDEzIDEzIDAgMCAwIDAtMjZtMCAyNGExMSAxMSAwIDEgMSAwLTIyIDExIDExIDAgMCAxIDAgMjIiLz48cGF0aCBmaWxsPSIjZDlkOWQ5IiBkPSJtMTAuOTU1IDE2LjA1NS0zLjk1LTQuMTI1LTEuNDQ1IDEuMzg1IDUuMzcgNS42MSA5LjQ5NS05LjYtMS40Mi0xLjQwNXoiLz48L3N2Zz4=)}body.dark #challenge-error-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSI+PHBhdGggZmlsbD0iI0IyMEYwMyIgZD0iTTE2IDNhMTMgMTMgMCAxIDAgMTMgMTNBMTMuMDE1IDEzLjAxNSAwIDAgMCAxNiAzbTAgMjRhMTEgMTEgMCAxIDEgMTEtMTEgMTEuMDEgMTEuMDEgMCAwIDEtMTEgMTEiLz48cGF0aCBmaWxsPSIjQjIwRjAzIiBkPSJNMTcuMDM4IDE4LjYxNUgxNC44N0wxNC41NjMgOS41aDIuNzgzem0tMS4wODQgMS40MjdxLjY2IDAgMS4wNTcuMzg4LjQwNy4zODkuNDA3Ljk5NCAwIC41OTYtLjQwNy45ODQtLjM5Ny4zOS0xLjA1Ny4zODktLjY1IDAtMS4wNTYtLjM4OS0uMzk4LS4zODktLjM5OC0uOTg0IDAtLjU5Ny4zOTgtLjk4NS40MDYtLjM5NyAxLjA1Ni0uMzk3Ii8+PC9zdmc+)}body.light{background-color:transparent;color:#313131}body.light a{color:#0051c3}body.light a:hover{color:#ee730a;text-decoration:underline}body.light .lds-ring div{border-color:#595959 transparent transparent}body.light .font-red{color:#fc574a}body.light .pow-button{background-color:#003681;border-color:#003681;color:#fff}body.light #challenge-success-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSIgdmlld0JveD0iMCAwIDI2IDI2Ij48cGF0aCBmaWxsPSIjMzEzMTMxIiBkPSJNMTMgMGExMyAxMyAwIDEgMCAwIDI2IDEzIDEzIDAgMCAwIDAtMjZtMCAyNGExMSAxMSAwIDEgMSAwLTIyIDExIDExIDAgMCAxIDAgMjIiLz48cGF0aCBmaWxsPSIjMzEzMTMxIiBkPSJtMTAuOTU1IDE2LjA1NS0zLjk1LTQuMTI1LTEuNDQ1IDEuMzg1IDUuMzcgNS42MSA5LjQ5NS05LjYtMS40Mi0xLjQwNXoiLz48L3N2Zz4=)}body.light #challenge-error-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSI+PHBhdGggZmlsbD0iI2ZjNTc0YSIgZD0iTTE2IDNhMTMgMTMgMCAxIDAgMTMgMTNBMTMuMDE1IDEzLjAxNSAwIDAgMCAxNiAzbTAgMjRhMTEgMTEgMCAxIDEgMTEtMTEgMTEuMDEgMTEuMDEgMCAwIDEtMTEgMTEiLz48cGF0aCBmaWxsPSIjZmM1NzRhIiBkPSJNMTcuMDM4IDE4LjYxNUgxNC44N0wxNC41NjMgOS41aDIuNzgzem0tMS4wODQgMS40MjdxLjY2IDAgMS4wNTcuMzg4LjQwNy4zODkuNDA3Ljk5NCAwIC41OTYtLjQwNy45ODQtLjM5Ny4zOS0xLjA1Ny4zODktLjY1IDAtMS4wNTYtLjM4OS0uMzk4LS4zODktLjM5OC0uOTg0IDAtLjU5Ny4zOTgtLjk4NS40MDYtLjM5NyAxLjA1Ni0uMzk3Ii8+PC9zdmc+)}a{background-color:transparent;color:#0051c3;text-decoration:none;transition:color .15s ease}a:hover{color:#ee730a;text-decoration:underline}.main-content{margin:8rem auto;max-width:60rem;width:100%}.heading-favicon{height:2rem;margin-right:.5rem;width:2rem}@media (width <= 720px){.main-content{margin-top:4rem}.heading-favicon{height:1.5rem;width:1.5rem}}.footer,.main-content{padding-left:1.5rem;padding-right:1.5rem}.main-wrapper{align-items:center;display:flex;flex:1;flex-direction:column}.font-red{color:#b20f03}.spacer{margin:2rem 0}.h1{font-size:2.5rem;font-weight:500;line-height:3.75rem}.h2{font-weight:500}.core-msg,.h2{font-size:1.5rem;line-height:2.25rem}.body-text,.core-msg{font-weight:400}.body-text{font-size:1rem;line-height:1.25rem}@media (width <= 720px){.h1{font-size:1.5rem;line-height:1.75rem}.h2{font-size:1.25rem}.core-msg,.h2{line-height:1.5rem}.core-msg{font-size:1rem}}#challenge-error-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSI+PHBhdGggZmlsbD0iI2ZjNTc0YSIgZD0iTTE2IDNhMTMgMTMgMCAxIDAgMTMgMTNBMTMuMDE1IDEzLjAxNSAwIDAgMCAxNiAzbTAgMjRhMTEgMTEgMCAxIDEgMTEtMTEgMTEuMDEgMTEuMDEgMCAwIDEtMTEgMTEiLz48cGF0aCBmaWxsPSIjZmM1NzRhIiBkPSJNMTcuMDM4IDE4LjYxNUgxNC44N0wxNC41NjMgOS41aDIuNzgzem0tMS4wODQgMS40MjdxLjY2IDAgMS4wNTcuMzg4LjQwNy4zODkuNDA3Ljk5NCAwIC41OTYtLjQwNy45ODQtLjM5Ny4zOS0xLjA1Ny4zODktLjY1IDAtMS4wNTYtLjM4OS0uMzk4LS4zODktLjM5OC0uOTg0IDAtLjU5Ny4zOTgtLjk4NS40MDYtLjM5NyAxLjA1Ni0uMzk3Ii8+PC9zdmc+);padding-left:34px}#challenge-error-text,#challenge-success-text{background-repeat:no-repeat;background-size:contain}#challenge-success-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSIgdmlld0JveD0iMCAwIDI2IDI2Ij48cGF0aCBmaWxsPSIjMzEzMTMxIiBkPSJNMTMgMGExMyAxMyAwIDEgMCAwIDI2IDEzIDEzIDAgMCAwIDAtMjZtMCAyNGExMSAxMSAwIDEgMSAwLTIyIDExIDExIDAgMCAxIDAgMjIiLz48cGF0aCBmaWxsPSIjMzEzMTMxIiBkPSJtMTAuOTU1IDE2LjA1NS0zLjk1LTQuMTI1LTEuNDQ1IDEuMzg1IDUuMzcgNS42MSA5LjQ5NS05LjYtMS40Mi0xLjQwNXoiLz48L3N2Zz4=);padding-left:42px}.text-center{text-align:center}.pow-button{background-color:#0051c3;border:.063rem solid #0051c3;border-radius:.313rem;color:#fff;font-size:.875rem;line-height:1.313rem;margin:2rem 0;padding:.375rem 1rem;transition-duration:.2s;transition-property:background-color,border-color,color;transition-timing-function:ease}.pow-button:hover{background-color:#003681;border-color:#003681;color:#fff;cursor:pointer}.footer{font-size:.75rem;line-height:1.125rem;margin:0 auto;max-width:60rem;width:100%}.footer-inner{border-top:1px solid #d9d9d9;padding-bottom:1rem;padding-top:1rem}.clearfix:after{clear:both;content:"";display:table}.clearfix .column{float:left;padding-right:1.5rem;width:50%}.diagnostic-wrapper{margin-bottom:.5rem}.footer .ray-id{text-align:center}.footer .ray-id code{font-family:monaco,courier,monospace}.core-msg,.zone-name-title{overflow-wrap:break-word}@media (width <= 720px){.diagnostic-wrapper{display:flex;flex-wrap:wrap;justify-content:center}.clearfix:after{clear:none;content:none;display:initial;text-align:center}.column{padding-bottom:2rem}.clearfix .column{float:none;padding:0;width:auto;word-break:keep-all}.zone-name-title{margin-bottom:1rem}}.loading-spinner{height:76.391px}.lds-ring{display:inline-block;position:relative}.lds-ring,.lds-ring div{height:1.875rem;width:1.875rem}.lds-ring div{animation:lds-ring 1.2s cubic-bezier(.5,0,.5,1) infinite;border:.3rem solid transparent;border-radius:50%;border-top-color:#313131;box-sizing:border-box;display:block;position:absolute}.lds-ring div:first-child{animation-delay:-.45s}.lds-ring div:nth-child(2){animation-delay:-.3s}.lds-ring div:nth-child(3){animation-delay:-.15s}@keyframes lds-ring{0%{transform:rotate(0)}to{transform:rotate(1turn)}}@media screen and (-ms-high-contrast:active),screen and (-ms-high-contrast:none){.main-wrapper,body{display:block}}.rtl .heading-favicon{margin-left:.5rem;margin-right:0}.rtl #challenge-success-text{background-position:100%;padding-left:0;padding-right:42px}.rtl #challenge-error-text{background-position:100%;padding-left:0;padding-right:34px}</style><meta http-equiv="refresh" content="390"></head><body class="no-js"><div class="main-wrapper" role="main"><div class="main-content"><noscript><div id="challenge-error-title"><div class="h2"><span id="challenge-error-text">Enable JavaScript and cookies to continue</span></div></div></noscript></div></div><script>(function(){window._cf_chl_opt={cvId: '3',cZone: "www.congress.gov",cType: 'managed',cNounce: '42744',cRay: '89d0ad692a5eb46e',cHash: '73ae19e5bc313c4',cUPMDTk: "\/118\/bills\/sres706\/BILLS-118sres706ats.xml?__cf_chl_tk=pxml7kV9pjJhOsVRhqQmjkSIA8ffaNj5e2CnpkY2xGU-1719944715-0.0.1.1-4201",cFPWv: 'g',cTTimeMs: '1000',cMTimeMs: '390000',cTplV: 5,cTplB: 'cf',cK: "",fa: "\/118\/bills\/sres706\/BILLS-118sres706ats.xml?__cf_chl_f_tk=pxml7kV9pjJhOsVRhqQmjkSIA8ffaNj5e2CnpkY2xGU-1719944715-0.0.1.1-4201",md: "HDnmaSURfJs7b9j3ECQZNWs6Fn5FZP1xsiH2YHEEnug-1719944715-1.1.1.1-ra4SIWe3hjQZ.yLL3ACg3YmO53QfpCLFSXSxu16IMl3vAl8xueH2IDgwJXHD4DGM73ycdeD7MzANR1JLmGn_AfreoAAIU6GcGUS9lXDxMO_kdoecO2qoY4c9Kdc5I7aDdgAqIGeWFMz_f9aq6i5ltOxbperFDm5XZvNWk0aqH0LQzSJD8cteKZlx5t6TJPsthw844YHMf5AJt__kqpujw0vdM3hbLfDJ0Eop5M7STMgT3MVvYpDNCO05eBGImNvm5Xn_npXu7pq8lY33ipOLVoEwAw7RoizvI4QBu4XM_zP5K4NJWkaErAhYvTB2QPXudtBwjtXIb8NltJ0L.YE6RQxukfk1T3Bgulw3EoiVD1X9ZXW9j7ciMKAxiawfXUfyuIkpJfZvcFABv9ApZ7vKjSKeTMNgGoysGI7b_0WLpM09SZTFX8c1_PU3mbrz4NhUEuR4NbpfyeDh7TCDTV63GeQPFbJJipPyDCXqd9mvMolcpEILvIlnE4z10YgcqNSKBBdoYCJikPYDxjN8_kHX6bQyjeTp4jfggLon5LJNtJhVEbC7rVPuiitAe5V7qGcZfpntcMpGFHCXgJmUFev__ckhMIcjgB7i0w0PvX.gFTyaDVoqJleTekb1oq.YYnKrp8m4H25bGmf4WuDpGAsrHjeXG0VISWsH3t8jAXGXEKmQRYQGl9JcnEsDcktvHdzRxJyqEQbzFsOre6CzfDD3OQDQabZ524809rB9mORgPVPaiaAP9htEXT7mbn.XaCf5FogpGaOoZ1D5lniVU3isl9DKPe.3Vtnz2N2J8GlOyYFdmPXBQxpqEYQ6VwbKn6Vs9wTvTAuhpyjpw1m59D5_DbRui0wB38PW0KxaLRxqo8.zKI3pVhKkkPbX.xA8zmCQ_Ht9prV1BKzh6Emy41ctJW0JYxp5_kJlCLSWgHZNLwIKGAokxs15GdHIRrIMkWMjXAoO20bnGJadGrEYHKMDoVIwqPSRt1aPplWV62E_RX1RECsRMxteSnCboXICFvFmlzva7K5UhQuabxnpPerbwp1WPpg4ZzFkSp_ZCTMnmhv8E6nxhkTb.8OLvt1NDXVgUab6AGRSCFOUsf0dyz2lAhHOmrLJvVot9ydioaspl_1kEQ7VSZ8MGSyXr9ShIPFgtaS4lA4IMROLlC_IDQdYKQlLFqtoW5py14Yr9WO5bh36rwEnER9EebJbfB.cFxswCF8HXX.t6efqiZDeqLuhd3XFB5_KTwAxJalmoPdXhALCAl94nDof7PeC47bLicY3Kn_Q.DdqEleTZ8sRvnxggOcamPtPU8WfKsaTEgpAY.I7gbI1DCIw3B5fQY6nChDh0Uyl2d3tZO7OrTdSaJrp1D5tCvOOv8t.PSWAbJn7ryAWVe9nmyk5ZjJ548YZ7GNe.31pJiA_TGM5je9EgUj_lf.dhBlllfktskSK7oIWM8ZTIzHKOpULSyezTlQ.1Ara",mdrd: "KOY_.DkVCKc3K26kinq2oDMkzzQ.VkiOX0x8I2sNvmE-1719944715-1.1.1.1-9ZGz.mGSYsCvPPJS.cdVIvT0xC0ld1jLzGN4fWM772WkC0lwNMuzdV6kuC.3l01m_HIrbvq83sIYmrBI4M36Vs.YRsu3761oUz8ydX4FUpIt4BySpS0ooRMISi3iipzhb5HnCHZaWK4WK7WYYkczqDAdxrwbhvhsil67PiGrXGtkBf5UevW9bGJ9yK_9quW8XfoR1Dg7mLhTtw_lEut5xD1hLgV9XRaLhSvwwNop9OVzXA0erJHTu85lLIAfviFHNBLYwt2pc6lyJsevMdx2tgoKcKkfE6ub7zCADfb0yCN5Z6mMt66D5L5kVTgSJEWcqA8q3OCexzTeE57MpokbJUTfwGC6Z4_TEGN9ZWjuMj34mZv4fVj0CoNylOHfOJHPRXpVY86XtIr6OGnf5fXLoWHz7gM1SHcJF5NTeG2m0Q.m.KopzIX8_rj2o0pV544A7E_XFIRDV6zED9mDuOvL0CD9OoziDzXXi7tqm86BQkg3pbRz2gGH7WjS3sgxrPwKvC9C_Qx0nqMcCKtrvuWMmR.io7sVyaeNvchLjzQowOi6LtRx6aTH8V_akCGYVs9MCudGF60DD6E0tf93gWOXYD4_NTxxY4iW5UYCJavYUTq.kQzQOo5H.tEagTNbF9aPFRegy3YajZteTtwUqih.P6nTJ0qVQlqdrgR.J877v_ZnfrV7nd4h2fpLK1.LCUup92G8Q4ucTHd5jyfkLlF9XSoHcTuyPqevrCAb0r7XrT2lgAxWrKMdwGCjEb6ARX89qAGHefd8ZcnFAd.brf61eXIMcDmvldViWZd7kaiWocY99cSepiDlzh_RR3PpobfiLzxQoyjXPfeytZJ3OL66ukoo3Jbun.wvpUOAlhkgP5L2EeE1dl20soCHukYHK8kpZqp.lKDYU9wnKQt5buWbTQLDwrS.w0yo36OqhgQ9Wxiyd8n1g5LmATinD0cZAqHfWVEdwjv8x5Dox1pY0T_TY_Z5WlnzJjhbBiKd9sc1Mov__IZ2Je4vWmgvtnoDxvZrV55WXLqRWZxl4Bt1LICKiNWfwg5r0nbkMoilItuw.87PYy.bKJ0ECQtROZc3WHqc1BdHkw7iVtGhepC6RmEWgH4xeKUd1SQZ7SLwj_i5WOSx5Y576v_PC1c_ES.uB2gOpvlyc6R1jVfRR5uOExYXa4C6nNzCLKce8FX4SSWNobmb9_ACwaVishhwMc0EsvZ5Z9YGRwOhWTClnbnXcizuWIOulC.med6SrlaWq0rnA2mz0oKkhLJijAN.Q1YGFuIj1RYPzE2PnL9gG_xLgl060XHGO5P.1vK2dZpVVLbajEzFpW4WJmiSRFNe830VSSnf41rDxhJyr6sCizjNr6lp_h08kx_YLjevBH7dluQmDNRG9a2x_3d.csImv9SLyEH3rzx4tagRk_Z7c_7a60vJx1Kptm74A1aOxPD0D2EI1290jMv5rtq5WH2mniCezWabsNYa1hmIgV0FMCuUdtSi_h.u6Hv_edoLnwKx1u1enS6t_AmatRX.TzAezxb.5LUW2L6VnanRwDEpWGWcYRdmtKJ6insesi5XLXcTXzqzsDZj9ssnvEw9SBnrfDjwoM7XnLoPSvkafrxeZDGEz5NsDxDr_nja9_96hRgjeSbYCSoNx3UbiL7Zy6erfyYdI_GUpqzkahaibNLwcX..zYNId3F0joztVXJe3JBgsUc4FXWhU8Un9O.RTonko8X7qgjANQlEqLytTbHcogptOiOQGY6AUv_5UH0T5r.GlkLq3VQFJZxbsNKXKfK6wPVDugxN9PWJx63B5Gt2xgZA4ETLoEr.L19IAnki7yuu2iymE9U4KVzyncZvPzKCRmDswoJXX8e2cDhLiuuX5bn8ooiMDcztOXg9r10O0cbegFMMfcz6mQfqOMNDWjaKMc5Ify0wZf0aiTOIZgWrAnXwm3r.bRBPeC5eBSQW3XbWaLeCrisLn9StbXvCTv.1ApUr3VceJeoPu1kFFLMYUUxy2QRDoeSQ7h.7uEmNR4v49dJDAdce9USSi19ioaryfONUbhGymGtmvBtFkaq2WQJzBH1rlkrV4wTc4l5pf_QS6v1_N_WMi0Lk2AiX8lWciH38KwNGkf2nlMEClEYVl1MX9.oplIfA6ylQuLZ3dLX.l5QdcKDuWRvIDYnAu5sgVLSoASGiCauQlKvjD4FeHo9I_aDudziwwRP7IMHUhcJa8f4mr6Y0VQJkLPDDJ5Cmt1dTQesEkZlIqMpfwu4Ek4TblLTRiyDgoJY7UUBK1wSLAAQt7sI.Zk5kS5ocDpg.FaOccMd_ruVbGkRh1MErdguJL4sazbNUspF8JDWeisD7pW_vJUFUPAfot.f_e8_BL9vwAuxC6T5UD9W4g.VCAUaTZOOrWTeNMWp70WG_qltVkLG97kle.U7oCh7yQrPuJ4Z2QN9HCsfMMGsWSEIUTeB50NnlRcWXFwliInlqdSuNXxChTbV.6iPk_U6HZXK6XxNsMeoBjqARy6Tp1CSrqq7kRMPmoq9fFjr8_lAIZ5ankdKbdCUvwfRHZlUvoQW8vFeRxGRaTXoyAU3KO4ByPj1iIvgsnz8FMpxm4UO9FwMZ9AUjWMg",cRq: {ru: 'aHR0cHM6Ly93d3cuY29uZ3Jlc3MuZ292LzExOC9iaWxscy9zcmVzNzA2L0JJTExTLTExOHNyZXM3MDZhdHMueG1s',ra: 'Q29uZ3Jlc3MtV2lraS8wLjQ=',rm: 'R0VU',d: 'gCi1sT3r885ljxKYmgwZHAlytnbg5OPNusq0FxDZgg5KSXqdw+exPTGnWEL8lJOSBGxWY2jwBdj7CKuAV6mUYc3LZXFJ6HZ8bAWXi0QS2L1uY26jrK0S7VyGWzZGoqV+hRA2SxXTZdL53a67TZqQBHmeybn/n5epkqNRE6pBUh2EE/aR++Afl9F8NyhBZbjIPPzl3GcX3+gdTFHdIB7I1tnHOo2K4obIQJqZ0a8u6oKfe7zkL5zwbbnHXb5f9nx/mHO5/t3iynomWp0Fio6mK/kk28H0DeGwScGsakBCNH38jqWzwmtiOvBf1CfEKWy/ORCLYkAVcrJ3jOZ1waOgZLxMP/1T8nA501y97259cPUJBPGzfP+MnCqrmHEwbPpGvTgva/AhqYP2bK72zJN80GakEvJNh1pXWmQk/XUJzEjJagyzb2nXFb5Jvn4J3wSbXXVQf1Ou1BYyUqM7svF1o56GiIlYf7qfTWh+XCZbjPs=',t: 'MTcxOTk0NDcxNS4wMDAwMDA=',cT: Math.floor(Date.now() / 1000),m: 'lMvffQ+36kpDyVCFaRk+BzFFLgDo9+/GoJvfC+xV1xI=',i1: 'nm1Y8MsB+Zlv2//GJ79enw==',i2: 'X4CVElYGDOLxz6kdEAn6eA==',zh: 'P/37dfGD+KUZFm0qKuE3N/q0gn3BGPttWTswwTYBAD8=',uh: 'FVHG7QmCIZ0ivo719LfrxFm2W3GVAQSMC1lFhLfL6/s=',hh: 'cTbG2zct7/9cr7abEWSg59sInRszec2mTWq3jspVNx4=',}};var cpo = document.createElement('script');cpo.src = '/cdn-cgi/challenge-platform/h/g/orchestrate/chl_page/v1?ray=89d0ad692a5eb46e';window._cf_chl_opt.cOgUHash = location.hash === '' && location.href.indexOf('#') !== -1 ? '#' : location.hash;window._cf_chl_opt.cOgUQuery = location.search === '' && location.href.slice(0, location.href.length - window._cf_chl_opt.cOgUHash.length).indexOf('?') !== -1 ? '?' : location.search;if (window.history && window.history.replaceState) {var ogU = location.pathname + window._cf_chl_opt.cOgUQuery + window._cf_chl_opt.cOgUHash;history.replaceState(null, null, "\/118\/bills\/sres706\/BILLS-118sres706ats.xml?__cf_chl_rt_tk=pxml7kV9pjJhOsVRhqQmjkSIA8ffaNj5e2CnpkY2xGU-1719944715-0.0.1.1-4201" + window._cf_chl_opt.cOgUHash);cpo.onload = function() {history.replaceState(null, null, ogU);}}document.getElementsByTagName('head')[0].appendChild(cpo);}());</script></body></html> Request failed with status code 429
Heres a log of http requests where congress.gov last returned successfully, and then began to respond with cloudfare 429.
2024-07-02T18:23:57.176Z [congressgov-client] http: >>> GET 'https://api.congress.gov/v3/bill/118/sres/737/text'
params: {
"api_key": "****************************************",
"format": "json"
}
2024-07-02T18:23:57.404Z [congressgov-client] http: <<< 200 (OK) from 'api.congress.gov/v3/bill/118/sres/737/text?api_key=****************************************&format=json'
headers: {
"date": "Tue, 02 Jul 2024 18:23:57 GMT",
"content-type": "application/json",
"transfer-encoding": "chunked",
"connection": "keep-alive",
"access-control-allow-origin": "*",
"age": "0",
"cache-control": "max-age=900",
"cf-cache-status": "DYNAMIC",
"cf-ray": "89d0ab7eef80b9f7-SEA",
"expires": "Tue, 02 Jul 2024 18:38:57 GMT",
"strict-transport-security": "max-age=31536000; preload",
"vary": "Accept-Encoding, Accept",
"via": "https/1.1 api-umbrella (ApacheTrafficServer [cMsSf ])",
"x-api-umbrella-request-id": "cgeio6mikaeqnvmqnrs0",
"x-cache": "MISS",
"x-content-type-options": "nosniff",
"x-ratelimit-limit": "5000",
"x-ratelimit-remaining": "4813",
"x-vcap-request-id": "695cbb1a-fc56-4e83-71ba-61c37f256f8f",
"x-xss-protection": "1; mode=block",
"x-frame-options": "DENY"
}
2024-07-02T18:23:57.905Z [congressgov-client] http: >>> GET 'https://www.congress.gov/118/bills/sres737/BILLS-118sres737ats.xml'
params: undefined
2024-07-02T18:23:57.985Z [congressgov-client] http: <<< 200 (OK) from 'www.congress.gov/118/bills/sres737/BILLS-118sres737ats.xml'
headers: {
"date": "Tue, 02 Jul 2024 18:23:58 GMT",
"content-type": "application/xml",
"transfer-encoding": "chunked",
"connection": "keep-alive",
"last-modified": "Wed, 19 Jun 2024 04:44:21 GMT",
"x-content-type-options": "nosniff",
"strict-transport-security": "max-age=31536000; includeSubDomains; preload",
"x-xss-protection": "1; mode=block",
"etag": "W/\"2839-61b36da155fd7-gzip\"",
"x-cacheable": "YES",
"cache-control": "public, max-age=1800",
"content-security-policy": "frame-ancestors http://*.house.gov https://*.house.gov http://*.senate.gov https://*.senate.gov http://*.loc.gov https://*.loc.gov http://*.congress.gov https://congress.gov https://*.congress.gov",
"cf-cache-status": "MISS",
"expires": "Tue, 02 Jul 2024 18:53:58 GMT",
"vary": "Accept-Encoding",
"server": "cloudflare",
"cf-ray": "89d0ab832c5fe264-ORD"
}
2024-07-02T18:23:58.194Z [congressgov-client] http: >>> GET 'https://www.congress.gov/118/bills/sres737/BILLS-118sres737ats.pdf'
params: undefined
2024-07-02T18:23:58.763Z [congressgov-client] http: <<< 200 (OK) from 'www.congress.gov/118/bills/sres737/BILLS-118sres737ats.pdf'
headers: {
"date": "Tue, 02 Jul 2024 18:23:58 GMT",
"content-type": "application/pdf",
"transfer-encoding": "chunked",
"connection": "keep-alive",
"last-modified": "Wed, 19 Jun 2024 04:44:23 GMT",
"cache-control": "public, max-age=31536000",
"expires": "Wed, 02 Jul 2025 18:23:58 GMT",
"vary": "Accept-Encoding",
"x-content-type-options": "nosniff",
"strict-transport-security": "max-age=31536000; includeSubDomains; preload",
"x-xss-protection": "1; mode=block",
"x-cacheable": "YES",
"content-security-policy": "frame-ancestors http://*.house.gov https://*.house.gov http://*.senate.gov https://*.senate.gov http://*.loc.gov https://*.loc.gov http://*.congress.gov https://congress.gov https://*.congress.gov",
"cf-cache-status": "REVALIDATED",
"server": "cloudflare",
"cf-ray": "89d0ab84fe13e264-ORD"
}
2024-07-02T18:23:58.965Z [congressgov-client] http: >>> GET 'https://www.congress.gov/118/bills/sres737/BILLS-118sres737ats.htm'
params: undefined
2024-07-02T18:23:59.059Z [congressgov-client] http: <<< 200 (OK) from 'www.congress.gov/118/bills/sres737/BILLS-118sres737ats.htm'
headers: {
"date": "Tue, 02 Jul 2024 18:23:59 GMT",
"content-type": "text/html",
"transfer-encoding": "chunked",
"connection": "keep-alive",
"last-modified": "Wed, 19 Jun 2024 04:44:21 GMT",
"vary": "Accept-Encoding",
"x-content-type-options": "nosniff",
"strict-transport-security": "max-age=31536000; includeSubDomains; preload",
"x-xss-protection": "1; mode=block",
"x-cacheable": "YES",
"cache-control": "public, max-age=1800",
"content-security-policy": "frame-ancestors http://*.house.gov https://*.house.gov http://*.senate.gov https://*.senate.gov http://*.loc.gov https://*.loc.gov http://*.congress.gov https://congress.gov https://*.congress.gov",
"cf-cache-status": "MISS",
"expires": "Tue, 02 Jul 2024 18:53:59 GMT",
"server": "cloudflare",
"cf-ray": "89d0ab89da81e264-ORD"
}
2024-07-02T18:23:59.270Z [congressgov-client] http: >>> GET 'https://api.congress.gov/v3/bill/118/sres/727/text'
params: {
"api_key": "****************************************",
"format": "json"
}
2024-07-02T18:23:59.503Z [congressgov-client] http: <<< 200 (OK) from 'api.congress.gov/v3/bill/118/sres/727/text?api_key=****************************************&format=json'
headers: {
"date": "Tue, 02 Jul 2024 18:23:59 GMT",
"content-type": "application/json",
"transfer-encoding": "chunked",
"connection": "keep-alive",
"access-control-allow-origin": "*",
"age": "0",
"cache-control": "max-age=900",
"cf-cache-status": "DYNAMIC",
"cf-ray": "89d0ab8c08f27651-SEA",
"expires": "Tue, 02 Jul 2024 18:38:59 GMT",
"strict-transport-security": "max-age=31536000; preload",
"vary": "Accept-Encoding, Accept",
"via": "https/1.1 api-umbrella (ApacheTrafficServer [cMsSf ])",
"x-api-umbrella-request-id": "cgeio780p7lh56sh3tvg",
"x-cache": "MISS",
"x-content-type-options": "nosniff",
"x-ratelimit-limit": "5000",
"x-ratelimit-remaining": "4812",
"x-vcap-request-id": "cbc715e9-6e83-42c3-77fb-1c108495eecb",
"x-xss-protection": "1; mode=block",
"x-frame-options": "DENY"
}
2024-07-02T18:24:00.008Z [congressgov-client] http: >>> GET 'https://www.congress.gov/118/bills/sres727/BILLS-118sres727ats.xml'
params: undefined
2024-07-02T18:24:00.093Z [congressgov-client] http: <<< 200 (OK) from 'www.congress.gov/118/bills/sres727/BILLS-118sres727ats.xml'
headers: {
"date": "Tue, 02 Jul 2024 18:24:00 GMT",
"content-type": "application/xml",
"transfer-encoding": "chunked",
"connection": "keep-alive",
"last-modified": "Thu, 13 Jun 2024 00:14:26 GMT",
"x-content-type-options": "nosniff",
"strict-transport-security": "max-age=31536000; includeSubDomains; preload",
"x-xss-protection": "1; mode=block",
"etag": "W/\"1347-61aba61bcc8e2-gzip\"",
"x-cacheable": "YES",
"cache-control": "public, max-age=1800",
"content-security-policy": "frame-ancestors http://*.house.gov https://*.house.gov http://*.senate.gov https://*.senate.gov http://*.loc.gov https://*.loc.gov http://*.congress.gov https://congress.gov https://*.congress.gov",
"cf-cache-status": "MISS",
"expires": "Tue, 02 Jul 2024 18:54:00 GMT",
"vary": "Accept-Encoding",
"server": "cloudflare",
"cf-ray": "89d0ab905875e264-ORD"
}
2024-07-02T18:24:00.299Z [congressgov-client] http: >>> GET 'https://www.congress.gov/118/bills/sres727/BILLS-118sres727ats.pdf'
params: undefined
2024-07-02T18:24:00.560Z [congressgov-client] http: <<< 200 (OK) from 'www.congress.gov/118/bills/sres727/BILLS-118sres727ats.pdf'
headers: {
"date": "Tue, 02 Jul 2024 18:24:00 GMT",
"content-type": "application/pdf",
"transfer-encoding": "chunked",
"connection": "keep-alive",
"last-modified": "Thu, 13 Jun 2024 00:14:28 GMT",
"cache-control": "public, max-age=31536000",
"expires": "Wed, 02 Jul 2025 18:24:00 GMT",
"vary": "Accept-Encoding",
"x-content-type-options": "nosniff",
"strict-transport-security": "max-age=31536000; includeSubDomains; preload",
"x-xss-protection": "1; mode=block",
"x-cacheable": "YES",
"content-security-policy": "frame-ancestors http://*.house.gov https://*.house.gov http://*.senate.gov https://*.senate.gov http://*.loc.gov https://*.loc.gov http://*.congress.gov https://congress.gov https://*.congress.gov",
"cf-cache-status": "MISS",
"server": "cloudflare",
"cf-ray": "89d0ab922a46e264-ORD"
}
2024-07-02T18:24:00.762Z [congressgov-client] http: >>> GET 'https://www.congress.gov/118/bills/sres727/BILLS-118sres727ats.htm'
params: undefined
2024-07-02T18:24:00.832Z [congressgov-client] http: <<< 200 (OK) from 'www.congress.gov/118/bills/sres727/BILLS-118sres727ats.htm'
headers: {
"date": "Tue, 02 Jul 2024 18:24:00 GMT",
"content-type": "text/html",
"transfer-encoding": "chunked",
"connection": "keep-alive",
"last-modified": "Thu, 13 Jun 2024 00:14:26 GMT",
"vary": "Accept-Encoding",
"x-content-type-options": "nosniff",
"strict-transport-security": "max-age=31536000; includeSubDomains; preload",
"x-xss-protection": "1; mode=block",
"x-cacheable": "YES",
"cache-control": "public, max-age=1800",
"content-security-policy": "frame-ancestors http://*.house.gov https://*.house.gov http://*.senate.gov https://*.senate.gov http://*.loc.gov https://*.loc.gov http://*.congress.gov https://congress.gov https://*.congress.gov",
"cf-cache-status": "MISS",
"expires": "Tue, 02 Jul 2024 18:54:00 GMT",
"server": "cloudflare",
"cf-ray": "89d0ab950cb3e264-ORD"
}
2024-07-02T18:24:01.054Z [congressgov-client] http: >>> GET 'https://api.congress.gov/v3/bill/118/sres/726/text'
params: {
"api_key": "****************************************",
"format": "json"
}
2024-07-02T18:24:01.274Z [congressgov-client] http: <<< 200 (OK) from 'api.congress.gov/v3/bill/118/sres/726/text?api_key=****************************************&format=json'
headers: {
"date": "Tue, 02 Jul 2024 18:24:01 GMT",
"content-type": "application/json",
"transfer-encoding": "chunked",
"connection": "keep-alive",
"access-control-allow-origin": "*",
"age": "0",
"cache-control": "max-age=900",
"cf-cache-status": "DYNAMIC",
"cf-ray": "89d0ab972b4c681f-SEA",
"expires": "Tue, 02 Jul 2024 18:39:01 GMT",
"strict-transport-security": "max-age=31536000; preload",
"vary": "Accept-Encoding, Accept",
"via": "https/1.1 api-umbrella (ApacheTrafficServer [cMsSf ])",
"x-api-umbrella-request-id": "cgeio7gcifb9uk29tk80",
"x-cache": "MISS",
"x-content-type-options": "nosniff",
"x-ratelimit-limit": "5000",
"x-ratelimit-remaining": "4811",
"x-vcap-request-id": "908b8f97-ea4f-4b2b-49ca-9dfd801a203e",
"x-xss-protection": "1; mode=block",
"x-frame-options": "DENY"
}
2024-07-02T18:24:01.775Z [congressgov-client] http: >>> GET 'https://www.congress.gov/118/bills/sres726/BILLS-118sres726ats.xml'
params: undefined
2024-07-02T18:24:01.837Z [congressgov-client] error: RetryCondition: GET Request to www.congress.gov/118/bills/sres726/BILLS-118sres726ats.xml failed with error
AxiosError: Request failed with status code 429
at settle (/Users/ryanparker/GitHub/collector/node_modules/axios/lib/core/settle.js:19:12)
at IncomingMessage.handleStreamEnd (/Users/ryanparker/GitHub/collector/node_modules/axios/lib/adapters/http.js:589:11)
at IncomingMessage.emit (node:events:531:35)
at endReadableNT (node:internal/streams/readable:1696:12)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
undefined
2024-07-02T18:24:01.838Z [congressgov-client] error: <<< Request failed for 'www.congress.gov/118/bills/sres726/BILLS-118sres726ats.xml' with error
Response: 429 (Too Many Requests)
<!DOCTYPE html><html lang="en-US"><head><title>Just a moment...</title><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta http-equiv="X-UA-Compatible" content="IE=Edge"><meta name="robots" content="noindex,nofollow"><meta name="viewport" content="width=device-width,initial-scale=1"><style>*{box-sizing:border-box;margin:0;padding:0}html{line-height:1.15;-webkit-text-size-adjust:100%;color:#313131}button,html{font-family:system-ui,-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Helvetica Neue,Arial,Noto Sans,sans-serif,Apple Color Emoji,Segoe UI Emoji,Segoe UI Symbol,Noto Color Emoji}@media (prefers-color-scheme:dark){body{background-color:#222;color:#d9d9d9}body a{color:#fff}body a:hover{color:#ee730a;text-decoration:underline}body .lds-ring div{border-color:#999 transparent transparent}body .font-red{color:#b20f03}body .pow-button{background-color:#4693ff;color:#1d1d1d}body #challenge-success-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSIgdmlld0JveD0iMCAwIDI2IDI2Ij48cGF0aCBmaWxsPSIjZDlkOWQ5IiBkPSJNMTMgMGExMyAxMyAwIDEgMCAwIDI2IDEzIDEzIDAgMCAwIDAtMjZtMCAyNGExMSAxMSAwIDEgMSAwLTIyIDExIDExIDAgMCAxIDAgMjIiLz48cGF0aCBmaWxsPSIjZDlkOWQ5IiBkPSJtMTAuOTU1IDE2LjA1NS0zLjk1LTQuMTI1LTEuNDQ1IDEuMzg1IDUuMzcgNS42MSA5LjQ5NS05LjYtMS40Mi0xLjQwNXoiLz48L3N2Zz4=)}body #challenge-error-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSI+PHBhdGggZmlsbD0iI0IyMEYwMyIgZD0iTTE2IDNhMTMgMTMgMCAxIDAgMTMgMTNBMTMuMDE1IDEzLjAxNSAwIDAgMCAxNiAzbTAgMjRhMTEgMTEgMCAxIDEgMTEtMTEgMTEuMDEgMTEuMDEgMCAwIDEtMTEgMTEiLz48cGF0aCBmaWxsPSIjQjIwRjAzIiBkPSJNMTcuMDM4IDE4LjYxNUgxNC44N0wxNC41NjMgOS41aDIuNzgzem0tMS4wODQgMS40MjdxLjY2IDAgMS4wNTcuMzg4LjQwNy4zODkuNDA3Ljk5NCAwIC41OTYtLjQwNy45ODQtLjM5Ny4zOS0xLjA1Ny4zODktLjY1IDAtMS4wNTYtLjM4OS0uMzk4LS4zODktLjM5OC0uOTg0IDAtLjU5Ny4zOTgtLjk4NS40MDYtLjM5NyAxLjA1Ni0uMzk3Ii8+PC9zdmc+)}}body{display:flex;flex-direction:column;min-height:100vh}body.no-js .loading-spinner{visibility:hidden}body.no-js .challenge-running{display:none}body.dark{background-color:#222;color:#d9d9d9}body.dark a{color:#fff}body.dark a:hover{color:#ee730a;text-decoration:underline}body.dark .lds-ring div{border-color:#999 transparent transparent}body.dark .font-red{color:#b20f03}body.dark .pow-button{background-color:#4693ff;color:#1d1d1d}body.dark #challenge-success-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSIgdmlld0JveD0iMCAwIDI2IDI2Ij48cGF0aCBmaWxsPSIjZDlkOWQ5IiBkPSJNMTMgMGExMyAxMyAwIDEgMCAwIDI2IDEzIDEzIDAgMCAwIDAtMjZtMCAyNGExMSAxMSAwIDEgMSAwLTIyIDExIDExIDAgMCAxIDAgMjIiLz48cGF0aCBmaWxsPSIjZDlkOWQ5IiBkPSJtMTAuOTU1IDE2LjA1NS0zLjk1LTQuMTI1LTEuNDQ1IDEuMzg1IDUuMzcgNS42MSA5LjQ5NS05LjYtMS40Mi0xLjQwNXoiLz48L3N2Zz4=)}body.dark #challenge-error-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSI+PHBhdGggZmlsbD0iI0IyMEYwMyIgZD0iTTE2IDNhMTMgMTMgMCAxIDAgMTMgMTNBMTMuMDE1IDEzLjAxNSAwIDAgMCAxNiAzbTAgMjRhMTEgMTEgMCAxIDEgMTEtMTEgMTEuMDEgMTEuMDEgMCAwIDEtMTEgMTEiLz48cGF0aCBmaWxsPSIjQjIwRjAzIiBkPSJNMTcuMDM4IDE4LjYxNUgxNC44N0wxNC41NjMgOS41aDIuNzgzem0tMS4wODQgMS40MjdxLjY2IDAgMS4wNTcuMzg4LjQwNy4zODkuNDA3Ljk5NCAwIC41OTYtLjQwNy45ODQtLjM5Ny4zOS0xLjA1Ny4zODktLjY1IDAtMS4wNTYtLjM4OS0uMzk4LS4zODktLjM5OC0uOTg0IDAtLjU5Ny4zOTgtLjk4NS40MDYtLjM5NyAxLjA1Ni0uMzk3Ii8+PC9zdmc+)}body.light{background-color:transparent;color:#313131}body.light a{color:#0051c3}body.light a:hover{color:#ee730a;text-decoration:underline}body.light .lds-ring div{border-color:#595959 transparent transparent}body.light .font-red{color:#fc574a}body.light .pow-button{background-color:#003681;border-color:#003681;color:#fff}body.light #challenge-success-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSIgdmlld0JveD0iMCAwIDI2IDI2Ij48cGF0aCBmaWxsPSIjMzEzMTMxIiBkPSJNMTMgMGExMyAxMyAwIDEgMCAwIDI2IDEzIDEzIDAgMCAwIDAtMjZtMCAyNGExMSAxMSAwIDEgMSAwLTIyIDExIDExIDAgMCAxIDAgMjIiLz48cGF0aCBmaWxsPSIjMzEzMTMxIiBkPSJtMTAuOTU1IDE2LjA1NS0zLjk1LTQuMTI1LTEuNDQ1IDEuMzg1IDUuMzcgNS42MSA5LjQ5NS05LjYtMS40Mi0xLjQwNXoiLz48L3N2Zz4=)}body.light #challenge-error-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSI+PHBhdGggZmlsbD0iI2ZjNTc0YSIgZD0iTTE2IDNhMTMgMTMgMCAxIDAgMTMgMTNBMTMuMDE1IDEzLjAxNSAwIDAgMCAxNiAzbTAgMjRhMTEgMTEgMCAxIDEgMTEtMTEgMTEuMDEgMTEuMDEgMCAwIDEtMTEgMTEiLz48cGF0aCBmaWxsPSIjZmM1NzRhIiBkPSJNMTcuMDM4IDE4LjYxNUgxNC44N0wxNC41NjMgOS41aDIuNzgzem0tMS4wODQgMS40MjdxLjY2IDAgMS4wNTcuMzg4LjQwNy4zODkuNDA3Ljk5NCAwIC41OTYtLjQwNy45ODQtLjM5Ny4zOS0xLjA1Ny4zODktLjY1IDAtMS4wNTYtLjM4OS0uMzk4LS4zODktLjM5OC0uOTg0IDAtLjU5Ny4zOTgtLjk4NS40MDYtLjM5NyAxLjA1Ni0uMzk3Ii8+PC9zdmc+)}a{background-color:transparent;color:#0051c3;text-decoration:none;transition:color .15s ease}a:hover{color:#ee730a;text-decoration:underline}.main-content{margin:8rem auto;max-width:60rem;width:100%}.heading-favicon{height:2rem;margin-right:.5rem;width:2rem}@media (width <= 720px){.main-content{margin-top:4rem}.heading-favicon{height:1.5rem;width:1.5rem}}.footer,.main-content{padding-left:1.5rem;padding-right:1.5rem}.main-wrapper{align-items:center;display:flex;flex:1;flex-direction:column}.font-red{color:#b20f03}.spacer{margin:2rem 0}.h1{font-size:2.5rem;font-weight:500;line-height:3.75rem}.h2{font-weight:500}.core-msg,.h2{font-size:1.5rem;line-height:2.25rem}.body-text,.core-msg{font-weight:400}.body-text{font-size:1rem;line-height:1.25rem}@media (width <= 720px){.h1{font-size:1.5rem;line-height:1.75rem}.h2{font-size:1.25rem}.core-msg,.h2{line-height:1.5rem}.core-msg{font-size:1rem}}#challenge-error-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSI+PHBhdGggZmlsbD0iI2ZjNTc0YSIgZD0iTTE2IDNhMTMgMTMgMCAxIDAgMTMgMTNBMTMuMDE1IDEzLjAxNSAwIDAgMCAxNiAzbTAgMjRhMTEgMTEgMCAxIDEgMTEtMTEgMTEuMDEgMTEuMDEgMCAwIDEtMTEgMTEiLz48cGF0aCBmaWxsPSIjZmM1NzRhIiBkPSJNMTcuMDM4IDE4LjYxNUgxNC44N0wxNC41NjMgOS41aDIuNzgzem0tMS4wODQgMS40MjdxLjY2IDAgMS4wNTcuMzg4LjQwNy4zODkuNDA3Ljk5NCAwIC41OTYtLjQwNy45ODQtLjM5Ny4zOS0xLjA1Ny4zODktLjY1IDAtMS4wNTYtLjM4OS0uMzk4LS4zODktLjM5OC0uOTg0IDAtLjU5Ny4zOTgtLjk4NS40MDYtLjM5NyAxLjA1Ni0uMzk3Ii8+PC9zdmc+);padding-left:34px}#challenge-error-text,#challenge-success-text{background-repeat:no-repeat;background-size:contain}#challenge-success-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSIgdmlld0JveD0iMCAwIDI2IDI2Ij48cGF0aCBmaWxsPSIjMzEzMTMxIiBkPSJNMTMgMGExMyAxMyAwIDEgMCAwIDI2IDEzIDEzIDAgMCAwIDAtMjZtMCAyNGExMSAxMSAwIDEgMSAwLTIyIDExIDExIDAgMCAxIDAgMjIiLz48cGF0aCBmaWxsPSIjMzEzMTMxIiBkPSJtMTAuOTU1IDE2LjA1NS0zLjk1LTQuMTI1LTEuNDQ1IDEuMzg1IDUuMzcgNS42MSA5LjQ5NS05LjYtMS40Mi0xLjQwNXoiLz48L3N2Zz4=);padding-left:42px}.text-center{text-align:center}.pow-button{background-color:#0051c3;border:.063rem solid #0051c3;border-radius:.313rem;color:#fff;font-size:.875rem;line-height:1.313rem;margin:2rem 0;padding:.375rem 1rem;transition-duration:.2s;transition-property:background-color,border-color,color;transition-timing-function:ease}.pow-button:hover{background-color:#003681;border-color:#003681;color:#fff;cursor:pointer}.footer{font-size:.75rem;line-height:1.125rem;margin:0 auto;max-width:60rem;width:100%}.footer-inner{border-top:1px solid #d9d9d9;padding-bottom:1rem;padding-top:1rem}.clearfix:after{clear:both;content:"";display:table}.clearfix .column{float:left;padding-right:1.5rem;width:50%}.diagnostic-wrapper{margin-bottom:.5rem}.footer .ray-id{text-align:center}.footer .ray-id code{font-family:monaco,courier,monospace}.core-msg,.zone-name-title{overflow-wrap:break-word}@media (width <= 720px){.diagnostic-wrapper{display:flex;flex-wrap:wrap;justify-content:center}.clearfix:after{clear:none;content:none;display:initial;text-align:center}.column{padding-bottom:2rem}.clearfix .column{float:none;padding:0;width:auto;word-break:keep-all}.zone-name-title{margin-bottom:1rem}}.loading-spinner{height:76.391px}.lds-ring{display:inline-block;position:relative}.lds-ring,.lds-ring div{height:1.875rem;width:1.875rem}.lds-ring div{animation:lds-ring 1.2s cubic-bezier(.5,0,.5,1) infinite;border:.3rem solid transparent;border-radius:50%;border-top-color:#313131;box-sizing:border-box;display:block;position:absolute}.lds-ring div:first-child{animation-delay:-.45s}.lds-ring div:nth-child(2){animation-delay:-.3s}.lds-ring div:nth-child(3){animation-delay:-.15s}@keyframes lds-ring{0%{transform:rotate(0)}to{transform:rotate(1turn)}}@media screen and (-ms-high-contrast:active),screen and (-ms-high-contrast:none){.main-wrapper,body{display:block}}.rtl .heading-favicon{margin-left:.5rem;margin-right:0}.rtl #challenge-success-text{background-position:100%;padding-left:0;padding-right:42px}.rtl #challenge-error-text{background-position:100%;padding-left:0;padding-right:34px}</style><meta http-equiv="refresh" content="390"></head><body class="no-js"><div class="main-wrapper" role="main"><div class="main-content"><noscript><div id="challenge-error-title"><div class="h2"><span id="challenge-error-text">Enable JavaScript and cookies to continue</span></div></div></noscript></div></div><script>(function(){window._cf_chl_opt={cvId: '3',cZone: "www.congress.gov",cType: 'managed',cNounce: '54047',cRay: '89d0ab9b5a98e264',cHash: '904d5acc85610cd',cUPMDTk: "\/118\/bills\/sres726\/BILLS-118sres726ats.xml?__cf_chl_tk=EpD0eA4xdg2NGZsJnWYqzbATLQPPujYcvTxmW3CYV5I-1719944641-0.0.1.1-3518",cFPWv: 'g',cTTimeMs: '1000',cMTimeMs: '390000',cTplV: 5,cTplB: 'cf',cK: "",fa: "\/118\/bills\/sres726\/BILLS-118sres726ats.xml?__cf_chl_f_tk=EpD0eA4xdg2NGZsJnWYqzbATLQPPujYcvTxmW3CYV5I-1719944641-0.0.1.1-3518",md: "IytZjOuSZtNko7XfxR8cAFhee5Jtvx4vGdMFZITDJcA-1719944641-1.1.1.1-lSY27oHB0tmoRABufn4aFPebauYPD19U5LoVQm.iq2RHe3wy0neNRM6obBVWG5Iept07bA.EUrNC8GmJp0BNlfAmeikyoD00hkzxf4S4jz7c0Y3mTAgBQXHcnFMhm9bxrY3MPbmnLvxf60Tvam1cSbtV15i3kyBt2cpVD3PwpuoSORR9QUbPJJccswljqlVGfhkM5SI2cQozNPr1QBe0Bf7645vQMwG4oWP.fsSN8vJpryD3I85ljXvtXRhaOD_ZNNHka6ee6XaLqt5NeV8jEyGH1n.4SMRjuRQLYZlWtvZAgxAzGqusoZlqMRLm6ZLjoVcGh1fCx7Lv3tx_86mVLUWN_d6u5tf5dZJzEKBpGF5wFCXb1_ha4iya.t7QrXWtJu9sVzqAU5EBYVbYtQqcB0FxO43V_umTOijjpctLx865XAO7XBaA6M1sbUBtZLTXvIAlpCmLTw28yu5vA3VDreW3kPazzYiaPXTK6shvADKDBC40bXQZ5N9ZukTvBCqoeTVO1Q33coldVJg4ppgcZhSPihjVCQXciHIFgX.5RabpJfIvp1BgBkPLLm0tFhw02qh_QHUQAbfoXvrwZNC1CrVfQ8rjsfS5xPTQngg92c7I80sUvBGp8Xf0Dspk2Ec3AIiHilkemmURMajdxl5sG6lpqBb0X_ZLTkf1VqCS84Sm5KEPwu6f_.2e4k2FRRbKhYksN.MBOpput2s22T0xQs8U1ezgAYWRqIA7.nZZdOiGT.H8H3.C.d48KdAx_oYyNDCA0cLnLswqkkozbwwzUofSGQMyaPki6uuVKmE1RPIWIJHv7aaRdaoo9I2OoTRwgdFytVDF25N30GVQzpFietXFCfIvjbr8CAJhL1oyY4nDLD2jwGmhvkpLdibj723ryzwZwIM8asTPCT8gG5wGs.wCdV_9ag8UUjqnovS_ZtPaRdTwB9n9n8RAbMmJwGRBHsc4R_2j.ytNjVtwzECW7b_xA6bvkxbTSVmt271CkyAdqRmNBKZCsa9TiVhU_TcmGwkhP6FKkrlf4CzgjQizVMDtUihiiRPxSD43gHq19jua4ki_D54etVcATSbmVYwiVnpYUgHgYTiz_sISK3OYQC2f_Ax1xWgWbX8FVgitny1ZHHojx_gZhUGpj6cHvycGqKadvKELskjowFGPDXrJt.8OXji1KWymfWhgYJA3G6N8et0SQNV7x3Cw8SGM4IAxCHX5M01Wg3bWqSR0PbJSaccp.ePTJ81WNQB.W03Z_YWUTRp06zy8bsSGf5LKWeGRUaAp332Asr4ddDvyqWtDt2Opi8ErOnTAjxGIdh9d9xZimtd_ikbH68Vg4oUAP20MZsSl7KQWTp3nq.aLnecAA.QnzCe5I_G5Jqq1O2PJoEp_zx1GLOEoztmDrgqIe9aOx0.wXcm2aT9VLsj3pPDM2kj3mzGfNw3uzu4ua2HX0f_.jMQDv1olyiC1FjwXL1Ww",mdrd: "OwwxUeV0HUXubVejZzIF9tNjjQHKaklhGuxxLfsD3ec-1719944641-1.1.1.1-Ecf.kgCGFGlRCs0Y.cNI3GQys5yqY4UdPWXq1V7L9RH.tCKixWMM9WKxgIV7VgG9hANbTS88Dia6HaKrP.0quWk6L5632vC8AEHsxkzuMvrR1uLB05fASmNl9jAi3Z_dEg3igipn3aWFN7oQWuj870Xf3l9C2RT0GPdV4l1TKuQtmtdF0E5d9mw0aXV7mgZu1R7j3P6cREV.x6AiR22z460IhUXdlwcgiVMPp0JSJ0kwylJdYmyxYyRp4wJ9nrrZCEmTLj5dKoetqZbail7bov_o_XOvW2qcFo02qwUlrsVz.aE7zFpc3TADy7IQabaU1QKtoQr.DyItNo3dl5S_pY0sciHhc4gBnP5Jn84JFqveHGbYlyq6fInpzNtrYH1uSCjKYAujvGSavdwoTgCOhlY8ZeJhFP2lpKPiWDWibhszF04oKVBPApwbNMWp2RNV2ZU47yTQAh66XkER8uPqiW3_P7ZiWuAeFBDAuDHRsP0EtghyY23XyxXWd_aPhP1sr2SPPeMGOW9aq5cCHww.vSKY1Bi0H_Zz9QGe.jIgYZKPs78JIXCZE7mEfzt6.uEbvbz.qTBO70UbKUrOyA_7N5c_eq5D1_PnGUtcupU_U949Hp_dtN2ArHHsQXW8R.2kb4i7VCQCGRVXQ1qVMvPZVqu.4.izXQY2V42Wab02IbUG4ufRmIJwBKd0msRB4RFE87sQhpPMTQUw7k2BpUxPE4ve22R46fW6eSdvS_0cYL2rl5ZvrXZqcruTDoV5yM_wFDQO3eAdlRyWroQ1GNTQv4yFt0n9IRFfnLROUkeSMQE7DM.hl5yKMCBPkWFQ_g4cG.J_sJh_xTTleymJGoLxGImBAj9TD0maABa1pZ5izOEglkS0X7wHijAWnet0KyRehecN0qDvpC2aZWPuCqjGalTb7QM5Elaf_bpF1WXHD92RoWiAa91v8htzohlk2aTbUO_q8sQHbiWB0kuKxpjpsd.Eog7adNjoDyUIjo_S_XFm6Pk3anNjyDW9s5vrOuQ75.LkEv1R33xAllMAgeI_7WQFR.qRVP5gU.eSEuGWSmPyXSmIzwX8.BZGJUhn8b4xltNVjBI7.5Jtai4Atd3THvjhsPaqAMLV9A6qQWpapKI.LjgH1221LRxGObmyl2GomEhwI9m2vml7OXUGd_i_Jd7Bjzcj_mN6Y2krBjiHyha7pgUKNYbdDZe8kd2WcQdSV9_ykWF6wkCPF1nyMZd3HUdgK6UB4EWeYUEqo5CxK1Nfj_5jvwCZf7KJvx_LuOd2BCVDu7WJY4SPM94V5WL2r67pAueMymFRcwJGrX0Bjulv5O3G16OzwrAMFV_U5HPVrD0YQnEq5JcLddXL.O6M1uMI2iWXVa87Bvx4T6LgQMle91EgFIlGJnOyHqfwp_5OEFQa0hA96ZJAd5LxkOVoA4rbNhClhg1kCczhrrh6OQZxZISJXFwC3P_28TLYLhhrbwBt3uFJTnIcl.naBc.YmKblveZPPQo.fA0_lwzj5x3vvEPSs7NcrOIPR6cT8WsVK3Pd21vjCc1yokUsZ5KhtQZKBuNPC.9NpWFqN0tGpNlwB9W_23JdIfQy6gXeY3Pdb4boTEXrZuCCKv1YHNllhqIZwTY8pJVv8tNA7O6xiFjgrYxQxbDFGEGuzwJdxxDrLcf3KTHNNKWGo_5xHR46kXvLG_BfwUhss2n0Igt49d99tQhh8uUqO2B8n8M2xnj4OEkaNdpWF_rhByK16gtbnDcWtscG6W_0JKlRd0wjmoJR2W3TYDknl3aTTY9mFH6AmIOMFBcwqVAuyCLZhEIZ4e78M9K4gjPtXVjPtQ7aOpz98kVJHkIBGYBSwmCU7L2ekds.9jZjak5yLZtx4B5MolC0DGeGjVPymPQgAtu.wYB5HLli8N0On37kE3jwANTo",cRq: {ru: 'aHR0cHM6Ly93d3cuY29uZ3Jlc3MuZ292LzExOC9iaWxscy9zcmVzNzI2L0JJTExTLTExOHNyZXM3MjZhdHMueG1s',ra: 'Q29uZ3Jlc3MtV2lraS8wLjQ=',rm: 'R0VU',d: 'yh1Ga3gvrd2r6pic02OuMOBoOxh+xqrxKIW59cdLZRu6Qn2x2blYvOBvWTLogOVumxxtwjvWM2ByblJOtIBubN38BWXYQcqesDVLCEkZkzRYldXrF1d9QMATQzjXZ/LjD3Gnb62mUwFNVqYJC8/huHny/qqr4DFY04J9H7pMNabEFX7rNU/fbShedG1NK9VbkJ9f3B0Q7ksf8AeifQiJq1MQkC7+SSI9CVhbas/SKacyRLvgI/QmLjIYzczXjZO1JKYbw9jrAFsFpgWjDNP+O9kobPvT5YYbBRe6bab+oVm5HSi7yDy6veZXMHAhxfPapB8XHz5lBqiYCCBioY+B/l3s8SrJXZ8HOTAungP8JZiL6K/Nx5FulpzjofJmZA1vRrjwnmcrVeGnlQRfFSUhx2Dnwvx++KBn8xHPduaWBRnotVgFfFXrB1hAlu7MJ/bOba2ib4W+JmIZBS/7Ybg8unJ3jK7mWqbDbN1fQ17HOlc=',t: 'MTcxOTk0NDY0MS4wMDAwMDA=',cT: Math.floor(Date.now() / 1000),m: 'CBjakUMhMhngM8+POuynECiPXoqgWYVyzysZNDpbzng=',i1: 'zN6o/aorUZxjjM0ky/32BA==',i2: 'r9CdUTlcFJlTIL4lo1wDrg==',zh: 'P/37dfGD+KUZFm0qKuE3N/q0gn3BGPttWTswwTYBAD8=',uh: 'FVHG7QmCIZ0ivo719LfrxFm2W3GVAQSMC1lFhLfL6/s=',hh: 'cTbG2zct7/9cr7abEWSg59sInRszec2mTWq3jspVNx4=',}};var cpo = document.createElement('script');cpo.src = '/cdn-cgi/challenge-platform/h/g/orchestrate/chl_page/v1?ray=89d0ab9b5a98e264';window._cf_chl_opt.cOgUHash = location.hash === '' && location.href.indexOf('#') !== -1 ? '#' : location.hash;window._cf_chl_opt.cOgUQuery = location.search === '' && location.href.slice(0, location.href.length - window._cf_chl_opt.cOgUHash.length).indexOf('?') !== -1 ? '?' : location.search;if (window.history && window.history.replaceState) {var ogU = location.pathname + window._cf_chl_opt.cOgUQuery + window._cf_chl_opt.cOgUHash;history.replaceState(null, null, "\/118\/bills\/sres726\/BILLS-118sres726ats.xml?__cf_chl_rt_tk=EpD0eA4xdg2NGZsJnWYqzbATLQPPujYcvTxmW3CYV5I-1719944641-0.0.1.1-3518" + window._cf_chl_opt.cOgUHash);cpo.onload = function() {history.replaceState(null, null, ogU);}}document.getElementsByTagName('head')[0].appendChild(cpo);}());</script></body></html> Request failed with status code 429
AxiosError: Request failed with status code 429
at settle (/Users/ryanparker/GitHub/collector/node_modules/axios/lib/core/settle.js:19:12)
at IncomingMessage.handleStreamEnd (/Users/ryanparker/GitHub/collector/node_modules/axios/lib/adapters/http.js:589:11)
at IncomingMessage.emit (node:events:531:35)
at endReadableNT (node:internal/streams/readable:1696:12)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
When attempting to download all the texts in all formats (xml, html, pdf), congress.gov will eventually block requests with a 429 response and a javascript challenge.
I noticed the challenge appeared after about 3000 downloads.
e.g. fetch the download links from the API:
Response:
Then go through and attempt to download all the available files from
https://www.congress.gov/...
. After about 3000 requests, you'll start to see the 429 response.The 429 response looks like this: