Closed tusharsinghsoam closed 1 year ago
palewire
commented
1 year ago Hmm. If that response isn't "ok" I'm guessing something failed in the download, yes? Is it possible your internet went out? does this problem consistently persist? If so, maybe the BLS changed the URL we are downloading from?
palewire
commented
1 year ago Looks like we are getting blocked from downloading by bot detection added to the BLS website.
<!DOCTYPE HTML> <html lang="en-us"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <title>Access Denied</title> </head> <style type="text/css"> .centerDiv { width: 60%; height:200px; margin: 0 auto; background-color:#FFFFFF ; } </style> <body> <div class="centerDiv"> <h1><a href=https://www.bls.gov><!--img src="/apology_objects/images/01.jpg" border="0"--></a><span style="font-family: Times, serif; color: #990000; font-size: 38px;">Bureau of Labor Statistics</span></h1> <h2>Access Denied</h2> <p>The BLS is committed to providing data promptly and according to established schedules. Automated retrieval programs (commonly called "robots" or "bots") can cause delays and interfere with other customers' timely access to information. Therefore, bot activity that doesn't conform to BLS usage policy is prohibited.</p> <p>We apologize for any inconvenience. If you believe we have made an error, please <a href=https://data.bls.gov/forms/opb.htm?akamai-0.36aa2c17.1681487650.919dabe>contact us</a>.</p> <p>Please contact your administrator with the error code: 0.36aa2c17.1681487650.919dabe</p> </div> </body> </html>I've attempted to patch this bug, and better test for its future recurrence. Please upgrade to version 1.0.18 and try again. Assuming it's fixed, I'll close this ticket. If you still have the problem, please speak up.
tusharsinghsoam
commented
1 year ago Hi Ben,
It has fixed the issue. I am able to run "cpi.update()" with the latest version (1.0.18). Thank you so much.
palewire
commented
1 year ago That's great news. I'm glad to hear it.
On Fri, Apr 14, 2023, at 1:16 PM, Tushar Singh Soam wrote:
Hi Ben,
It has fixed the issue. I am able to run "cpi.update()" with the latest version (1.0.18). Thank you so much.
— Reply to this email directly, view it on GitHub https://github.com/palewire/cpi/issues/74#issuecomment-1509199008, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAACOCL6UP3PBAJ63Y75IH3XBGWCVANCNFSM6AAAAAAW6NIYL4. You are receiving this because you modified the open/close state.Message ID: @.***>
Following is the traceback for the issue I faced during cpi.update():
Version of cpi used: 1.0.17
AssertionError Traceback (most recent call last) Cell In [40], line 1 ----> 1 cpi.update()
File ~\AppData\Local\Programs\Python\Python311\Lib\site-packages\cpi__init__.py:163, in update() 157 def update(): 158 """ 159 Updates the Consumer Price Index dataset at the core of this library. 160 161 Requires an Internet connection. 162 """ --> 163 Downloader().update()
File ~\AppData\Local\Programs\Python\Python311\Lib\site-packages\cpi\download.py:75, in Downloader.update(self) 73 # Download the TSVs 74 logger.debug(f"Downloading {len(self.FILE_LIST)} files from the BLS") ---> 75 [self.get_tsv(file) for file in self.FILE_LIST] 77 # Insert the TSVs 78 logger.debug("Loading data into SQLite database")
File ~\AppData\Local\Programs\Python\Python311\Lib\site-packages\cpi\download.py:75, in(.0)
73 # Download the TSVs
74 logger.debug(f"Downloading {len(self.FILE_LIST)} files from the BLS")
---> 75 [self.get_tsv(file) for file in self.FILE_LIST]
77 # Insert the TSVs
78 logger.debug("Loading data into SQLite database")
File ~\AppData\Local\Programs\Python\Python311\Lib\site-packages\cpi\download.py:109, in Downloader.get_tsv(self, file) 107 tsv_path = self.get_data_dir() / f"{file}.tsv" 108 response = requests.get(url) --> 109 assert response.ok 110 with open(tsv_path, "w") as fp: 111 fp.write(response.text)
AssertionError: