HTTP Error 401: UNAUTHORIZED

[nrenich]

I am receiving this error on multiple rigs while building: (same results for register as well)

Traceback (most recent call last):
  File "dataserv-client", line 10, in <module>
  File "E:\dataserv-client\dataserv_client\cli.py", line 153, in main
  File "E:\dataserv-client\dataserv_client\api.py", line 135, in poll
  File "E:\dataserv-client\dataserv_client\api.py", line 115, in ping
  File "E:\dataserv-client\dataserv_client\messaging.py", line 110, in ping
  File "E:\dataserv-client\dataserv_client\messaging.py", line 59, in _url_query
  File "E:\dataserv-client\dataserv_client\messaging.py", line 45, in _url_query
  File "C:\Python34\lib\urllib\request.py", line 161, in urlopen
  File "C:\Python34\lib\urllib\request.py", line 469, in open
  File "C:\Python34\lib\urllib\request.py", line 579, in http_response
  File "C:\Python34\lib\urllib\request.py", line 507, in error
  File "C:\Python34\lib\urllib\request.py", line 441, in _call_chain
  File "C:\Python34\lib\urllib\request.py", line 587, in http_error_default
urllib.error.HTTPError: HTTP Error 401: UNAUTHORIZED

[F483]

This may be because your clock is not set correctly, can you check that they are?

[lucvanbraekel]

Same 401 error here on multiple rigs, while polling and also when trying to re-register with an empty config.json. Clocks are perfectly synchronized with nptd.

[F483]

Will be looking into this very soon. The require more logging server side to expose the problem, as I cannot reproduce the error. See https://github.com/Storj/dataserv/issues/41 for issue this depends on.

[nrenich]

Verified clocks are synced.

[Luzr]

I also have this error on a 5TB build.

[F483]

I added some logging, especially for authentication errors and it seems to be only invalid dates that are signed, we will see if there are others over item.

As to why the majority seem to be off by about 18 hours, I do not know. Suggestions?

@Luzr @nrenich @lucvanbraekel can you test again and tell me when you did so, so I can find the logs or even better message me in slack before you do.

2015-09-03 17:57:28,616 WARNING dataserv.Farmer 81: Header date to old! 64801 >= 15
2015-09-03 17:57:29,315 WARNING dataserv.Farmer 81: Header date to old! 10803 >= 15
2015-09-03 17:57:29,617 WARNING dataserv.Farmer 81: Header date to old! 61204 >= 15
2015-09-03 17:57:29,725 WARNING dataserv.Farmer 81: Header date to old! 64801 >= 15
2015-09-03 17:57:29,914 WARNING dataserv.Farmer 81: Header date to old! 61193 >= 15
2015-09-03 17:57:30,013 WARNING dataserv.Farmer 81: Header date to old! 3602 >= 15
2015-09-03 17:57:30,025 WARNING dataserv.Farmer 81: Header date to old! 64803 >= 15

[Luzr]

Just tested at 5:11 PM CDT Chicago (time from Google).

Command ran: dataserv-client --url=http://status.driveshare.org:5000 --max_size 6818260582400 --store_path D:\Storj build

[heunland]

I just ran this register command at 5:11pm Mexico City time: I:\storj_beta_B\dataserv_client_203\dataserv-client>dataserv-client.exe --url ht tp://status.driveshare.org:5000 register Traceback (most recent call last): File "dataserv-client", line 10, in File "E:\dataserv-client\dataserv_client\cli.py", line 153, in main File "E:\dataserv-client\dataserv_client\api.py", line 72, in register File "E:\dataserv-client\dataserv_client\messaging.py", line 100, in register File "E:\dataserv-client\dataserv_client\messaging.py", line 59, in _url_query File "E:\dataserv-client\dataserv_client\messaging.py", line 45, in _url_query

File "C:\Python34\lib\urllib\request.py", line 161, in urlopen File "C:\Python34\lib\urllib\request.py", line 469, in open File "C:\Python34\lib\urllib\request.py", line 579, in http_response File "C:\Python34\lib\urllib\request.py", line 507, in error File "C:\Python34\lib\urllib\request.py", line 441, in _call_chain File "C:\Python34\lib\urllib\request.py", line 587, in http_error_default urllib.error.HTTPError: HTTP Error 401: UNAUTHORIZED

[lucvanbraekel]

I have a script running that starts polling in a 60 second loop, it tries every 60 seconds and fails every time with a 401 error.

It strikes me that the time differences in your logging are always more or less multiples of 3600 seconds. Essentially 1 hour, 17 hours and 18 hours in your quoted log.

[jdogresorg]

I also just ran this command and am still getting 401 errors.... UTC -8:00

[jjohnson@CryptoDev:/] date
Thu Sep  3 15:27:46 PDT 2015
[jjohnson@CryptoDev:/] dataserv-client --url http://status.driveshare.org:5000 register
Traceback (most recent call last):
  File "/usr/local/bin/dataserv-client", line 10, in <module>
    cli.main(sys.argv[1:])
  File "/usr/local/lib/python3.4/site-packages/dataserv_client/cli.py", line 153, in main
    return getattr(client, command_name)(**arguments)
  File "/usr/local/lib/python3.4/site-packages/dataserv_client/api.py", line 72, in register
    registered = self.messenger.register(payout_address)
  File "/usr/local/lib/python3.4/site-packages/dataserv_client/messaging.py", line 100, in register
    self.auth_address(), payout_addr
  File "/usr/local/lib/python3.4/site-packages/dataserv_client/messaging.py", line 59, in _url_query
    raise e  # pragma: no cover
  File "/usr/local/lib/python3.4/site-packages/dataserv_client/messaging.py", line 45, in _url_query
    response = urllib.request.urlopen(req)
  File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 161, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 469, in open
    response = meth(req, response)
  File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 579, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 507, in error
    return self._call_chain(*args)
  File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 441, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 587, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 401: UNAUTHORIZED`

[lucvanbraekel]

Could it have something to do with the timezone setting on my machine?

/etc/timezone: US/Central

date: Thu Sep 3 17:31:56 CDT 2015

[heunland]

I am in the same timezone as lucvanbraekel (UTC -6:00) don´t know if that has anything to do with this problem though

[littleskunk]

yes it is the timezone. Server is running with UTC -5:00 but we are not. Set the Windows time to the same time and register is possible.

[lucvanbraekel]

I have another rig running in timezone "Europe/Brussels" (CEST) and there everything runs fine without any 401 error.

[littleskunk]

header_date = email.utils.formatdate( timeval=time.mktime(datetime.datetime.now().timetuple()), localtime=True, usegmt=True)

Windows: Fri, 04 Sep 2015 01:10:38 +0200 Linux: Fri, 04 Sep 2015 07:07:26 +0800

Both have the same time but the server is parsing it wrong.

[nrenich]

Confirmed work around for me. I switched my rigs to UTC -5 and they're up and running. No need to re-register either, it just worked. Thanks @littleskunk

[heunland]

same here, workaround works great for me. Switching clock to UTC-5 did the trick.

[littleskunk]

@fabe not sure if my bugfix will work. please check it carefully. on my server i was able to pass the time validation but after that i get invalide address. looks like my server is not clean but i hope i fixed the bug.

[Luzr]

I also changed my timezone to -5 and it seems to be working.

[lucvanbraekel]

I tried some things. First, I changed my timezone to UTC. I kept getting the 401 errors. Then, I changed the timezone from US/Central to US/Eastern. No 401 error, Farmer not found, after re-registering everything is working perfect. But I would prefer having a correct timezone on my system, either the correct local time or UTC.

[littleskunk]

Bugfix ready. Unit test passed. Fabe has to review it and restart his server. Dont forget to set your time back if he restarts his server.

[nrenich]

It would be great if a Dev team member could post in #general when the fix and reboot will happen so that we can have a smooth transition back to our timezone of choice.

[F483]

I added the fix by @littleskunk It seems the time zones were indeed not handled correctly. Could everyone try again and report back if its fixed or not? @nrenich @lucvanbraekel @Luzr @heunland @jdogresorg

[lucvanbraekel]

It works! Problem solved, no more 401 errors.

[F483]

thanks @littleskunk any everyone else for the fantastic help!

[heunland]

polling after switching back to my correct time zone without errors! problem solved!