search

HelloZeroNet / ZeroNet

ZeroNet - Decentralized websites using Bitcoin crypto and BitTorrent network
https://zeronet.io
Other
18.35k stars 2.27k forks source link

siteCmd command did not work #1979

Open nulcgit opened 5 years ago

nulcgit commented 5 years ago

Step 1: Please describe your environment

Step 2: Describe the problem:

bitcoren@name:~/ZeroNet-py3$ python3 zeronet.py siteCmd 1KoLHoz1nSddbuf6HbfP19F7Ht7xNBSa6m fileGet content.json
- Starting ZeroNet...
- Libsecpk256k1 loaded
- Version: 0.7.0 r4053, Python 3.6.7 (default, Oct 22 2018, 11:32:17) 
[GCC 8.2.0], Gevent: 1.4.0
FileServer IPv6 not supported: [Errno 101] Network is unreachable
- Unhandled exception: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
  File "zeronet.py", line 23, in main
    main.start()
  File "/home/bitcoren/ZeroNet-py3/src/main.py", line 515, in start
    actions.call(config.action, action_kwargs)
  File "/home/bitcoren/ZeroNet-py3/src/main.py", line 111, in call
    back = func(**kwargs)
  File "/home/bitcoren/ZeroNet-py3/src/main.py", line 333, in siteCmd
    res = json.loads(ws.recv())
  File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
rllola commented 5 years ago

I have been able to reproduce it.

- Starting ZeroNet...
- Libsecpk256k1 loaded
- Version: 0.7.0 r4056, Python 3.6.7 (default, Oct 22 2018, 11:32:17) 
[GCC 8.2.0], Gevent: 1.4.0
FileServer IPv6 not supported: [Errno 101] Network is unreachable
- Unhandled exception: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
  File "zeronet.py", line 23, in main
    main.start()
  File "/home/lola/Workspace/ZeroNet/Dev/src/main.py", line 515, in start
    actions.call(config.action, action_kwargs)
  File "/home/lola/Workspace/ZeroNet/Dev/src/main.py", line 111, in call
    back = func(**kwargs)
  File "/home/lola/Workspace/ZeroNet/Dev/src/main.py", line 333, in siteCmd
    res = json.loads(ws.recv())
  File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Did you do first python3 zeronet.py siteDownload 1KoLHoz1nSddbuf6HbfP19F7Ht7xNBSa6m to download the site?

EDIT: I also have this error in the log 

[02:22:51] Ui.UiServer Wrapper key not found: 8fed24854d187a84baf4364e1795f05c84b234ec92e90f9899bc369523292cdf
[02:22:51] Ui.UiServer Error 403:
nulcgit commented 5 years ago

Yes, at first I downloaded the site in the browser.

rllola commented 5 years ago

When I download the site from the web interface I don't get this error... Only when I do it with the command... :thinking:

rllola commented 5 years ago

What may be happening is that your site is not properly downloaded...

Try to do check files and your command again.

nulcgit commented 5 years ago 
bitcoren@name:~/ZeroNet-py3$ python3 zeronet.py siteCmd 1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D siteList
- Starting ZeroNet...
- Libsecpk256k1 loaded
- Version: 0.7.0 r4056, Python 3.6.7 (default, Oct 22 2018, 11:32:17) 
[GCC 8.2.0], Gevent: 1.4.0
FileServer IPv6 not supported: [Errno 101] Network is unreachable
- Unhandled exception: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
  File "zeronet.py", line 23, in main
    main.start()
  File "/home/bitcoren/ZeroNet-py3/src/main.py", line 515, in start
    actions.call(config.action, action_kwargs)
  File "/home/bitcoren/ZeroNet-py3/src/main.py", line 111, in call
    back = func(**kwargs)
  File "/home/bitcoren/ZeroNet-py3/src/main.py", line 333, in siteCmd
    res = json.loads(ws.recv())
  File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
rllola commented 5 years ago

Could you check your log file to see if your have also an Wrapper key not found error ?

nulcgit commented 5 years ago 
[2019-04-11 04:01:12,299] DEBUG    Site:1HeLLo..Tf3D Need connections: 6, Current: 8, Total: 672
[2019-04-11 04:01:13,305] DEBUG    Site:1KoLHo..Sa6m Need connections: 1, Current: 1, Total: 1
[2019-04-11 04:01:13,308] DEBUG    ConnServer Conn#91 204.44.72.141 [?] > Connecting... (site: None)
[2019-04-11 04:01:14,192] DEBUG    ConnServer Conn#91 204.44.72.141 [v2] > Closing connection: Removing, waiting_requests: 0, sites: 1, buff: 0...
[2019-04-11 04:01:14,195] DEBUG    Site:1KoLHo..Sa6m Tracker announce result: zero://204.44.72.141:10398 (sites: 4, new peers: 0, add: ['ipv4']) in 0.887s
[2019-04-11 04:01:14,310] DEBUG    Site:1Name2..hM9F Need connections: 6, Current: 6, Total: 741
[2019-04-11 04:01:15,315] DEBUG    Site:1uPDaT..NACf Need connections: 3, Current: 4, Total: 14
[2019-04-11 04:01:15,419] WARNING  Site:1uPDaT..NACf Tracker http://open.acgnxtracker.com:80/announce announce failed: ConnectionResetError: [Errno 104] Connection reset by peer in SiteAnnouncer.py line 214 > SiteAnnouncer.py line 349 > Sit$
[2019-04-11 04:01:15,420] DEBUG    Site:1uPDaT..NACf Tracker failed, skipping to next one...
[2019-04-11 04:01:16,319] DEBUG    ConnServer Site announce tracker done in 4.021s, sleeping for 129.313s...
[2019-04-11 04:01:24,996] DEBUG    ChartCollector Global collectors done in 0.004s
[2019-04-11 04:01:24,998] DEBUG    ChartCollector Global collectors inserted in 0.001s
[2019-04-11 04:01:35,271] DEBUG    Ui.UiServer 127.0.0.1 - - [2019-04-11 04:01:35] "GET /Websocket?wrapper_key=27385e594c2d4a559d8600aef95c74bdc0c33b89a151cd604b47cf5156f34620 HTTP/1.1" 101 129 0.002576
nulcgit commented 5 years ago 
[2019-04-11 17:08:47,186] INFO     Site:1KoLHo..Sa6m Publishing content.json to 5/1 peers (connected: 1) diffs: [] (0.00k)...
[2019-04-11 17:08:47,201] DEBUG    Ui.UiServer 127.0.0.1 - - [2019-04-11 17:08:47] "GET /Websocket?wrapper_key=1c6862ed7cb7afa258f801b207085f5626a45829d2a3edb0895f8ebacd32ad30 HTTP/1.1" 101 129 0.047000
[2019-04-11 17:08:47,295] DEBUG    Site:1KoLHo..Sa6m Websocket send error: WebSocketError: Socket is dead in UiWebsocket.py line 214 > websocket.py line 352
[2019-04-11 17:08:47,295] INFO     Site:1KoLHo..Sa6m [OK] 194.1.237.140:13216: Thanks, file content.json updated! 1/5
[2019-04-11 17:08:47,295] DEBUG    RateLimit Added to queue (15.00s left): 49 update 1KoLHoz1nSddbuf6HbfP19F7Ht7xNBSa6m content.json 
[2019-04-11 17:08:47,311] INFO     Site:1KoLHo..Sa6m Published content.json to 1 peers, publishing to 5 more peers in the background
[2019-04-11 17:08:47,311] DEBUG    Site:1KoLHo..Sa6m Websocket send error: WebSocketError: Socket is dead in UiWebsocket.py line 214 > websocket.py line 352
[2019-04-11 17:08:47,311] DEBUG    Site:1KoLHo..Sa6m Websocket send error: WebSocketError: Socket is dead in UiWebsocket.py line 214 > websocket.py line 352
[2019-04-11 17:09:02,323] DEBUG    RateLimit Calling: 49 update 1KoLHoz1nSddbuf6HbfP19F7Ht7xNBSa6m content.json
[2019-04-11 17:09:43,344] DEBUG    ConnServer Conn#55 77.249.123.145 [v2] > Closing connection: MessageLoop ended (closed: False), waiting_requests: 0, sites: 1, buff: 0...
HelloZeroNet commented 5 years ago

For some reasons the client returns not valid json result. I have added a logging to this case in Rev4057: https://github.com/HelloZeroNet/ZeroNet/commit/9f5600b7f7a3c545ee3dc46305d717437a58b645 Please update your client and see whats the error message (the value of res_raw contains the actually returned value) It should look like:

$ python3 zeronet.py --silent siteCmd 1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D siteList
{'error': 'Invalid result: Expecting value: line 1 column 1 (char 0)', 'res_raw': 'Error...
nulcgit commented 5 years ago 
bitcoren@name:~/ZeroNet-py3$ python3 zeronet.py --silent siteCmd 1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D siteList
FileServer IPv6 not supported: [Errno 101] Network is unreachable
{'error': 'Invalid result: Expecting value: line 1 column 1 (char 0)', 'res_raw': ''}

bitcoren@name:~/ZeroNet-py3$ python3 zeronet.py siteCmd 1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D siteList
- Starting ZeroNet...
- Libsecpk256k1 loaded
- Version: 0.7.0 r4057, Python 3.6.7 (default, Oct 22 2018, 11:32:17) 
[GCC 8.2.0], Gevent: 1.4.0
FileServer IPv6 not supported: [Errno 101] Network is unreachable
- Unhandled exception: [Errno 32] Broken pipe
Traceback (most recent call last):
  File "zeronet.py", line 23, in main
    main.start()
  File "/home/bitcoren/ZeroNet-py3/src/main.py", line 520, in start
    actions.call(config.action, action_kwargs)
  File "/home/bitcoren/ZeroNet-py3/src/main.py", line 111, in call
    back = func(**kwargs)
  File "/home/bitcoren/ZeroNet-py3/src/main.py", line 333, in siteCmd
    res_raw = ws.recv()
  File "/usr/local/lib/python3.6/dist-packages/websocket/_core.py", line 313, in recv
    opcode, data = self.recv_data()
  File "/usr/local/lib/python3.6/dist-packages/websocket/_core.py", line 330, in recv_data
    opcode, frame = self.recv_data_frame(control_frame)
  File "/usr/local/lib/python3.6/dist-packages/websocket/_core.py", line 357, in recv_data_frame
    self.send_close()
  File "/usr/local/lib/python3.6/dist-packages/websocket/_core.py", line 390, in send_close
    self.send(struct.pack('!H', status) + reason, ABNF.OPCODE_CLOSE)
  File "/usr/local/lib/python3.6/dist-packages/websocket/_core.py", line 253, in send
    return self.send_frame(frame)
  File "/usr/local/lib/python3.6/dist-packages/websocket/_core.py", line 278, in send_frame
    l = self._send(data)
  File "/usr/local/lib/python3.6/dist-packages/websocket/_core.py", line 448, in _send
    return send(self.sock, data)
  File "/usr/local/lib/python3.6/dist-packages/websocket/_socket.py", line 157, in send
    return _send()
  File "/usr/local/lib/python3.6/dist-packages/websocket/_socket.py", line 139, in _send
    return sock.send(data)
  File "/usr/local/lib/python3.6/dist-packages/gevent/_socket3.py", line 439, in send
    return _socket.socket.send(self._sock, data, flags)
BrokenPipeError: [Errno 32] Broken pipe
[2019-04-11 19:50:15,579] DEBUG    Ui.UiServer 127.0.0.1 - - [2019-04-11 19:50:15] "GET / HTTP/1.1" 200 4417 0.002488
[2019-04-11 19:50:34,924] DEBUG    Ui.UiServer 127.0.0.1 - - [2019-04-11 19:50:34] "GET /Websocket?wrapper_key=e5b074bdf2fb3b760ebd024bdfd29edb6456b35e3f562c$
[2019-04-11 19:50:52,656] DEBUG    ChartCollector Global collectors done in 0.004s
[2019-04-11 19:50:52,658] DEBUG    ChartCollector Global collectors inserted in 0.001s
[2019-04-11 19:51:02,731] DEBUG    Ui.UiServer 127.0.0.1 - - [2019-04-11 19:51:02] "GET /Websocket?wrapper_key=27385e594c2d4a559d8600aef95c74bdc0c33b89a151cd$
[2019-04-11 19:51:13,539] DEBUG    BroadcastServer Sending discover request (force: False)
[2019-04-11 19:51:13,544] DEBUG    ConnServer Conn#131 178.170.68.153 [?] > Connecting... (site: None)
[2019-04-11 19:51:13,549] DEBUG    BroadcastServer Broadcast using ips ['194.1.237.140'] on port 1544: discoverRequest
[2019-04-11 19:51:13,550] DEBUG    BroadcastServer Got from ('194.1.237.140', 37561): discoverRequest
[2019-04-11 19:51:13,849] DEBUG    ConnServer Conn#131 178.170.68.153 [v2] > Closing connection: Removing, waiting_requests: 0, sites: 1, buff: 0...
[2019-04-11 19:51:13,850] DEBUG    Site:1HeLLo..Tf3D Tracker announce result: zero://178.170.68.153:17127 (sites: 4, new peers: 0, add: ['ipv4']) in 0.306s
[2019-04-11 19:51:14,554] DEBUG    Site:1KoLHo..Sa6m Need connections: 2, Current: 2, Total: 2
[2019-04-11 19:51:15,561] DEBUG    Site:1Name2..hM9F Need connections: 6, Current: 7, Total: 520
[2019-04-11 19:51:16,568] DEBUG    Site:1uPDaT..NACf Need connections: 3, Current: 3, Total: 25
[2019-04-11 19:51:17,573] DEBUG    ConnServer Site announce tracker done in 4.035s, sleeping for 129.298s...
[2019-04-11 19:52:52,533] DEBUG    Db:ContentDb Optional size: 0.0MB/2200.7MB, Need delete: -2200.7MB
[2019-04-11 19:52:53,083] DEBUG    SiteManager Saved sites in 0.02s (generate: 0.00s, write: 0.02s)
[2019-04-11 19:52:53,084] DEBUG    SiteManager Updated merger sites in 0.000s
[2019-04-11 19:53:26,880] DEBUG    ConnServer Conn#132 185.142.236.207 [?] > Connecting... (site: None)
[2019-04-11 19:53:27,163] DEBUG    ConnServer Conn#132 185.142.236.207 [v2] > Closing connection: Removing, waiting_requests: 0, sites: 1, buff: 0...
[2019-04-11 19:53:27,164] DEBUG    Site:1HeLLo..Tf3D Tracker announce result: zero://185.142.236.207:10443 (sites: 4, new peers: 0, add: ['ipv4']) in 0.284s
[2019-04-11 19:53:27,880] DEBUG    Site:1KoLHo..Sa6m Need connections: 3, Current: 2, Total: 3
[2019-04-11 19:53:27,880] DEBUG    ConnServer Conn#133 194.1.237.140 [?] > Connecting... (site: Site 1KoLHo..Sa6m)
[2019-04-11 19:53:27,885] DEBUG    ConnServer Conn#133 194.1.237.140 [?] > Closing connection: 194.1.237.140 Connect error: ConnectionRefusedError: [Errno 11$
[2019-04-11 19:53:27,886] DEBUG    Site:1KoLHo..Sa6m Connected before: 2, after: 2. Check site: True.
[2019-04-11 19:53:28,887] DEBUG    Site:1Name2..hM9F Need connections: 6, Current: 7, Total: 524
[2019-04-11 19:53:29,893] DEBUG    Site:1uPDaT..NACf Need connections: 3, Current: 3, Total: 25
[2019-04-11 19:53:30,900] DEBUG    ConnServer Site announce tracker done in 4.023s, sleeping for 129.311s...

siteList should return a list of sites. Why doesn't the command do this?

HelloZeroNet commented 5 years ago

I have added changes to Rev4059 that should help to find whats the problem.

Now it should print - Connecting to ws://127.0.0.1:43110/Websocket?wrapper_key=... Does the wrapper_key value there matches the wrapper_key in your data/sites.json within the "1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D": { key?

If the error still not found: Please try to run: wsdump.py -vv ws://127.0.0.1:43110/Websocket?wrapper_key=... Then, if connected: {"cmd": "siteList", "id": 1}

nulcgit commented 5 years ago

I renamed the plugin UiPassword to disabled-UiPassword and everything works. Please make local commands to work with UiPassword. Many thanks for the support.

purplesyringa commented 5 years ago

Why did you close this? A big is still a bug even if you have found the reason.

HelloZeroNet commented 5 years ago

Not sure if white-listing 127.0.0.1 requests for ui_password by default is a good idea. Maybe adding a nonce for api calls or just let access /Websocket without password as it needs the wrapper_key anyway

nulcgit commented 5 years ago

siteCmd command did not work in version 4080 with "UiPassword" again

bitcoren@ubuntu:/hdd/ZeroNet-py3$ python3 zeronet.py siteCmd 1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D siteList
- Starting ZeroNet...
- Libsecpk256k1 loaded
- Version: 0.7.0 r4080, Python 3.6.7 (default, Oct 22 2018, 11:32:17) 
[GCC 8.2.0], Gevent: 1.4.0
FileServer IPv6 not supported: [Errno 101] Network is unreachable
- Connecting to ws://192.168.1.21:43110/Websocket?wrapper_key=becccc9b458bb3038d1d39b4d509f18dad0583a7d94cb9265ed24bb3aa2ab579
- Unhandled exception: [Errno 32] Broken pipe
Traceback (most recent call last):
  File "zeronet.py", line 23, in main
    main.start()
  File "/hdd/ZeroNet-py3/src/main.py", line 529, in start
    actions.call(config.action, action_kwargs)
  File "/hdd/ZeroNet-py3/src/main.py", line 111, in call
    back = func(**kwargs)
  File "/hdd/ZeroNet-py3/src/main.py", line 338, in siteCmd
    res_raw = ws.recv()
  File "/home/bitcoren/.local/lib/python3.6/site-packages/websocket/_core.py", line 313, in recv
    opcode, data = self.recv_data()
  File "/home/bitcoren/.local/lib/python3.6/site-packages/websocket/_core.py", line 330, in recv_data
    opcode, frame = self.recv_data_frame(control_frame)
  File "/home/bitcoren/.local/lib/python3.6/site-packages/websocket/_core.py", line 357, in recv_data_frame
    self.send_close()
  File "/home/bitcoren/.local/lib/python3.6/site-packages/websocket/_core.py", line 390, in send_close
    self.send(struct.pack('!H', status) + reason, ABNF.OPCODE_CLOSE)
  File "/home/bitcoren/.local/lib/python3.6/site-packages/websocket/_core.py", line 253, in send
    return self.send_frame(frame)
  File "/home/bitcoren/.local/lib/python3.6/site-packages/websocket/_core.py", line 278, in send_frame
    l = self._send(data)
  File "/home/bitcoren/.local/lib/python3.6/site-packages/websocket/_core.py", line 448, in _send
    return send(self.sock, data)
  File "/home/bitcoren/.local/lib/python3.6/site-packages/websocket/_socket.py", line 157, in send
    return _send()
  File "/home/bitcoren/.local/lib/python3.6/site-packages/websocket/_socket.py", line 139, in _send
    return sock.send(data)
  File "/home/bitcoren/.local/lib/python3.6/site-packages/gevent/_socket3.py", line 439, in send
    return _socket.socket.send(self._sock, data, flags)
BrokenPipeError: [Errno 32] Broken pipe