3.9.3.1986 host not found error

My Remarkable 2 came with 3.0 out of the box, I updated it to 3.2.2.1581 with remarkable-update.

I just now updated it through remarkable-update to 3.2.3.1595 then to 3.5.2.1807 and finally to 3.9.3.1986 in steps in one sitting.

From the docs I remembered running Reenabling after a system update, so I ran this and after a reboot tried to pair my remarkable 2 with the rmfakecloud instance I have running with its existing account. (All worked just fine on 3.2.2.1581, haven't touched the docker or server setup since) I requested a one-time code from my rmfakecloud instance and entered it to pair my device, but alas, I was greeted with "Host not found" after entering the code.

So, I tried to run the full automatic installer rmfakecloud-proxy script again. Restarted both the RM2 and my hosted docker instance of rmfakecloud, still no luck. I did notice the ~/rmfakecloud/proxy.cfg file was only containing URL=, and would reset to that after each install or setcloud run of the automatic installer. I currently have it set to the domain where I host my rmfakecloud docker instance.

However, the /etc/systemd/system/proxy.service contents seem to be as expected, including the ExecStart line ending with my entered domain.

My docker instance of rmfakecloud is reversed proxied by nginx with ssl, so I enter https://foo.example.com as domain.

Is there more info I can provide? I believe this covers all I have tried and done after the update to 3.9.3.1986 from 3.2.2.1581 where it still worked fine. I updated with the understanding that v0.0.17 would support 3.9.3 (Software compatibility with 3.9.3)

Thanks in advance!

Additional troubleshooting:

Running echo Q | openssl s_client -connect localhost:443 -verify_hostname local.appspot.com -CAfile /etc/ssl/certs/ca-certificates.crt 2>&1 | grep Verify:

reMarkable: ~/ echo Q | openssl s_client -connect localhost:443  -verify_hostname local.appspot.com -CAfile /etc/ssl/certs/ca-certificates.crt 2>&1 | grep Verify
Verify return code: 0 (ok)
    Verify return code: 0 (ok)

Running ping my.remarkable.com:

reMarkable: ~/ ping my.remarkable.com
PING my.remarkable.com (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: seq=0 ttl=64 time=0.185 ms

Running ping local.remarkable.com:

reMarkable: ~/ ping local.remarkable.com
PING local.remarkable.com (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: seq=0 ttl=64 time=0.220 ms

Running wget -qO- https://cloud.example.com/:

reMarkable: ~/ wget -qO- https://cloud.example.com
wget: note: TLS certificate validation not implemented
wget: TLS error from peer (alert code 40): handshake failure

Running wget -qO- https://local.appspot.com:

reMarkable: ~/ wget -qO- https://local.appspot.com
wget: note: TLS certificate validation not implemented
<!doctype html><html lang="en"><head><meta charset="utf-8"/><meta name="viewport" content="width=device-width,initial-scale=1"/><meta name="theme-color" content="#000000"/><meta name="description" content="RM FakeApi"/><title>rmfakecloud</title><script defer="defer" src="/static/js/main.9c2de5b1.js"></script><link href="/static/css/main.d94d89ba.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>

Running systemctl status proxy -l:

reMarkable: ~/ systemctl status proxy -l
● proxy.service - reverse proxy
     Loaded: loaded (/etc/systemd/system/proxy.service; enabled; vendor preset: disabled)
     Active: active (running) since Thu 2024-01-11 18:04:38 CET; 13min ago
   Main PID: 187 (rmfake-proxy)
     CGroup: /system.slice/proxy.service
             └─ 187 /home/root/rmfakecloud/rmfake-proxy -cert /home/root/rmfakecloud/proxy.bundle.crt -key /home/root/rmfakecloud/proxy.key https://cloud.example.com

Jan 11 18:04:38 reMarkable systemd[1]: Started reverse proxy.
Jan 11 18:04:38 reMarkable rmfake-proxy[187]: 2024/01/11 18:04:38 cert-file=/home/root/rmfakecloud/proxy.bundle.crt key-file=/home/root/rmfakecloud/proxy.key listen-addr=:443 upstream-url=https://cloud.example.com

Running journalctl -u proxy:

reMarkable: ~/ journalctl -u proxy
Jan 07 19:59:39 reMarkable systemd[1]: Started reverse proxy.
Jan 07 19:59:39 reMarkable rmfake-proxy[187]: 2024/01/07 19:59:39 cert-file=/home/root/rmfakecloud/proxy.bundle.crt key-file=/home/root/rmfakecloud/proxy.key listen-addr=:443 upstream-url=https://cloud.example.com
Jan 07 19:59:40 reMarkable rmfake-proxy[187]: 2024/01/07 19:59:40 http: proxy error: dial tcp: lookup cloud.example.com on [::1]:53: read udp [::1]:57830->[::1]:53: read: connection refused
Jan 07 19:59:50 reMarkable rmfake-proxy[187]: 2024/01/07 19:59:50 http: proxy error: dial tcp: lookup cloud.example.com on [::1]:53: read udp [::1]:33451->[::1]:53: read: connection refused
Jan 07 20:00:10 reMarkable rmfake-proxy[187]: 2024/01/07 20:00:10 http: proxy error: dial tcp: lookup cloud.example.com on [::1]:53: read udp [::1]:34685->[::1]:53: read: connection refused
Jan 07 20:00:50 reMarkable rmfake-proxy[187]: 2024/01/07 20:00:50 http: proxy error: dial tcp: lookup cloud.example.com on [::1]:53: read udp [::1]:45949->[::1]:53: read: connection refused
Jan 07 20:02:10 reMarkable rmfake-proxy[187]: 2024/01/07 20:02:10 http: proxy error: dial tcp: lookup cloud.example.com on [::1]:53: read udp [::1]:37807->[::1]:53: read: connection refused
Jan 07 20:04:50 reMarkable rmfake-proxy[187]: 2024/01/07 20:04:50 http: proxy error: dial tcp: lookup cloud.example.com on [::1]:53: read udp [::1]:54323->[::1]:53: read: connection refused
Jan 07 21:09:58 reMarkable rmfake-proxy[187]: terminated
Jan 07 21:09:58 reMarkable systemd[1]: Stopping reverse proxy...
Jan 07 21:09:58 reMarkable systemd[1]: proxy.service: Deactivated successfully.
Jan 07 21:09:58 reMarkable systemd[1]: Stopped reverse proxy.
Jan 07 21:09:59 reMarkable systemd[1]: Started reverse proxy.
Jan 07 21:09:59 reMarkable rmfake-proxy[710]: 2024/01/07 21:09:59 cert-file=/home/root/rmfakecloud/proxy.bundle.crt key-file=/home/root/rmfakecloud/proxy.key listen-addr=:443 upstream-url=https://cloud.example.com
Jan 07 21:10:54 reMarkable systemd[1]: Stopping reverse proxy...
Jan 07 21:10:54 reMarkable rmfake-proxy[710]: terminated
Jan 07 21:10:54 reMarkable systemd[1]: proxy.service: Deactivated successfully.
Jan 07 21:10:54 reMarkable systemd[1]: Stopped reverse proxy.
Jan 07 21:11:29 reMarkable systemd[1]: Started reverse proxy.
Jan 07 21:11:29 reMarkable rmfake-proxy[3744]: 2024/01/07 21:11:29 cert-file=/home/root/rmfakecloud/proxy.bundle.crt key-file=/home/root/rmfakecloud/proxy.key listen-addr=:443 upstream-url=https://cloud.example.com
Jan 11 18:04:04 reMarkable rmfake-proxy[3744]: terminated
Jan 11 18:04:04 reMarkable systemd[1]: Stopping reverse proxy...
Jan 11 18:04:05 reMarkable systemd[1]: proxy.service: Deactivated successfully.
Jan 11 18:04:05 reMarkable systemd[1]: Stopped reverse proxy.
-- Boot 4fd951ff4e8441558cebc17226d83046 --
Jan 11 18:04:38 reMarkable systemd[1]: Started reverse proxy.
Jan 11 18:04:38 reMarkable rmfake-proxy[187]: 2024/01/11 18:04:38 cert-file=/home/root/rmfakecloud/proxy.bundle.crt key-file=/home/root/rmfakecloud/proxy.key listen-addr=:443 upstream-url=https://cloud.example.com

Running update-ca-certificates:

reMarkable: ~/ update-ca-certificates
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.

Running systemctl stop xochitl and then QT_LOGGING_RULES=rm.network.*=true xochitl | grep -A3 QUrl, staying like this for several minutes unchanged:

reMarkable: ~/ systemctl stop xochitl
reMarkable: ~/ QT_LOGGING_RULES=rm.network.*=true xochitl | grep -A3 QUrl
qt.core.qobject.connect: QObject::connect(QObject, Unknown): invalid nullptr parameter
debug: 2023-12-12T17:51:56Z tags/releases/3.9.3-device-RC 728f5ad
debug: we're running on an epaper device
[2024-01-11 18:11:57.623] [CSLNetworkManager] [info] Waiting for netlink thread to start
[2024-01-11 18:11:57.629] [CSLNetworkManager] [info] Network information has changed, connectedToWifi = true
[2024-01-11 18:11:57.629] [CSLNetworkManager] [info] Netlink thread is ready
17:12:51.026 rm.network.client        Http::GET QUrl("https://internal.cloud.remarkable.com:443/discovery/v1/webapp")
17:12:51.357 rm.network.client        Received hostname for AbstractNetworkClient::WebApp : "192.168.1.100:3030"
17:12:51.358 rm.network.client        Http::POST QUrl("https:///token/json/2/device/new")
17:12:51.386 rm.network.authenticationFailed to fetch token "Host  not found" ""

cloud.example.com is the only replaced text in the output to keep my actual hosted domain private.

I am affected by the same bug. The differences in my setup compared to @ArnoldDeRuiter are:

I did a fresh install of the rmfakecloud server and client software, never used either before.
I don't use SSL at the moment, just plain HTTPS.

Everything else is exactly the same:

My software version is also 3.9.3.1986
I'm also getting a "Host not found" error on the tablet after entering the verification code
I ran most of the commands from @ArnoldDeRuiter's debug output and I get the same results.

I think the most interesting part is the following:

The error message on the tablet and in the xochitl log is "Host not found" - with two spaces between Host and not! So what it's saying is "Host <empty host name> not found".
That also matches the Http::POST QUrl("https:///token/json/2/device/new") line: It's trying to post to "https://<empty host name>/token/json/2/device/new".

So if we can somehow get the proper hostname into the <empty host name> part, that bug should be solved.

What is the content of your https://local.appspot.com/discovery/v1/webapp, considering local.appspot.com is the domain you used to host rmfakecloud?

Also, what is the content of your variable STORAGE_URL, passed to rmfakecloud?

The variable has to be a valid URL, like STORAGE_URL=https://local.appspot.com. If this is just STORAGE_URL=local.appspot.com (without the scheme), the golang library consider this is just a path, so the host part is considered empty, without error.

Let me know if adding https:// (or http://) to the STORAGE_URL helps.

What is the content of your https://local.appspot.com/discovery/v1/webapp, considering local.appspot.com is the domain you used to host rmfakecloud?

In the experiments above (where it failed), the content of http://192.168.178.45:3000/discovery/v1/webapp was:

{"Host":"192.168.178.45:3000","Status":"OK"}

Also, what is the content of your variable STORAGE_URL, passed to rmfakecloud?

In the experiments above, I had STORAGE_URL=http://192.168.178.45:3000.

I've since discovered something:

with rmfakecloud-proxy, port 3000 (STORAGE_URL=http://192.168.178.45:3000)

This is the original configuration, as above.

STORAGE_URL=http://192.168.178.45:3000 http://192.168.178.45:3000/discovery/v1/webapp: {"Host":"192.168.178.45:3000","Status":"OK"}

reMarkable: ~/rmfakecloud/ QT_LOGGING_RULES=rm.network.*=true xochitl | grep -A3 QUrl
qt.core.qobject.connect: QObject::connect(QObject, Unknown): invalid nullptr parameter
debug: 2023-12-12T17:51:56Z tags/releases/3.9.3-device-RC 728f5ad
debug: we're running on an epaper device
[2024-01-17 13:33:47.579] [CSLNetworkManager] [info] Waiting for netlink thread to start
[2024-01-17 13:33:47.582] [CSLNetworkManager] [info] Network information has changed, connectedToWifi = true
[2024-01-17 13:33:47.582] [CSLNetworkManager] [info] Netlink thread is ready
13:35:05.960 rm.network.client        Http::GET QUrl("https://internal.cloud.remarkable.com:443/discovery/v1/webapp")
13:35:06.184 rm.network.client        Received hostname for AbstractNetworkClient::WebApp : "192.168.178.45:3000"
13:35:06.186 rm.network.client        Http::POST QUrl("https:///token/json/2/device/new")
13:35:06.197 rm.network.authenticationFailed to fetch token "Host  not found" ""

without rmfakecloud-proxy

Here I ran ./installer.sh uninstall, just to figure out which URL it's trying to reach:

reMarkable: ~/rmfakecloud/ QT_LOGGING_RULES=rm.network.*=true xochitl | grep -A3 QUrl
qt.core.qobject.connect: QObject::connect(QObject, Unknown): invalid nullptr parameter
debug: 2023-12-12T17:51:56Z tags/releases/3.9.3-device-RC 728f5ad
debug: we're running on an epaper device
[2024-01-17 14:07:10.520] [CSLNetworkManager] [info] Waiting for netlink thread to start
[2024-01-17 14:07:10.523] [CSLNetworkManager] [info] Network information has changed, connectedToWifi = true
[2024-01-17 14:07:10.524] [CSLNetworkManager] [info] Netlink thread is ready
14:07:48.851 rm.network.client        Http::GET QUrl("https://internal.cloud.remarkable.com:443/discovery/v1/webapp")
14:07:49.027 rm.network.client        Received hostname for AbstractNetworkClient::WebApp : "webapp.cloud.remarkable.com"
14:07:49.028 rm.network.client        Http::POST QUrl("https://webapp.cloud.remarkable.com/token/json/2/device/new")
14:07:59.322 rm.network.authenticationFailed to fetch token "Error transferring https://webapp.cloud.remarkable.com/token/json/2/device/new - server replied: " "Invalid One-time-code\n"

So it turns out it's trying to reach webapp.cloud.remarkable.com, which is not remapped in the /etc/hosts file. (Just ignore the "Invalid One-time-code" error; that's because I didn't get a proper one-time code from the official rm cloud, I just put in something invalid, so this error is expected and shows that it's working.)

with rmfakecloud-proxy, port 80 (STORAGE_URL=http://192.168.178.45)

Finally, I changed the configuration and port remappings of the rmfakecloud server to use the default port 80, and also re-installed the proxy with the URL http://192.168.178.45, no port:

STORAGE_URL=http://192.168.178.45 http://192.168.178.45/discovery/v1/webapp: {"Host":"192.168.178.45","Status":"OK"}

I also added the following line to /etc/hosts on the tablet:

127.0.0.1 webapp.cloud.remarkable.com

... but that doesn't have any effect.

reMarkable: ~/ QT_LOGGING_RULES=rm.network.*=true xochitl | grep -A3 QUrl
qt.core.qobject.connect: QObject::connect(QObject, Unknown): invalid nullptr parameter
debug: 2023-12-12T17:51:56Z tags/releases/3.9.3-device-RC 728f5ad
debug: we're running on an epaper device
[2024-01-17 14:58:32.166] [CSLNetworkManager] [info] Waiting for netlink thread to start
[2024-01-17 14:58:32.170] [CSLNetworkManager] [info] Network information has changed, connectedToWifi = true
[2024-01-17 14:58:32.171] [CSLNetworkManager] [info] Netlink thread is ready
15:01:21.853 rm.network.client        Http::GET QUrl("https://internal.cloud.remarkable.com:443/discovery/v1/webapp")
15:01:22.076 rm.network.client        Received hostname for AbstractNetworkClient::WebApp : "192.168.178.45"
15:01:22.077 rm.network.client        Http::POST QUrl("https://192.168.178.45/token/json/2/device/new")
15:01:22.089 rm.network.authenticationFailed to fetch token "Connection refused" ""

So finally it gets resolved if I omit the port number (3000). However, now it's trying to query https, and I don't have https configured yet on the rmfakecloud server, so nothing is running on port 443. That's why we get the Connection refused error.

Also, if I enter http://192.168.178.45/token/json/2/device/new (without https) into the browser, I get a "Page not found." error, so I don't think enabling https will help.

How to proceed?

I think you should install the proxy with http://192.168.178.45:3000 as upstream, and use the default value for STORAGE_URL=https://local.appspot.com/ on the rmfakecloud side.

local.appspot.com should be already configured to redirect to 127.0.0.1 in the /etc/resolv.conf, so to the proxy. The proxy will then proxify the requests to its configured upstream, either http or https.

I think you should install the proxy with http://192.168.178.45:3000 as upstream, and use the default value for STORAGE_URL=https://local.appspot.com/ on the rmfakecloud side.

local.appspot.com should be already configured to redirect to 127.0.0.1 in the /etc/resolv.conf, so to the proxy. The proxy will then proxify the requests to its configured upstream, either http or https.

Indeed, this was what seemed to have caused the issue I was facing after updating my reMarkable 2.

I just now removed the STORAGE_URL variable I had set in my docker, so it would become the default https://local.appspot.com/ and now it seems to work. :)

I think you should install the proxy with http://192.168.178.45:3000 as upstream, and use the default value for STORAGE_URL=https://local.appspot.com/ on the rmfakecloud side.

Yes, that did it! It's working now!

Just to be super explicit: I did the following. On the tablet, I set the proxy upstream URL as follows:

reMarkable: ~/ ./installer.sh setcloud http://192.168.178.45:3000
Setting cloud sync to: http://192.168.178.45:3000

On the server, I start the rmfakecloud Docker container with the following environment var (amongst others):

STORAGE_URL=https://local.appspot.com/

@ArnoldDeRuiter: Sorry for hijacking your issue. For me, it is solved. Does it work for you as well?

@ArnoldDeRuiter: Sorry for hijacking your issue. For me, it is solved. Does it work for you as well?

Oops, sorry for the noise, I didn't see your comment above. Seems it's fixed for both of us now!

@mintar Thanks for chiming in here as well, it seemed to be the trigger to spark the conversation. :slightly_smiling_face:

ddvk / rmfakecloud