Taskserver not syncing over local network

manoelpqueiroz commented 1 year ago

Hi everyone,

I have set up a Taskserver on a Raspberry Pi 4, but now I am unable to make it sync over the local network. I have tried searching for the term raspberry and local network, but there are very few cases of issues and none seem to pertain to mine.

Also, I have tried looking for a logging document to better provide you with context, but haven't found it for the Taskwarrior client. I am certain it's a very simple configuration I am missing, but haven't been able to get from the documentation, would really appreciate your help with this one.

Context

I have a Raspberry Pi 4 in which I have successfully set up Taskserver. Its hostname is rpi and so is the output of the hostname command.

$ hostname
rpi

$ ps -leaf | grep taskd
1 S root          45       2  0  80   0 -     0 -      18:18 ?        00:00:00 [khungtaskd]
4 S taskd-u+     732       1  0  80   0 -  3229 -      18:18 ?        00:00:00 /usr/local/bin/taskd server --data /var/taskd
0 S myuser      3959    3007  0  80   0 -  1605 pipe_r 21:58 pts/0    00:00:00 grep --color=auto --exclude-dir=.bzr --exclude-dir=CVS --exclude-dir=.git --exclude-dir=.hg --exclude-dir=.svn --exclude-dir=.idea --exclude-dir=.tox taskd

$ sudo systemctl status taskd
● taskd.service - Secure server providing multi-user, multi-client access to task data
Loaded: loaded (/etc/systemd/system/taskd.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2023-09-18 09:04:06 -03; 12h ago
Docs: http://taskwarrior.org/docs/#taskd
Main PID: 732 (taskd)
Tasks: 1 (limit: 9238)
Memory: 3.6M
CPU: 696ms
CGroup: /system.slice/taskd.service
└─732 /usr/local/bin/taskd server --data /var/taskd

Sep 18 09:04:06 rpi systemd[1]: Started Secure server providing multi-user, multi-client access to task data.

Now, I have a client set up in another local machine (hostname home-laptop) which can currently access rpi via SSH, for instance, but after generating the certificates, copying them to my client and asking to sync, it raises a failure message:

$ task sync init
Please confirm that you wish to upload all your tasks to the Taskserver (yes/no) y
Syncing with rpi:53589

Name or service not known
Sync failed.  Could not connect to the Taskserver.

Diagnostics

taskd diag on the server:

taskd 1.1.0
Platform: Linux
Hostname: rpi

Compiler
Version: 11.3.0
Caps: +stdc +stdc_hosted +200809 +200809 +LP64 +c8 +i32 +l64 +vp64 +time_t64
Compliance: C++11

Build Features
Built: Jul 25 2023 19:40:37
Commit: 3caa8e2
CMake: 3.22.1
libuuid: libuuid + uuid_unparse_lower
libgnutls: 3.7.3
Build type: release

Configuration
TASKDDATA: /var/taskd
root: /var/taskd (readable)
config: /var/taskd/config (readable)
CA: /var/taskd/ca.cert.pem
Certificate: /var/taskd/server.cert.pem (missing)
Key: /var/taskd/server.key.pem (missing)
CRL: /var/taskd/server.crl.pem
Log: /var/taskd.log (found)
PID File: /var/taskd.pid (found)
Server: rpi:53589
Max Request: 1048576 bytes
Ciphers:
Trust: strict

:warning: One thing I find strange is that it shows certificate and key as missing, but inside /var/taskd, they are present with the respective group:

$ ls -la /var/taskd
total 72
drwxr-xr-x  4 taskd-user taskd-group  4096 Aug  5 16:37 .
drwxr-xr-x 15 root       root         4096 Aug  6 17:43 ..
-rw-------  1 taskd-user taskd-group  2045 Aug  6 17:14 ca.cert.pem
-rw-------  1 taskd-user taskd-group 10732 Aug  6 17:16 ca.key.pem
-rw-------  1 taskd-user taskd-group  2041 Aug  6 17:13 client.cert.pem
-rw-------  1 taskd-user taskd-group 10735 Aug  6 17:13 client.key.pem
-rw-r--r--  1 taskd-user taskd-group   428 Aug  5 16:37 config
drwx------  3 taskd-user taskd-group  4096 Aug  5 18:03 orgs
drwxrwxr-x  2 myuser     myuser       4096 Sep 18 21:49 pki
-rw-------  1 taskd-user taskd-group  2041 Aug  6 17:14 server.cert.pem
-rw-------  1 taskd-user taskd-group  1101 Aug  6 17:14 server.crl.pem
-rw-------  1 taskd-user taskd-group 10722 Aug  6 17:13 server.key.pem

task diag on the client in home-laptop:

task 2.6.2
Platform: Linux

Compiler
Version: 13.1.1 20230429
Caps: +stdc +stdc_hosted +LP64 +c8 +i32 +l64 +vp64 +time_t64
Compliance: C++17

Build Features
CMake: 3.26.3
libuuid: libuuid + uuid_unparse_lower
libgnutls: 3.8.0
Build type:

Configuration
File: /home/clientuser/.taskrc (found), 6210 bytes, mode 100755
Data: /home/clientuser/.task (found), dir, mode 40755
Locking: Enabled
GC: Enabled
$EDITOR: /usr/bin/nvim
Server: rpi:53589
CA: /home/clientuser/.task/ca.cert.pem, readable, 3775 bytes
Certificate: /home/clientuser/.task/clientuser.cert.pem, readable, 3771 bytes
Key: /home/clientuser/.task/clientuser.key.pem, readable, 24742 bytes
Trust: strict
Ciphers: NORMAL
Creds: defaultgroup/clientuser/************************************

Hooks
System: Enabled
Location: /home/clientuser/.task/hooks
(-none-)

Tests
Terminal: 223x33
Dups: Scanned 962 tasks for duplicate UUIDs:
No duplicates found
Broken ref: Scanned 962 tasks for broken references:
No broken references found

To check if this is a problem during connection or during the configuration of my Taskserver, I have created a second user called test-1 and set up a Taskwarrior client in rpi itself. In this case, syncing is working fine as it should be expected, so something is preventing the connection between rpi and home-laptop.

task diag on the client on rpi:

task 2.6.1
Platform: Linux

Compiler
Version: 11.2.0
Caps: +stdc +stdc_hosted +LP64 +c8 +i32 +l64 +vp64 +time_t64
Compliance: C++17

Build Features
CMake: 3.18.4
libuuid: libuuid + uuid_unparse_lower
libgnutls: 3.7.1
Build type: None

Configuration
File: /home/myuser/.taskrc (found), 1641 bytes, mode 100664
Data: /home/myuser/.task (found), dir, mode 40755
Locking: Enabled
GC: Enabled
Server: rpi:53589
CA: /home/myuser/.task/ca.cert.pem, readable, 3775 bytes
Certificate: /home/myuser/.task/test-1.cert.pem, readable, 3771 bytes
Key: /home/myuser/.task/test-1.key.pem, readable, 24745 bytes
Trust: strict
Ciphers: NORMAL
Creds: defaultgroup/test-1/************************************

Hooks
System: Enabled
Location: /home/myuser/.task/hooks
(-none-)

Tests
Terminal: 223x33
Dups: Scanned 5 tasks for duplicate UUIDs:
No duplicates found
Broken ref: Scanned 5 tasks for broken references:
No broken references found

As a final document, here is what is registered in the log:

2023-09-18 12:04:07 ==== taskd 1.1.0 3caa8e2 ====
2023-09-18 12:04:07 Serving from /var/taskd
2023-09-18 12:04:07 Using address rpi
2023-09-18 12:04:07 Using port 53589
2023-09-18 12:04:07 Using family
2023-09-18 12:04:07 Queue size 10 requests
2023-09-18 12:04:07 Request size limit 1048576 bytes
2023-09-18 12:04:07 IP logging on
2023-09-18 12:04:07 CA          /var/taskd/ca.cert.pem
2023-09-18 12:04:07 Certificate /var/taskd/server.cert.pem
2023-09-18 12:04:07 Private Key /var/taskd/server.key.pem
2023-09-18 12:04:07 CRL         /var/taskd/server.crl.pem
2023-09-18 12:04:07 Server starting
2023-09-18 12:04:07 Server ready
2023-09-19 00:51:24 [1] 'sync' from 'defaultgroup/test-1' using 'task 2.6.1' at 127.0.0.1:57796
2023-09-19 00:51:24 [1] Client key '' + 1 txns
2023-09-19 00:51:24 [1] Stored 1 tasks, merged 0 tasks
2023-09-19 00:51:24 [1] New sync key '9cef790c-0c88-41c7-a0b2-223cfd456b3e'
2023-09-19 00:51:24 [1] Wrote 2
2023-09-19 00:51:24 [1] Serviced in 0.002797s

Additional Context

I'm certain the problem is with my hostname somehow, but I'm not sure how to go about it. One thing that might be relevant is that I am using this Raspberry Pi with cloud-init, so when I want to SSH into it, I use the .local domain:

$ ssh myuser@rpi.local

What could I be doing wrong here? Any hints? I apologise beforehand for reaching out through here, but I exhausted the references available in documentation and forums like Stack Exchange. I really appreciate your help.

djmitche commented 1 year ago

Have you tried setting the server to rpi.local:53589? It looks like it's unable to resolve the hostname rpi.

manoelpqueiroz commented 1 year ago

I confess I haven't tried because the documentation explicitly informs that the CN variable inside the vars file must match hostname -f. If that is indeed the case, I would have to change my machine's hostname to rpi.local, which in turn would require a SSH command to be ssh myuser@rpi.local.local, and so forth and so on.

What I have tried more recently:

Changed the CN variable to rpi.local;
Generated new keys inside the pki folder;
Copied them to $TASKDDATA;
Ran taskdctl restart;
Removed and added clientuser in defaultgroup;
Generated a new client key for clientuser inside the pki folder;
Copied clientuser.*.pem and ca.cert.pem to home-laptop;
Reconfigured taskd.credentials with the new UUID for clientuser.

However, I still get an error:

$ task sync init
Please confirm that you wish to upload all your tasks to the Taskserver (yes/no) y
Syncing with rpi.local:53589

Could not connect to rpi.local 53589
Sync failed.  Could not connect to the Taskserver.

This error is raised almost immediately after the command is run on the CLI. In contrast, when I had the previous Name or service not known, it took a few seconds between running the command and receiving the error, which leads me to believe the client is able to at least find the server, but unable to resolve its name.

djmitche commented 1 year ago

Hm, that's a different error at least. Can you connect to that host and port with something like curl?

manoelpqueiroz commented 10 months ago

@djmitche, sorry for the late reply. I have made some additional tests since I last tested (still no success in setting taskserver up).

General connectivity

Using curl:

curl rpi.local:53589
curl: (52) Empty reply from server

Using wget seems to indicate the connection is at least successful (however note the first attempt in which the connection is refused through IPv6:

wget -S -t 3 rpi.local:53589
--2024-01-27 11:42:00--  http://rpi.local:53589/
Resolving rpi.local (rpi.local)... 2004:14c:1a9:870e:e75b:1ff:fed9:cac0, 192.160.0.5
Connecting to rpi.local (rpi.local)|2004:14c:1a9:870e:e75b:1ff:fed9:cac0|:53589... failed: Connection refused.
Connecting to rpi.local (rpi.local)|192.160.0.5|:53589... connected.
HTTP request sent, awaiting response... No data received.
Retrying.

--2024-01-27 11:42:01--  (try: 2)  http://rpi.local:53589/
Connecting to rpi.local (rpi.local)|192.160.0.5|:53589... connected.
HTTP request sent, awaiting response... No data received.
Retrying.

--2024-01-27 11:42:03--  (try: 3)  http://rpi.local:53589/
Connecting to rpi.local (rpi.local)|192.160.0.5|:53589... connected.
HTTP request sent, awaiting response... No data received.
Giving up.

Taskserver troubleshooting guide

Can confirm that openssl s_client -CAfile ~/.task/ca.cert.pem -host rpi.local -port 53589 returns Verify return code: 0;
Can confirm that certtool -i < server.cert.pem | grep Subject: returns a non-empty string outputting the CN and O variables;
Can confirm that certificate on the client is still valid with certtool -i --infile ~/.task/clientuser.cert.pem

Checking the process ID and port:

sudo lsof -i TCP:53589 -s TCP:LISTEN
COMMAND PID       USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
taskd   750 taskd-user    3u  IPv4  20882      0t0  TCP rpi:53589 (LISTEN)

netstat -tl | grep 53589
tcp        0      0 rpi:53589   0.0.0.0:*               LISTEN

Checking connectivity from the client:

Starting Nmap 7.94 ( https://nmap.org ) at 2024-01-27 12:24 -03
Nmap scan report for rpi.local (192.160.0.5)
Host is up (0.10s latency).
Other addresses for rpi.local (not scanned): 2004:14c:1a9:870e:e75b:1ff:fed9:cac0
Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds

Starting Nmap 7.94 ( https://nmap.org ) at 2024-01-27 12:25 -03
Nmap scan report for rpi.local (192.160.0.5)
Host is up (0.011s latency).
Other addresses for rpi.local (not scanned): 2004:14c:1a9:870e:e75b:1ff:fed9:cac0

PORT      STATE SERVICE
53589/tcp open  unknown

Nmap done: 1 IP address (1 host up) scanned in 0.45 seconds

Syncing in debug mode on the client:

task rc.debug=1 rc.debug.tls=2 sync init

Please confirm that you wish to upload all your tasks to the Taskserver (yes/no) y
c: INFO Server certificate will be verified.
c: 2 Initializing needed PKCS #11 modules
c: 2 p11: Initializing module: p11-kit-trust
c: 2 p11: No login requested.
c: 2 p11: No login requested.
c: 2 added 6 protocols, 29 ciphersuites, 19 sig algos and 10 groups into priority list
c: 2 Keeping ciphersuite 13.02 (GNUTLS_AES_256_GCM_SHA384)
c: 2 Keeping ciphersuite 13.03 (GNUTLS_CHACHA20_POLY1305_SHA256)
c: 2 Keeping ciphersuite 13.01 (GNUTLS_AES_128_GCM_SHA256)
c: 2 Keeping ciphersuite 13.04 (GNUTLS_AES_128_CCM_SHA256)
c: 2 Keeping ciphersuite c0.2c (GNUTLS_ECDHE_ECDSA_AES_256_GCM_SHA384)
c: 2 Keeping ciphersuite cc.a9 (GNUTLS_ECDHE_ECDSA_CHACHA20_POLY1305)
c: 2 Keeping ciphersuite c0.ad (GNUTLS_ECDHE_ECDSA_AES_256_CCM)
c: 2 Keeping ciphersuite c0.0a (GNUTLS_ECDHE_ECDSA_AES_256_CBC_SHA1)
c: 2 Keeping ciphersuite c0.2b (GNUTLS_ECDHE_ECDSA_AES_128_GCM_SHA256)
c: 2 Keeping ciphersuite c0.ac (GNUTLS_ECDHE_ECDSA_AES_128_CCM)
c: 2 Keeping ciphersuite c0.09 (GNUTLS_ECDHE_ECDSA_AES_128_CBC_SHA1)
c: 2 Keeping ciphersuite c0.30 (GNUTLS_ECDHE_RSA_AES_256_GCM_SHA384)
c: 2 Keeping ciphersuite cc.a8 (GNUTLS_ECDHE_RSA_CHACHA20_POLY1305)
c: 2 Keeping ciphersuite c0.14 (GNUTLS_ECDHE_RSA_AES_256_CBC_SHA1)
c: 2 Keeping ciphersuite c0.2f (GNUTLS_ECDHE_RSA_AES_128_GCM_SHA256)
c: 2 Keeping ciphersuite c0.13 (GNUTLS_ECDHE_RSA_AES_128_CBC_SHA1)
c: 2 Keeping ciphersuite 00.9d (GNUTLS_RSA_AES_256_GCM_SHA384)
c: 2 Keeping ciphersuite c0.9d (GNUTLS_RSA_AES_256_CCM)
c: 2 Keeping ciphersuite 00.35 (GNUTLS_RSA_AES_256_CBC_SHA1)
c: 2 Keeping ciphersuite 00.9c (GNUTLS_RSA_AES_128_GCM_SHA256)
c: 2 Keeping ciphersuite c0.9c (GNUTLS_RSA_AES_128_CCM)
c: 2 Keeping ciphersuite 00.2f (GNUTLS_RSA_AES_128_CBC_SHA1)
c: 2 Keeping ciphersuite 00.9f (GNUTLS_DHE_RSA_AES_256_GCM_SHA384)
c: 2 Keeping ciphersuite cc.aa (GNUTLS_DHE_RSA_CHACHA20_POLY1305)
c: 2 Keeping ciphersuite c0.9f (GNUTLS_DHE_RSA_AES_256_CCM)
c: 2 Keeping ciphersuite 00.39 (GNUTLS_DHE_RSA_AES_256_CBC_SHA1)
c: 2 Keeping ciphersuite 00.9e (GNUTLS_DHE_RSA_AES_128_GCM_SHA256)
c: 2 Keeping ciphersuite c0.9e (GNUTLS_DHE_RSA_AES_128_CCM)
c: 2 Keeping ciphersuite 00.33 (GNUTLS_DHE_RSA_AES_128_CBC_SHA1)
c: 2 Advertizing version 3.4
c: 2 Advertizing version 3.3
c: 2 Advertizing version 3.2
c: 2 Advertizing version 3.1
c: 2 HSK[0x57cecf38ea80]: sent server name: 'rpi.local'
c: 2 EXT[0x57cecf38ea80]: client generated SECP256R1 shared key
Timer Config::load (/home/myuser/.taskrc) 0.000556 sec
Parse Tree (before command-specifіc processing)
_original_args
task rc.debug=1 rc.debug.tls=2 sync init
_args
word basename='task' raw='task' BINARY
pair modifier='debug' name='rc' raw='rc.debug=1' separator='=' value='1' CONFIG ORIGINAL
pair modifier='debug.tls' name='rc' raw='rc.debug.tls=2' separator='=' value='2' CONFIG ORIGINAL
identifier canonical='synchronize' raw='sync' ORIGINAL CMD ALLOWSMISC
identifier raw='init' ORIGINAL MISCELLANEOUS

pending.data rw - T0529+000~000-000 L0529+000
completed.data rw - T0479+000~000-000 L0479+000
undo.data rw - T0000+000~000-000 L0000+000
backlog.data rw - T0000+000~000-000 L0000+000

Perf task 2.6.2 - 20240127T155503Z init:1856 load:17324 gc:0 filter:0 commit:297 sort:0 render:0 hooks:1 other:1803273 total:1805427

Syncing with rpi.local:53589

Configuration override rc.debug=1
Configuration override rc.debug.tls=2
Handshake failed. The certificate is NOT trusted. The name in the certificate does not match the expected.
Sync failed.  Could not connect to the Taskserver.

It looks that after all the .local part, which is needed to SSH into the server, is what makes it impossible to conciliate between my server's hostname (rpi) and the CN variable.

Is it possible to reopen this issue for us to assess this?

djmitche commented 10 months ago

Sure

djmitche commented 10 months ago

"The name in the certificate does not match the expected" suggests that the name in the certificate does not match the expected name (c: 2 HSK[0x57cecf38ea80]: sent server name: 'rpi.local'). But you said in an earlier comment that you changed the name in the certificate to be rpi.local -- so that doesn't quite add up. When you ran the openssl s_client command, what CN did you see? For example with github.com I see

subject=C = US, ST = California, L = San Francisco, O = "GitHub, Inc.", CN = github.com

What happens if you add -verify and -verify_hostname to that command?

djmitche commented 7 months ago

Closing since taskserver is no longer supported.

GothenburgBitFactory / taskwarrior