Open ptandler opened 2 years ago
Hi, I'm also trying to understand the kopia server snapshot
usage
How can I print the fingerprint again in case I missed to write it down when it was created?
This is not generated. This is the "cleaned up" fingerprint of the TLS certificate:
openssl x509 -noout -fingerprint -sha256 -in <public.crt> | sed 's/://g' | cut -f 2 -d =
I think the problem here goes beyond just documentation. The server seems to be variously broken and/or very problematic to work with. I've spent the last couple of days beating my head against the server with the following results:
It's confusing that some server
commands accept control params for directly accessing a remote server (also noting the inconsistent use of the params --url
and --address
accross various subcommands, as above), whereas others do not, and appear to require connecting to a repository before they can be used. This can maybe be aided by documentation, but the CLI would be much more usable if all server operations were implemented on the control API, or alternatively (and much less preferably), operations that actually operate on the respository were put under a nested subcommand, ie server repository users list
or something.
server status
command on the host does not work, always returns 400 Bad Request
.server status
command executed remotely, using the correct control username and password returns 400 Bad Request
server status
command specifies that it reads the env vars KOPIA_SERVER_USERNAME
and KOPIA_SERVER_PASSWORD
as the values for the args --server-control-username
and --server-control-password
respectively, however the variables for the control user when starting the server are KOPIA_CONTROL_USER
and KOPIA_CONTROL_PASSWORD
, resulting in conflicts for configuration via env.--server-control-username
/--server-username
).server refresh
command on the host does not work, always returns 400 Bad Request
.server refresh
command executed remotely, using the correct control username and password returns 400 Bad Request
server refresh
command specifies that it reads the env vars KOPIA_SERVER_USERNAME
and KOPIA_SERVER_PASSWORD
as the values for the args --server-control-username
and --server-control-password
respectively, however the variables for the control user when starting the server are KOPIA_CONTROL_USER
and KOPIA_CONTROL_PASSWORD
, resulting in conflicts for configuration via env.--server-control-username
/--server-username
).It's possible all the control-api commands produce these results, but the above two are the ones that I tested.
I could not find any way to get both the CLI and the server to see that they're connected to a repository other than to first create the repository from the CLI (e.g. kopia respository create ...
), then to log into the server web UI and connect the repository from there too. This is quite confusing - is the server not using the same configs as the CLI or something?
After creating a user via kopia server users add user@host
, connecting to the repository on a client kopia repository connect server ...
, and receiving the expected response:
Connecting to server 'https://127.0.0.1:51515' as 'user@host'...
Connected to repository API Server.
NOTICE: Kopia will check for updates on GitHub every 7 days, starting 24 hours after first use.
To disable this behavior, set environment variable KOPIA_CHECK_FOR_UPDATES=false
Alternatively you can remove the file "/home/user/.config/kopia/repository.config.update-info.json".
No repository commands work:
$ kopia snapshot create /home/user/tmp
ERROR failed to open repository: unable to establish session for purpose=: error establishing session: unable to initialize session: rpc error: code = PermissionDenied desc = access denied for user@host: EOF
ERROR open repository: unable to open repository: unable to establish session for purpose=: error establishing session: unable to initialize session: rpc error: code = PermissionDenied desc = access denied for user@host: EOF
$ kopia snapshot list
ERROR failed to open repository: unable to establish session for purpose=: error establishing session: unable to initialize session: rpc error: code = PermissionDenied desc = access denied for user@host: EOF
ERROR open repository: unable to open repository: unable to establish session for purpose=: error establishing session: unable to initialize session: rpc error: code = PermissionDenied desc = access denied for user@host: EOF
$ kopia repository status
ERROR failed to open repository: unable to establish session for purpose=: error establishing session: unable to initialize session: rpc error: code = PermissionDenied desc = access denied for user@host: EOF
ERROR open repository: unable to open repository: unable to establish session for purpose=: error establishing session: unable to initialize session: rpc error: code = PermissionDenied desc = access denied for user@host: EOF
Maybe this is because the server refresh
command is broken and can't be executed, however restarting the server also has no impact, and repository connect
succeeds so I suspect this is a more fundamental problem.
I tried enabling ACLs but they made no difference.
So my steps to this point, performed on Kopia v0.13.0 (I'm doing dev locally, so the server host here is a docker container, but AFAICT this is all equally valid for a hand-rolled server install):
kopia server start ...
kopia repository create filesystem ...
kopia server users add ...
kopia repository connect ...
At this point I think I've exhausted all the options I can find as a user, without putting my dev hat on and going to try to hack the code. It doesn't help that the debug
log level rarely seems to output anything more than the info
level, which outputs the bare minimum - decent logging would probably make diagnosing this stuff vastly easier.
Also some UX adjustments would be nice. For example kopia server users add User@Host
will gladly prompt for a password, twice, only to stop you with
$ kopia server users add User@Host
Enter new password for user User@Host:
Re-enter new password for verification:
ERROR error setting user profile: username must be specified as lowercase 'user@hostname'
Same happens with kopia repository connect server --url=http://somehost:51515
. It validates arguments only after prompting for credentials.
$ kopia repository connect server --url=http://localhost:51515
Connecting to server 'http://localhost:51515' as 'username@hostname'...
Enter password to open repository:
ERROR failed to open repository: invalid server address, must be 'https://host:port'
ERROR error connecting to API server: invalid server address, must be 'https://host:port'
As a side question: is there a reason why Kopia connects to the repository server over HTTPS only? I've seen a generic "due to gRPC" answer here, but no further explanation. There was an issue about the same topic, but it went stale: #1215. A quick google search reveals that gRPC can work over plain HTTP. See e.g. grpc/grpc-go#555
I want to report that I am also experiencing "permission denied" errors when connecting to a Kopia server over TLS. Having followed the documentation quite closely.
kopia repo connect server --url=https://192.168.1.106:51515 --server-cert-fingerprint ####
[info] ERROR failed to open repository: unable to establish session for purpose=: error establishing session: unable to initialize session: rpc error: code = PermissionDenied desc = access denied for ####@#####: EOF
This error message occurs on the client (Windows 11), but on the server host (macOS) nothing comes up in the stdout.
Also, can we get actual docker environment variables instead of this command line argument stuff shoehorned into a compose file?
KOPIA_PASSWORD="<password-for-the-repository>" \
KOPIA_SERVER_CONTROL_PASSWORD="<server-control-password>" \
kopia server start \
--tls-generate-cert \
--tls-cert-file ~/my.cert \
--tls-key-file ~/my.key \
--address 0.0.0.0:51515 \
--server-control-username control
Having to mix mounted volumes with absolute cert file paths when there are often already mount points for these is quite patchy and there is no reason to do so.
Additionally, there should absolutely be documented accepted environment variables by the container like many other docker apps have.
https://github.com/imagegenius/docker-kopia?tab=readme-ov-file#parameters
https://immich.app/docs/install/environment-variables/
and of course, no mention in the documentation to go the very common path of no-tls on a secure home network. The client works without it, but the server just throws errors that it is expecting TLS.
Connect Error: INTERNAL: internal server error: connect error: error opening repository: error connecting to API server: unable to establish session for purpose=: error establishing session: Session(): rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: tls: first record does not look like a TLS handshake"
I was wondering about "GRPC needs TLS" too, so I asked my friend ClaudeAI about it:
Let me clarify this:
gRPC does not strictly require TLS (Transport Layer Security). You can run gRPC without TLS in what's called "plaintext" mode. However, using TLS with gRPC is highly recommended and considered best practice for several reasons:
- Security: TLS provides encryption and authentication
- Modern browser requirements: Web browsers require TLS for HTTP/2, which gRPC uses
- Production environments: Most production deployments require TLS for security
You can use gRPC without TLS in:
- Development environments
- Testing scenarios
- Internal networks where security requirements are different
- Situations where performance is critical and the network is already secured
Besides that there is this fingerprint being used to identify or validate the server, which would need a replacement when running without TLS.
I'm pretty new to kopia and really impressed! In general the documentation is really good and easy to understand fro me. Thank you very much for all you work!
However, now I tried to setup a kopia server and there are some things I missed at https://kopia.io/docs/repository-server/ so these points could be improved. (I might provide a PR for the documentation ... once I understand the details ... :smiley:)
kopia server start
, link to https://kopia.io/docs/reference/command-line/common/server-start/sudo ufw allow 51515 comment 'kopia server via https'
)kopia server start
does not start the server in the background, but instead provides the server itself. I did expect it otherwise, so it might be mentioned. I guess now, the server won't be started automatically then, right?kopia server snapshot
(cancel / pause / resume) do? I don't understand the explanation on https://kopia.io/docs/reference/command-line/common/server-snapshot/(*) Could it be that the headings level got messed up? It looks like "Auto-Generated TLS Certificate" is a sub heading of "Configuring Allowed Users - Kopia v0.7":
But when I read more carefully, it talks about kopia v0.8 as well, so I guess there is a top level heading missing?
Some more experiences during my first steps, as hints how the documentation could be improved:
How can I print the fingerprint again in case I missed to write it down when it was created?
Once I started a kopia server, I try to connect (on server machine):
This error message is not really helpful and should at least be explained. But when connection from a remote machine this did work:
When I try to get the server status from a remote machine, this fails as well:
(BTW: Seems to be inconsistent to use
--address
here and--url
at other places. Or does it have different semantics?)Also, when I try to open the server web UI in a browser, the credentials I created before using
kopia server user add
are not accepted.When I try to sync a local repo to the server I also get an error:
Is syncing to a server not supported or did I miss something here? It did work when I tried
filesystem
orsftp
.$ kopia --version 0.10.7 build: 5d87d817335f6d547e094ab80062113dc3a1fdf4 from: kopia/kopia