fb929
commented
2 weeks ago Could you please show the full status on the client? Something like this
Open 3m0W33D opened 2 weeks ago
fb929
commented
2 weeks ago Could you please show the full status on the client? Something like this
3m0W33D
commented
2 weeks ago Hi here is a more complete status of the client,
fb929
commented
2 weeks ago It appears that the client has no access to the server.
EXTERNAL_LISTEN_HOST. any-sync-* daemon ports at this address.3m0W33D
commented
2 weeks ago Hi I have checked and found that there should be no firewall on the server as it is intra-net and nothing is configured and no outbound rules that prevent the client from accessing it. Checking the client logs I have found this error related to file upload, could this be the issue?
[2024-11-07 00:26:39.959] [warn] {"level":"INFO","ts":"2024-11-07T00:26:39.959+0800","logger":"common.commonspace.headsync","msg":"sync done:","spaceId":"bafyreifmexay7hod7kwhvzqdngrr4xkmo75xpsxsfvklfqjfvhg7v4hfbm.1oiopl51kvxi4","newIds":0,"changedIds":0,"removedIds":1,"already deleted ids":0,"peerId":"12D3KooWKpBQ6MvzYSLpw4PZxvMo8aVgZtB8E3SPhvnaUPVrzyXN"}
[2024-11-07 00:26:44.266] [warn] {"level":"ERROR","ts":"2024-11-07T00:26:44.266+0800","logger":"filesync","msg":"retry uploading file error","fileId":"bafybeiemyumeexpluyipvxikwypoxo6yrq3hboqqmqmae3trhnjw4d43m4","error":"walk file blocks: walk DAG: process batch: select blocks to upload: add to file: NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors; NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors; NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors; NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors; NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors; NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors; NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors; NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors; NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors; NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors","objectId":"bafyreiacbjus5aeqklttuhasndchjmzxcc7jpzxzpuqo3qxo6xiu4pg4sm"}Yes, it seems that’s the issue - filenode cannot load data from MinIO.
Try commenting out AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in the .env.override file and restarting the stand with configuration regeneration using make restart.
3m0W33D
commented
2 weeks ago Found the issue the AWS_SECRET_ACCESS_KEY had special characters that messed up the initialization script??? Generated the password using bitwarden with special characters. I think its fine and we can close this issue but maybe a mention somewhere to use non-special characters that bash script cannot register?
fb929
commented
1 week ago We have no restrictions on the use of special characters in AWS_SECRET_ACCESS_KEY or AWS_ACCESS_KEY_ID on our side.
However, MinIO does have such restrictions :)
You need to check the MinIO logs; most likely, the login credentials didn’t pass certain checks. A similar error occurs when the AWS_ACCESS_KEY_ID (or AWS_SECRET_ACCESS_KEY) are too short.
JakeHadley
commented
1 week ago @fb929 I'm happy to make a new issue for my question, but it seems to be somewhat related to this issue. To allow a client access to the self-hosted server, do we need to add the ip address our client is running on to the env override? Does that mean that it's not feasible to have the mobile app connect to the server outside of the home network? It's only able to be synced on the home network?
I'm wondering because I was assuming I'd be able to host the server and expose it via a cloudflare tunnel to get access outside the home network. But if we have to add every ip address to the .env, that's not really feasible.
fb929
commented
2 days ago @JakeHadley I don't quite understand what you mean. Basically, the server's IP address must be accessible for the client to connect, specifically ensuring connection through the ports used by the any-sync-* daemons. You can get the list of ports using the following command:
$ grep ANY_SYNC_.*PORT= .env
ANY_SYNC_NODE_1_PORT=1001
ANY_SYNC_NODE_1_QUIC_PORT=1011
ANY_SYNC_NODE_2_PORT=1002
ANY_SYNC_NODE_2_QUIC_PORT=1012
ANY_SYNC_NODE_3_PORT=1003
ANY_SYNC_NODE_3_QUIC_PORT=1013
ANY_SYNC_COORDINATOR_PORT=1004
ANY_SYNC_COORDINATOR_QUIC_PORT=1014
ANY_SYNC_FILENODE_PORT=1005
ANY_SYNC_FILENODE_QUIC_PORT=1015
ANY_SYNC_CONSENSUSNODE_PORT=1006
ANY_SYNC_CONSENSUSNODE_QUIC_PORT=1016Here, _QUIC_ refers to UDP ports, while all the others are TCP.
JakeHadley
commented
2 days ago @JakeHadley I don't quite understand what you mean. Basically, the server's IP address must be accessible for the client to connect, specifically ensuring connection through the ports used by the any-sync-* daemons. You can get the list of ports using the following command:
$ grep ANY_SYNC_.*PORT= .env ANY_SYNC_NODE_1_PORT=1001 ANY_SYNC_NODE_1_QUIC_PORT=1011 ANY_SYNC_NODE_2_PORT=1002 ANY_SYNC_NODE_2_QUIC_PORT=1012 ANY_SYNC_NODE_3_PORT=1003 ANY_SYNC_NODE_3_QUIC_PORT=1013 ANY_SYNC_COORDINATOR_PORT=1004 ANY_SYNC_COORDINATOR_QUIC_PORT=1014 ANY_SYNC_FILENODE_PORT=1005 ANY_SYNC_FILENODE_QUIC_PORT=1015 ANY_SYNC_CONSENSUSNODE_PORT=1006 ANY_SYNC_CONSENSUSNODE_QUIC_PORT=1016Here,
_QUIC_refers to UDP ports, while all the others are TCP.
I probably am not understanding the nomenclature of what the EXTERNAL_LISTEN_HOST is.
For instance, if anytype is running/hosted on a VPS with the public ip of 123.123.123.123, is that what I give to that env variable? And if my phone's anytype app wanted to connect to that anytype server on the vps, would it need to have all of those ports you listed exposed and publicly available? I'm connecting through a cloudflare tunnel, like I mentioned earlier, so is there not a way to say "my anytype server is hosted at 123.123.123.123 port 1001", and map a domain name to it?
fb929
commented
2 days ago I probably am not understanding the nomenclature of what the
EXTERNAL_LISTEN_HOSTis.
EXTERNAL_LISTEN_HOST is the host (or IP) that clients will use to connect to the server. If you want clients to connect via the public IP, then specify it. However, keep in mind that in this case, access to your self-hosted server will effectively be public (unless you restrict access to specific client IP addresses using a firewall). Regarding the naming of "EXTERNAL," it's called that because, relative to the internal Docker network, it is considered "EXTERNAL."
For instance, if anytype is running/hosted on a VPS with the public ip of 123.123.123.123, is that what I give to that env variable? And if my phone's anytype app wanted to connect to that anytype server on the vps, would it need to have all of those ports you listed exposed and publicly available?
Yes, that's correct.
I'm connecting through a cloudflare tunnel, like I mentioned earlier, so is there not a way to say "my anytype server is hosted at 123.123.123.123 port 1001", and map a domain name to it?
say where? If you mean "say in the any-sync-dockercompose configuration," that is done using the EXTERNAL_LISTEN_HOST variable.
JakeHadley
commented
2 days ago I probably am not understanding the nomenclature of what the
EXTERNAL_LISTEN_HOSTis.EXTERNAL_LISTEN_HOST is the host (or IP) that clients will use to connect to the server. If you want clients to connect via the public IP, then specify it. However, keep in mind that in this case, access to your self-hosted server will effectively be public (unless you restrict access to specific client IP addresses using a firewall). Regarding the naming of "EXTERNAL," it's called that because, relative to the internal Docker network, it is considered "EXTERNAL."
For instance, if anytype is running/hosted on a VPS with the public ip of 123.123.123.123, is that what I give to that env variable? And if my phone's anytype app wanted to connect to that anytype server on the vps, would it need to have all of those ports you listed exposed and publicly available?
Yes, that's correct.
I'm connecting through a cloudflare tunnel, like I mentioned earlier, so is there not a way to say "my anytype server is hosted at 123.123.123.123 port 1001", and map a domain name to it?
say where? If you mean "say in the any-sync-dockercompose configuration," that is done using the EXTERNAL_LISTEN_HOST variable.
I'm no good at networking, but if I'm mapping a domain name to the server ip and port that anytype is running on, I'm going to have to create 12 entries in the cloudflare tunnel public hostname that will allow anytype to correctly communicate. I'm assuming this would be true for other providers or even if I was just port forwarding from my home network. This seems doable, but a little untenable in terms of self-hosting. Otherwise, if I didn't port forward or tunnel, then my devices would only be able to sync when connected to the home network that the server is hosted on. Am I understanding all that correctly?
fb929
commented
2 days ago I can't confirm because I haven't dealt with the "Cloudflare Tunnel" setup, but it seems plausible.
Have you read a contributing guide?
Current Behavior
I have just setup a new self-hosted network with this docker compose, however, the client keeps trying to sync images/attachments but fails to do so and keeps loading. I am using the default docker-compose and only changed the
I have tried to investigate the issue by checking the docker logs of the coordinator node. There seems to be some incompatibility with the filenode???
Coordinator node logs
File node logs
I am unsure what the issue is as I am able to sync notes except attachments to the server. Peer to peer sync still allows me to sync attachments to other devices such as my android device. Hope someone can shed light on this issue
Expected Behavior
Able to sync images/attachments to the server
Steps To Reproduce
Using the default settings and only change the following settings (censored out some information) STORAGE_DIR=/data AWS_ACCESS_KEY_ID= AWS_SECRET_ACCESS_KEY= EXTERNAL_LISTEN_HOST=192.0.0.1
Environment