Nextcloud: Adding local server yields error 400

[Jimmyson]

I am looking to test the PhotoPrism Backup functionality with my local Nextcloud server. Both services are running in Docker containers, and both are exposed and accessible. I am utilizing HTTP rather than the HTTPS, as the endpoints are both contained within my local network.

When adding the Nextcloud server to PhotoPrism, I am informed that a connection could not be established. Looking over the Logs, I am presented with the following information. I attempted a connection twice

2020-09-10 11:17:18 DEBU POST /api/v1/accounts (400) [55.223016ms]
2020-09-10 11:17:18 DEBU api: abort /api/v1/accounts with code 400 (Could not connect, please try again)
2020-09-10 11:17:18 ERRO could not connect
2020-09-10 11:17:09 DEBU GET /api/v1/status (200) [43.564µs]
2020-09-10 11:16:36 DEBU POST /api/v1/accounts (400) [53.83915ms]
2020-09-10 11:16:36 DEBU api: abort /api/v1/accounts with code 400 (Could not connect, please try again)
2020-09-10 11:16:36 ERRO could not connect
2020-09-10 11:16:07 DEBU GET /api/v1/status (200) [50.171µs]

Using Cyberduck, I am able to connect to both PhotoPrism and Nextcloud using WebDAV, confirming that both instances are accessible.

[lastzero]

Did you try connecting without providing the exact path, just http://celluloid:8080/? We're performing auto-detection for common WebDAV path patterns. Not sure if your browser might filter or block passwords sent via http, I haven't seen a warning like in your screenshot before when testing locally. In general, you should always use HTTPS if possible, especially on servers that are exposed to the public internet.

Also note that Nextcloud WebDAV support might be broken under certain conditions, see WebDAV: Uploading to Nextcloud may result in 0 byte files #443.

[Jimmyson]

I have tried that address too, and still receiving the error. My server is not exposed to the Internet, but my internal network is resolving the calls. I am not utilizing HTTPS for any of my Docker Instances, as (from my understanding) it requires a Domain, and I am not sure how to setup HTTPS for a local, internal server. Reverse Proxies are a thing, but I am not understanding the documentation to set it up... The web browser I am utilizing is Firefox Dev 81.0b9

[lastzero]

Are you sure the Nextcloud server can be properly reached from within the Docker container using the host name? Name resolution can be different in some cases.

Do you see anything in the logs of your Nextcloud server so that you know the connection works on the network level? Any helpful error messages?

[Jimmyson]

Are you sure the Nextcloud server can be properly reached from within the Docker container using the host name? Name resolution can be different in some cases.

I am a little unsure on what you mean here.

Do you see anything in the logs of your Nextcloud server so that you know the connection works on the network level? Any helpful error messages?

Nothing in my Nextcloud logs are indication connection errors. Attaching to the Docker container does not log any GET or POST HTTP requests coming from Photoprism. Although, I am able to see requests from Firefox and also Nextcloud itself.

[lastzero]

It seems the PhotoPrism container cannot reach the Nextcloud instance for some reason, maybe because the host name celluloid only works on your host and not from within the PhotoPrism container? Is it configured in your local /etc/hosts file? What is the IP address of celluloid?

[Jimmyson]

My system hosts file contained a record for 127.0.1.1 celluloid My system's IP address is a static IP set by my router for 192.168.1.4. I added such record to the hosts file, and no luck.

Nextcloud Hosts:

root@99a60b068cc4:/var/www/html# cat /etc/hosts
127.0.0.1       localhost
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.18.0.4      99a60b068cc4

Photoprism Hosts:

root@338d3676cf17:/photoprism# cat /etc/hosts
127.0.0.1       localhost
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.18.0.3      338d3676cf17

Looking over the Docker Network setup I have a couple of interfaces setup.

jimmyson@celluloid:~$ docker network ls
NETWORK ID          NAME                     DRIVER              SCOPE
4e560fc93795        bridge                   bridge              local
93af30e04fa7        host                     host                local
b452fee5e1f0        jimmyson_default         bridge              local
332995b3c2e4        jimmyson_gitea-network   bridge              local
e7c1c66af704        none                     null                local

Inspecting the Network interfaces of both Nextcloud and Photoprism, they are sharing the same Network Interface...

jimmyson@celluloid:~$ docker inspect photoprism -f "{{json .NetworkSettings.Networks }}"
{"jimmyson_default":{"IPAMConfig":null,"Links":null,"Aliases":["photoprism","338d3676cf17"],"NetworkID":"b452fee5e1f0c309865bebdda82ca90ac03b980c56b8d2481b9bdca46846a96e","EndpointID":"9a7f4d19bb89b9a80909522b556364e4cb3d3653cf2c605e91b184c62e2b8e45","Gateway":"172.18.0.1","IPAddress":"172.18.0.3","IPPrefixLen":16,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":"02:42:ac:12:00:03","DriverOpts":null}}
jimmyson@celluloid:~$ docker inspect nextcloud -f "{{json .NetworkSettings.Networks }}"
{"jimmyson_default":{"IPAMConfig":null,"Links":null,"Aliases":["99a60b068cc4","nextcloud"],"NetworkID":"b452fee5e1f0c309865bebdda82ca90ac03b980c56b8d2481b9bdca46846a96e","EndpointID":"f60226cf3d58c236199cf09ee6c20c3fbeeb2a2bb473f3f288a06187ddfe3644","Gateway":"172.18.0.1","IPAddress":"172.18.0.4","IPPrefixLen":16,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":"02:42:ac:12:00:04","DriverOpts":null}}

[lastzero]

Without spending much time going through your complete networking setup, it seems obvious that PhotoPrism gets HTTP error 400 (invalid request) when trying to connect to 127.0.1.1 (localhost) via WebDAV as it will connect to itself.

Docker Compose typically takes care of networking so that you don't need to enter any IP addresses into /etc/hosts. Also you typically don't let it access your host for security reasons, it's a container ("jail") after all.

Here, for example, the photoprism service may connect to photoprism-db simply by using the name as host:

https://dl.photoprism.org/docker/docker-compose.yml

[lastzero]

Any progress / feedback on this? Was my suspicion going in the right direction?

[Jimmyson]

I haven't had any luck so far with the WebDAV access issue, but have not touched it for the last week.

[Elmardus]

I have the same issue, adding either the full path or the docker internal hostname, I both receive the same 400 error:

time="2020-11-10T15:58:15Z" level=error msg="could not connect"
time="2020-11-10T15:58:15Z" level=debug msg="api: abort /api/v1/accounts with code 400 (Could not connect, please try again)"
time="2020-11-10T15:58:15Z" level=debug msg="POST /api/v1/accounts (400) [235.410629ms]"

With the following Service URLs: https://nextcloud.domain.tld/remote.php/dav/files/Username/ or: http://nextcloud-app/remote.php/dav/files/Username/

[lastzero]

Can the domains nextcloud-app and nextcloud.domain.tld be resolved from inside the container? Do they maybe point to localhost or 127.0.0.1?

[Elmardus]

They resolve properly in the container, I used getent hosts nextcloud-app and getent hosts nextcloud.domain.tld, they resolve to the nextcloud-app container IP and my public IP respectively.

[Elmardus]

Anything else I can try? I would love to use Photoprism, but whatever I try I can't link up Nextcloud.

[lastzero]

I'd love to debug this further with you, but need to take care of existing bug reports first. Did you take a look at #443? It seems Nextcloud's behaviour depends a lot on the Web server (nginx, apache,...) and its specific configuration.

[lastzero]

Is this still an issue?

[Elmardus]

Yes it's still an issue. I'm now running PhotoPrism 201207-a43f8be2-Linux-x86_64 via docker-compose. I've looked at #443, I also use the nextcloud-apache container (no php-fpm/fast-cgi), but since I can't add the WebDAV server I haven't run into the same 0-byte file issue as discribed there. Anything else I can do to help debug?

[lastzero]

I'm happy to help once the release is out of the door.

[greendragon2010]

I have this issue as well. I have them on the same host machine, and I even moved photoprism to it's own IP address and still having issues. It connects successfully when I test it with try.nextcloud.com. I've used hostnames (LAN and docker hostnames) and I've used the ip address. I made sure they that all of these were valid in the nextcloud config.
I'm using linuxserver io for nextcloud and it is set up to use ngninx. I am using your docker compose for photoprism.
Latest Version I've tested on: Nextcloud: 20.0.3 Photoprism: 201209-e87ab1cb-Linux-x86_64 I enable debug for photoprism and the log messages just said the same as above. Searching all the nextcloud logs can't see any logs pertaining to the failure.

[praul]

Same here. I added WebDAV-Server to photoprisms docker-compose, they share a network, but no connection possible, neither with http://externalip:port/ http://internalip:port/ http://127.0.0.1:port/ or with something like this photoprism:insecure@tcp(webdav:80)

[praul]

I found somewhat of a workaround: If you first add a remote nextcloud server, you can change the server adress afterwards to local and it works!!

[lastzero]

Interesting, maybe there is an issue with how we parse the host name!?

[greendragon2010]

praul thank you. The workaround worked for me. That gave me, more detail errors that photoprism had issues with the self sign certificates for nextcloud. Once I turned off enforcing https (only access to either is local lan) on nextcloud sync is starting.

[lastzero]

Oh, you've been using invalid HTTPS certificates? Blocking them is a feature, not a bug - but we should show a helpful error message in this case!

[greendragon2010]

I think that was part of the issue, just was used to having a pop up saying to ignore it, or can't connect because of it. I've successfully have been able to remove, add, and modify accounts to my local owncloud server. I completely agree with blocking invalid HTTPS certificates, but I agree at least having a log or notification about the invalid certificate would be helpful. I completely forgot about the certificates and it rerouting http to https connections till I got the error with the workaround.
Wonder if that was the original posters issue as well?

[Elmardus]

The certificate I use is valid, and I am not able to get a connection using http either

[praul]

I am not using any encryption or certificate, just plain http. It works with the workaround, but not in the initial setup

Also: While the workaround works, photoprism is not saving any configured folders, and always backups to root

[iamarealperson1]

I am also having this issue too, trying to connect to Nextcloud over HTTPS. My local Nextcloud WebDAV address is: http://192.168.1.1:450/remote.php/dav/files/xxxx/ where 192.168.1.1 is my server's IP on the network.

In my case I could not connect to my remote Nextcloud server with Web DAV address: https://nextcloud.mysite.com/remote.php/dav/files/xxxx/

In my Photoprism logs i see: 2020-12-28 00:55:03 WARN sync: Error 1045: Access denied for user 'photoprism'@'photoprism_photoprism_1.nextcloud_default' (using password: YES) where I think 'photoprism' is my username for an account I have crated on Nextcloud for sync purposes.

If I run getent hosts nextcloud from within my container it returns 172.18.0.2 nextcloud which is correct.

Any ideas what to try?

[lastzero]

Started a preview build, you may test when it's done: https://drone.photoprism.app/photoprism/photoprism/915

[doreks]

t add a remote nextcloud server, you can change the server adress afterwards to

please, could you give us more details on how the procedure works and in which menus...

[lastzero]

I was able to sync via WebDAV using these settings, which connect directly to ownCloud via Docker's internal network and without using HTTPS:

[FrankelJb]

Sorry for the necrobump. I am encountering the same error as this now when connecting to Nextcloud.

My setup is using k3s, a smaller Kubernetes distribution from Rancher. DNS resolution is working correctly as I am not receiving a 404 not found, but rather the same 400 error and "Could not connect, please try again". I am able to connect to the WebDAV from my local PC and I can curl the endpoint from a command line running inside the Photoprism container.

I'd be happy to share my setup as I'm confident that this is related to Photoprism.

[lastzero]

Is there any kind of proxy, like nginx or k3s magic, in between or can PhotorPrism directly access the Apache server Nextcloud is running on?

[FrankelJb]

Networking on k3s/k8s is pretty much magic as there is a lot existing plumbing to make things work. I've tried to use both the direct connection using the inter-pod networking, via http://nextcloud:8080, and via the ingress (using Traefik). I can access the Traefik endpoint using davs://<etc> on my LAN.

I also tried the Nginx based NextCloud and the logs contain 127.0.0.1 - admin 25/May/2021:18:10:07 +0000 "PROPFIND /index.php" 400.

I'm still using self-signed certs, the davs connection asks me if I want to continue. Could this be causing PhotoPrism to reject the connection?

[lastzero]

Yes, you need valid certs when using https, but looks like you're using http? We've learned that nginx doesn't support http features needed for webdav fully, so there might be similar issues.

[FrankelJb]

I was using the Apache version first and encountered the same challenge. This leads me to believe this isn't on the Nextcloud side

[lastzero]

Did you enter the full webdav resource path or just the host as shown above?

[FrankelJb]

Thanks @lastzero! It was a problem with an old version of Nextcloud. I upgraded to v20 and everything is working again.

I love the software, keep up the great work and thanks for taking time to help.

[timdonovanuk]

Holy complexity batman.

Spent 3 hours trying to get Nextcloud working with Photoprism (so I can share albums without exposing my entire collection). The only error to go on has been level=error msg="could not connect"

I'm guessing its because my nextcloud certificate isn't valid? Without a lot of effort AFAIK it's not easy to get a valid certificate for a .home domain... (you have to become your own CA and then self sign root certificates etc. etc.)? It also seems to be a huge pain in the ass to disable HTTPS on Nextcloud (honestly I'm new to nextcloud but the entire thing looks like a php hot mess).

Please can we have a button "My webdav is running locally so I really don't need to care about invalid ceritifactes"? I understand the intention is to keep things secure, but this is actually going to have the opposite effect - you're going to force people to expose their nextcloud directly to the internet so they can actually generate a proper certificate, and then make us route Photoprism webdav connections via public URLs!

[timdonovanuk]

I have confirmed, by putting nextcloud out on the world wide web for everyone to attempt to hack into that I can now successfully connect Photoprism to it...:cold_sweat:

[lastzero]

Get a let's encrypt DNS wildcard cert.

[lastzero]

Or use HTTP on the internal network 👍

[timdonovanuk]

As mentioned you cannot easily get certs for local domains (e.g. .home/.local) and as mentioned, Nextcloud (at least the latest version) does not allow http traffic, it automatically forwards to https.

[lastzero]

Right, you'd need to pay for a regular domain. Can be managed for free at DigitalOcean.

[timdonovanuk]

So back to my original point of good intentions of trying to make things secure, but the unintended consequence is making it so you have to expose nextcloud directly to the internet and then route Photoprism webdav connections via public URL :)

As it transpires, it looks like Photoprism strips EXIF data when sharing stuff over Webdav so I need to abandon this anyway.

[lastzero]

No, you can use an internal network together with HTTP (not less secure) or pay for a domain to use HTTPS incl proper validation :)

[lastzero]

There's a dropdown to select the resolution. EXIF only gets removed for thumbs, not originals. It would be too complex to continuously sync Exif data across all thumbs and sizes. Also you might not want to share it when using a small thumb.

[timdonovanuk]

@lastzero I'm not sure you can run NextCloud in HTTP (not easily anyway :( )

Sorry I opened another ticket for the EXIF issue, as its HEIC related. I don't see any resolution selector anywhere when sharing though?

[lastzero]

We're running Nextcloud via HTTP using their official Docker container for testing. Why shouldn't this work?

[timdonovanuk]

Hmm I'm using the ghcr.io/linuxserver/nextcloud version. It only exposes port HTTPS on port 433. Port 80 has nothing on it, even if you expose it. I'll give the offical one a go, but I see the linuxserver version is just as popular! Cheers :)

[justmytwospence]

Just another confirmation from me that http://nextcloud (when photoprism and nextcloud are on the same docker network) does not work at first, but DOES work if I use my external network address (https://nextcloud.mydomain.com) and then change it to http://nextcloud.