search

bolcom / unFTP

A FTP(S) server with a couple of twists written in Rust. Follow and talk to us on https://t.me/unftp. Docs on https://unftp.rs
Apache License 2.0
274 stars 21 forks source link

Unable to get docker container working #164

Open mabushey opened 1 year ago

mabushey commented 1 year ago

Hello,

This looks perfect for what I need.

My docker run:

docker run \
--name paperless-ftp \
-e ROOT_DIR=/ftp \
-e AUTH_TYPE=json \
-e AUTH_JSON_PATH=/credentials.json \
-e UNFTP_PASSIVE_PORTS=50000-50005 \
-p 2121:2121 \
-p 50000:50000 \
-p 50001:50001 \
-p 50002:50002 \
-p 50003:50003 \
-p 50004:50004 \
-p 50005:50005 \
-p 8080:8080 \
-v /home/michael/docker/paperless/paperless-ngx/credentials.json:/credentials.json \
-v /home/michael/docker/paperless/paperless-ngx/consume:/ftp \
-ti \
bolcom/unftp:v0.14.2-alpine

I created a credentials.json file:

[
  {
    "username": "scan",
    "password": "REDACTED"
  }
]

This works - I can read it from inside the container and the ftp and the log contains:

   Dec 06 01:42:56.282 INFO PASS: User User(username: "scan", name: None, surname: None) logged in

First question: If I exec in and run touch /ftp/test I get this:

$ ls -l /home/michael/docker/paperless/paperless-ngx/consume
total 0
-rw-r--r-- 1    1010    1010 0 Dec  5 17:57 test
-rw-rw-r-- 1 michael michael 0 Dec  5 17:34 this_is_a_test.mkv

Is there anyway to set the uid/gid to 1000?

Second question:

$ docker ps | grep ftp
00c7917fb8a8   bolcom/unftp:v0.14.2-alpine                  "/unftp/unftp -vv"       21 minutes ago   Up 21 minutes          0.0.0.0:2121->2121/tcp, :::2121->2121/tcp, 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp, 0.0.0.0:50000-50005->50000-50005/tcp, :::50000-50005->50000-50005/tcp   paperless-ftp

from the log:

 Dec 06 01:57:21.667 INFO Listening on passive port 172.17.0.2:50000

from lftp:

$ ftp 192.168.192.103 2121                                                                                                                                                                                                                  ✹ ✭master 
Connected to 192.168.192.103.
220 Welcome to unFTP
Name (192.168.192.103:michael): scan
331 Password Required
Password: 
230 User logged in, proceed
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> ls
227 Entering Passive Mode (172,17,0,2,195,81)
Passive mode address mismatch.
ftp> exit
221 Bye!

I believe it's translating between the docker IP and the LAN IP. While dockers 0.0.0.0:2121->2121/tcp binds to all IPs, it appears unftp only wants the docker IP.

If I locally test it with the docker IP:

ftp> ls -l
227 Entering Passive Mode (172,17,0,2,195,81)
150 Sending directory list
226 Listed the directory
ftp> ls
227 Entering Passive Mode (172,17,0,2,195,83)
150 Sending directory list
226 Listed the directory

apparently more permission problems...

$ docker exec -it paperless-ftp ls -l /ftp
total 0
-rw-r--r--    1 unftp    unftp            0 Dec  6 01:57 test
-rw-rw-r--    1 1000     1000             0 Dec  6 01:34 this_is_a_test.mkv
robklg commented 12 months ago

Hi and welcome!

bolcom/unftp:v0.14.2-alpine

Be aware, the latest version is v0.14.4.

Is there anyway to set the uid/gid to 1000?

This is in the Dockerfile. So you could build your own (edit alpine.Dockerfile.template; and run make docker-image). Or you could try overriding by adding --user 1000 to the docker run command.

Passive mode address mismatch.

By default it uses the IP from the connection. If there was network translation this method won't work. unFTP has no way of knowing what the right passive host is. In this case you need to configure unFTP to handle it differently, using:

-e UNFTP_PASSIVE_HOST

You can set the argument to either a DNS hostname or a specific IP address. So:

-e UNFTP_PASSIVE_HOST=192.168.192.103

Or indeed a hostname that it will resolve at startup.

Hope that helps!

mabushey commented 12 months ago

@robklg Thank you for your time and answers!

Problem 1: Adding --user 1000:1000 to the docker run fixed the permission issues. I didn't realize that was a docker option, thank you!

Problem 2:

 Dec 06 17:14:37.998 INFO Resolved 'doc.[REDACTED]' to 192.168.192.103                                                                                                                                                                                                                                 
 Dec 06 17:14:37.998 INFO Using passive host option 'Ip(192.168.192.103)'

Fixed, thank you!

Almost there, but one new problem:

If I use Filezilla and upload to /, the file goes in /tmp. I'm mounting my "scan" volume to /ftp and using the ENV ROOT_DIR=/ftp. Doesn't seem to work, but I do find the docs confusing on this.

Scanner connecting:

  Dec 06 17:35:12.884 INFO Incoming control connection from 192.168.192.15:39726
 trace-id: 0x96d6d920ec11ed15
  source: 192.168.192.15:39726
   Dec 06 17:35:12.884 INFO Starting control loop
   Dec 06 17:35:12.891 INFO PASS: User User(username: "scan", name: None, surname: None) logged in
   Dec 06 17:35:12.908 WARN CWD: Failed to change directory "/ftp": storage error: 550 Permanent file not available 
   Dec 06 17:35:12.913 INFO QUIT: User logged out
 Dec 06 17:35:17.736 INFO Incoming control connection from 192.168.192.15:39728
 trace-id: 0xec119895e6e6aa37
  source: 192.168.192.15:39728
   Dec 06 17:35:17.737 INFO Starting control loop
   Dec 06 17:35:17.744 INFO PASS: User User(username: "scan", name: None, surname: None) logged in
   Dec 06 17:35:17.754 WARN CWD: Failed to change directory "/~": storage error: 550 Permanent file not available 
   Dec 06 17:35:17.764 INFO QUIT: User logged out

The scanner really wants to cd to /~. Isn't that what the ENV ROOT_DIR=/ftp should do?

Thanks again!

robklg commented 12 months ago

The scanner really wants to cd to /~. Isn't that what the ENV ROOT_DIR=/ftp should do?

Yes you are reasoning it correctly. Took me a moment to see the problem, I think the problem is you miss the UNFTP_ prefix there. So -e UNFTP_ROOT_DIR=/ftp ought to work. That also means the user credentials are probably also not picked up either (!).

So that means our documentation needs improvement. Till then, to list all the arguments + the env vars, you can use

docker run -ti -u 1000 bolcom/unftp:v0.14.4-alpine /unftp/unftp -h

mabushey commented 12 months ago

@robklg

Thanks again, you are correct. My new docker run:

docker run \
--name paperless-ftp \
--user 1000:1000 \
-e UNFTP_ROOT_DIR=/ftp \
-e UNFTP_AUTH_TYPE=json \
-e UNFTP_AUTH_JSON_PATH=/credentials.json \
-e UNFTP_PASSIVE_PORTS=50000-50005 \
-e UNFTP_PASSIVE_HOST=doc.[REDACTED] \
-p 2121:2121 \
-p 50000:50000 \
-p 50001:50001 \
-p 50002:50002 \
-p 50003:50003 \
-p 50004:50004 \
-p 50005:50005 \
-p 8080:8080 \
-v /home/michael/docker/paperless/paperless-ngx/credentials.json:/credentials.json \
-v /home/michael/docker/paperless/paperless-ngx/consume:/ftp \
-ti \
bolcom/unftp:v0.14.4-alpine

Now in Filezilla / shows up as the contents of /ftp. I still have an issue that the scanner is insisting on going into the home directory, which does not seem to work:

Dec 06 18:26:39.772 INFO Incoming control connection from 192.168.192.15:39732
 trace-id: 0xbaedf5cbce01f4e2
  source: 192.168.192.15:39732
   Dec 06 18:26:39.773 INFO Starting control loop
   Dec 06 18:26:39.782 INFO PASS: User User(username: "scan", name: None, surname: None) logged in
   Dec 06 18:26:39.791 WARN CWD: Failed to change directory "/~": storage error: 550 Permanent file not available 
   Dec 06 18:26:39.796 INFO QUIT: User logged out
robklg commented 12 months ago

I think the scanner is trying to cd ~ into its homedir. Which unFTP doesn’t understand. So that’s why unFTP tries to change to the literal ~, making it/~. I think ~ is more of a shell expansion feature. I don’t know if it’s common for ftp servers to implement this as well. It’s something for us to look into…

Meanwhile, you could try as a workaround to create a symlink there:

ln -s /ftp/\~ /

I admit it’s a bit dirty but I think it should solve your issue for now.