search

drakkan / sftpgo

Full-featured and highly configurable SFTP, HTTP/S, FTP/S and WebDAV server - S3, Google Cloud Storage, Azure Blob
https://sftpgo.com
GNU Affero General Public License v3.0
9.12k stars 713 forks source link

[Bug]: sftpgo-plugin-auth doesn't load in drakkan/sftpgo:v2.5.4-plugins #1373

Closed Macleykun closed 1 year ago

Macleykun commented 1 year ago

⚠️ This issue respects the following points: ⚠️

Bug description

When setting up sftpgo with the auth plugin i noticed the following error which i believe may be bug. When i try to use the sftpgo-plugin-auth i see this in my docker container:

{"level":"error","time":"2023-07-28T16:30:43.024","sender":"service","message":"unable to initialize plugin system: Unrecognized remote plugin message: Incorrect Usage: flag provided but not defined: -ldap-url\nThis usually means\n  the plugin was not compiled for this architecture,\n  the plugin is missing dynamic-link libraries necessary to run,\n  the plugin is not executable by this process due to file permissions, or\n  the plugin failed to negotiate the initial go-plugin protocol handshake\n\nAdditional notes about plugin:\n  Path: /usr/local/bin/sftpgo-plugin-auth\n  Mode: -rwxr-xr-x\n  Owner: 0 [root] (current: 1000 [sftpgo])\n  Group: 0 [root] (current: 1000 [sftpgo])\n  ELF architecture: EM_X86_64 (current architecture: amd64)\n"}

I am using a Ryzen CPU on my host machine > VMware workstation VM reports x86_64 > the container itself reports also x86_64.

The reason why i report this as a bug, is that i use the drakkan/sftpgo:v2.5.4-plugins container and only when providing the configuration for the sftpgo-plugin-auth i recieve this error. It also runs fine withoud specifying the sftpgo-plugin-auth information.

Steps to reproduce

  1. docker run --name some-sftpgo \ -p 443:8080 \ -p 2022:2022 \ --mount type=bind,source=/my/own/sftpgodata,target=/srv/sftpgo \ --mount type=bind,source=/my/own/sftpgohomeconfig,target=/var/lib/sftpgo \ --mount type=bind,source=${PWD}/mcdonalds-ssl,target=/tmp \ --mount type=bind,source=${PWD}/sftpgo.json,target=/etc/sftpgo/sftpgo.json \ -e SFTPGO_GRACE_TIME=30 \ -d "drakkan/sftpgo:v2.5.4-plugins"
  2. docker logs some-sftpgo sftpgo-config.zip

Expected behavior

The container should start with the plugin chosen to be loaded by sftpgo.

SFTPGo version

2.5.4

Data provider

local (soon AD)

Installation method

Community Docker image

Configuration

{
  "sftpd": {
    "max_auth_tries": 0,
    "enabled_ssh_commands": [
      "md5sum",
      "sha1sum",
      "sha256sum",
      "cd",
      "pwd",
      "scp"
    ],
    "keyboard_interactive_authentication": true,
    "password_authentication": true,
    "folder_prefix": ""
  },
  "httpd": {
    "bindings": [
      {
        "port": 8080,
        "address": "",
        "enable_web_admin": true,
        "enable_web_client": true,
        "enable_rest_api": true,
        "enabled_login_methods": 0,
        "enable_https": true,
        "certificate_file": "/tmp/sftp.mcdonalds.local.full.pem",
        "certificate_key_file": "/tmp/sftp.mcdonalds.local.key",
        "min_tls_version": 12,
        "client_auth_type": 0,
        "hide_login_url": 0,
        "oidc": {
          "client_id": "",
          "client_secret": "",
          "config_url": "",
          "redirect_base_url": "",
          "scopes": [
            "openid",
            "profile",
            "email"
          ],
          "username_field": "",
          "role_field": "",
          "implicit_roles": false,
          "custom_fields": [],
          "insecure_skip_signature_check": false,
          "debug": false
        },
        "security": {
          "enabled": true,
          "allowed_hosts": [],
          "allowed_hosts_are_regex": false,
          "hosts_proxy_headers": [],
          "https_redirect": true,
          "https_host": "",
          "https_proxy_headers": []
        },
        "branding": {
          "web_admin": {
            "name": "This will show in the tab",
            "short_name": "Shows mostly everywhere",
            "favicon_path": "",
            "logo_path": "",
            "login_image_path": "",
            "disclaimer_name": "",
            "disclaimer_path": "",
            "default_css": "",
            "extra_css": []
          }
        }
      }
    ],
    "openapi_path": "openapi",
    "web_root": "",
    "certificate_file": "/tmp/sftp.mcdonalds.local.full.pem",
    "certificate_key_file": "/tmp/sftp.mcdonalds.local.key",
    "ca_certificates": [
        "/tmp/rootca.crt"
    ],
    "max_upload_file_size": 0
  },
  "smtp": {
    "host": "",
    "port": 25,
    "from": "",
    "user": "",
    "password": "",
    "auth_type": 0,
    "encryption": 0,
    "domain": "",
    "templates_path": "templates",
    "debug": 0
  },
  "plugins": [
    {
      "cmd": "/usr/local/bin/sftpgo-plugin-auth",
      "type": "auth",
      "auth_options": {
        "scope": 1
      },
      "args": [
        "--ldap-url=ldap://192.168.15.168:389",
        "--ldap-base-dn=dc=mcdonalds,dc=local",
        "--ldap-bind-dn=CN=Wesley de Vree,CN=Users,DC=mcdonalds,DC=local",
        "--ldap-password=TestWindowsAD2023!",
        "--skip-tls-verify=1"
      ]
    }
  ]
}

Relevant log output

{"level":"info","time":"2023-07-28T16:30:42.995","sender":"service","message":"starting SFTPGo 2.5.4-cc381443-2023-07-25T05:50:51Z +metrics +azblob +gcs +s3 +bolt +mysql +pgsql +sqlite +unixcrypt +portable, config dir: ., config file: , log max size: 10 log max backups: 5 log max age: 28 log level: debug, log compress: false, log utc time: false, load data from: \"\", grace time: 30 secs"}
{"level":"info","time":"2023-07-28T16:30:42.997","sender":"config","message":"unable to read env files from \"env.d\": open env.d: no such file or directory"}
{"level":"debug","time":"2023-07-28T16:30:43.007","sender":"config","message":"config file used: '\"/etc/sftpgo/sftpgo.json\"', config loaded: {Common:{IdleTimeout:15 UploadMode:0 Actions:{ExecuteOn:[] ExecuteSync:[] Hook:} SetstatMode:0 RenameMode:0 TempPath: ProxyProtocol:0 ProxyAllowed:[] ProxySkipped:[] StartupHook: PostConnectHook: PostDisconnectHook: DataRetentionHook: MaxTotalConnections:0 MaxPerHostConnections:20 AllowListStatus:0 AllowSelfConnections:0 DefenderConfig:{Enabled:false Driver:memory BanTime:30 BanTimeIncrement:50 Threshold:15 ScoreInvalid:2 ScoreValid:1 ScoreLimitExceeded:3 ScoreNoAuth:0 ObservationTime:30 EntriesSoftLimit:100 EntriesHardLimit:150} RateLimitersConfig:[{Average:0 Period:1000 Burst:1 Type:2 Protocols:[SSH FTP DAV HTTP] GenerateDefenderEvents:false EntriesSoftLimit:100 EntriesHardLimit:150}] idleTimeoutAsDuration:0 idleLoginTimeout:0 defender:<nil> allowList:<nil> rateLimitersList:<nil> proxyAllowed:[] proxySkipped:[]} ACME:{Email: KeyType:4096 CertsPath:certs CAEndpoint:https://acme-v02.api.letsencrypt.org/directory Domains:[] RenewDays:30 HTTP01Challenge:{Port:80 WebRoot: ProxyHeader:} TLSALPN01Challenge:{Port:0} accountConfigPath: accountKeyPath: lockPath: tempDir:} SFTPD:{Banner:SFTPGo_2.5.4 Bindings:[{Address: Port:2022 ApplyProxyConfig:true}] MaxAuthTries:0 HostKeys:[] HostCertificates:[] HostKeyAlgorithms:[] Moduli:[] KexAlgorithms:[] Ciphers:[] MACs:[] TrustedUserCAKeys:[] RevokedUserCertsFile: LoginBannerFile: EnabledSSHCommands:[md5sum sha1sum sha256sum cd pwd scp] KeyboardInteractiveAuthentication:true KeyboardInteractiveHook: PasswordAuthentication:true FolderPrefix: certChecker:<nil> parsedUserCAKeys:[]} FTPD:{Bindings:[{Address: Port:0 ApplyProxyConfig:true TLSMode:0 CertificateFile: CertificateKeyFile: MinTLSVersion:12 ForcePassiveIP: PassiveIPOverrides:[] PassiveHost: ClientAuthType:0 TLSCipherSuites:[] PassiveConnectionsSecurity:0 ActiveConnectionsSecurity:0 Debug:false ciphers:[]}] Banner:SFTPGo 2.5.4 ready BannerFile: CertificateFile: CertificateKeyFile: CACertificates:[] CARevocationLists:[] ActiveTransfersPortNon20:true DisableActiveMode:false EnableSite:false HASHSupport:0 CombineSupport:0 PassivePortRange:{Start:50000 End:50100} acmeDomain:} WebDAVD:{Bindings:[{Address: Port:0 EnableHTTPS:false CertificateFile: CertificateKeyFile: MinTLSVersion:12 ClientAuthType:0 TLSCipherSuites:[] Prefix: ProxyAllowed:[] ClientIPProxyHeader: ClientIPHeaderDepth:0 DisableWWWAuthHeader:false allowHeadersFrom:[]}] CertificateFile: CertificateKeyFile: CACertificates:[] CARevocationLists:[] Cors:{AllowedOrigins:[] AllowedMethods:[] AllowedHeaders:[] ExposedHeaders:[] AllowCredentials:false Enabled:false MaxAge:0 OptionsPassthrough:false OptionsSuccessStatus:0 AllowPrivateNetwork:false} Cache:{Users:{ExpirationTime:0 MaxSize:50} MimeTypes:{Enabled:true MaxSize:1000 CustomMappings:[]}} acmeDomain:} ProviderConf:{Driver:sqlite Name:sftpgo.db Host: Port:0 Username: Password: SSLMode:0 DisableSNI:false TargetSessionAttrs: RootCert: ClientCert: ClientKey: ConnectionString: SQLTablesPrefix: TrackQuota:2 PoolSize:0 UsersBaseDir: Actions:{ExecuteOn:[] ExecuteFor:[] Hook:} ExternalAuthHook: ExternalAuthScope:0 PreLoginHook: PostLoginHook: PostLoginScope:0 CheckPasswordHook: CheckPasswordScope:0 UpdateMode:0 PasswordHashing:{BcryptOptions:{Cost:10} Argon2Options:{Memory:65536 Iterations:1 Parallelism:2} Algo:bcrypt} PasswordValidation:{Admins:{MinEntropy:0} Users:{MinEntropy:0}} PasswordCaching:true DelayedQuotaUpdate:0 CreateDefaultAdmin:false NamingRules:1 IsShared:0 Node:{Host: Port:0 Proto:http} BackupsPath:backups} HTTPDConfig:{Bindings:[{Address: Port:8080 EnableWebAdmin:true EnableWebClient:true EnableRESTAPI:true EnabledLoginMethods:0 EnableHTTPS:true CertificateFile:/tmp/sftp.mcdonalds.local.full.pem CertificateKeyFile:/tmp/sftp.mcdonalds.local.key MinTLSVersion:12 ClientAuthType:0 TLSCipherSuites:[] ProxyAllowed:[] ClientIPProxyHeader: ClientIPHeaderDepth:0 HideLoginURL:0 RenderOpenAPI:true WebClientIntegrations:[] OIDC:{ClientID: ClientSecret: ConfigURL: RedirectBaseURL: UsernameField: RoleField: ImplicitRoles:false Scopes:[openid profile email] CustomFields:[] InsecureSkipSignatureCheck:false Debug:false provider:<nil> verifier:<nil> providerLogoutURL: oauth2Config:<nil>} Security:{Enabled:true AllowedHosts:[] AllowedHostsAreRegex:false HostsProxyHeaders:[] HTTPSRedirect:true HTTPSHost: HTTPSProxyHeaders:[] STSSeconds:0 STSIncludeSubdomains:false STSPreload:false ContentTypeNosniff:false ContentSecurityPolicy: PermissionsPolicy: CrossOriginOpenerPolicy: ExpectCTHeader: proxyHeaders:[]} Branding:{WebAdmin:{Name:This will show in the tab ShortName:Shows mostly everywhere LogoPath: LoginImagePath: FaviconPath: DisclaimerName: DisclaimerPath: DefaultCSS: ExtraCSS:[]} WebClient:{Name: ShortName: LogoPath: LoginImagePath: FaviconPath: DisclaimerName: DisclaimerPath: DefaultCSS: ExtraCSS:[]}} allowHeadersFrom:[]}] TemplatesPath:templates StaticFilesPath:static OpenAPIPath:openapi WebRoot: CertificateFile:/tmp/sftp.mcdonalds.local.full.pem CertificateKeyFile:/tmp/sftp.mcdonalds.local.key CACertificates:[/tmp/rootca.crt] CARevocationLists:[] SigningPassphrase: TokenValidation:0 MaxUploadFileSize:0 Cors:{AllowedOrigins:[] AllowedMethods:[] AllowedHeaders:[] ExposedHeaders:[] AllowCredentials:false Enabled:false MaxAge:0 OptionsPassthrough:false OptionsSuccessStatus:0 AllowPrivateNetwork:false} Setup:{InstallationCode: InstallationCodeHint:Installation code} HideSupportLink:false acmeDomain:} HTTPConfig:{Timeout:20 RetryWaitMin:2 RetryWaitMax:30 RetryMax:3 CACertificates:[] Certificates:[] SkipTLSVerify:false Headers:[] customTransport:<nil>} CommandConfig:{Timeout:30 Env:[] Commands:[]} KMSConfig:{Secrets:{URL: MasterKeyPath: MasterKeyString: masterKey:}} MFAConfig:{TOTP:[{Name:Default Issuer:SFTPGo Algo:sha1 algo:0}]} TelemetryConfig:{BindPort:0 BindAddress:127.0.0.1 EnableProfiler:false AuthUserFile: CertificateFile: CertificateKeyFile: TLSCipherSuites:[] MinTLSVersion:12} PluginsConfig:[{Type:auth NotifierOptions:{FsEvents:[] ProviderEvents:[] ProviderObjects:[] LogEvents:[] RetryMaxTime:0 RetryQueueMaxSize:0} KMSOptions:{Scheme: EncryptedStatus:} AuthOptions:{Scope:1} Cmd:/usr/local/bin/sftpgo-plugin-auth Args:[--ldap-url=ldap://192.168.15.168:389 --ldap-base-dn=dc=mcdonalds,dc=local --ldap-bind-dn=CN=Wesley de Vree,CN=Users,DC=mcdonalds,DC=local --ldap-password=TestWindowsAD2023! --skip-tls-verify=1] SHA256Sum: AutoMTLS:false kmsID:0}] SMTPConfig:{Host: Port:25 From: User: Password: AuthType:0 Encryption:0 Domain: TemplatesPath:templates Debug:0 OAuth2:{Provider:0 Tenant: ClientID: ClientSecret: RefreshToken: mu:<nil> config:<nil> accessToken:<nil>}}}"}
{"level":"info","time":"2023-07-28T16:30:43.007","sender":"kms","message":"secret provider registered for scheme: \"builtin\", encrypted status: \"AES-256-GCM\""}
{"level":"info","time":"2023-07-28T16:30:43.007","sender":"kms","message":"secret provider registered for scheme: \"local\", encrypted status: \"Secretbox\""}
{"level":"debug","time":"2023-07-28T16:30:43.007","sender":"dataprovider_sqlite","message":"sqlite database handle created, connection string: \"file:sftpgo.db?cache=shared&_foreign_keys=1\""}
{"level":"debug","time":"2023-07-28T16:30:43.009","sender":"dataprovider_sqlite","message":"sql database is up to date, current version: 28"}
{"level":"debug","time":"2023-07-28T16:30:43.011","sender":"dataprovider_sqlite","message":"absolute backup path \"/var/lib/sftpgo/backups\""}
{"level":"debug","time":"2023-07-28T16:30:43.011","sender":"plugins","message":"initialize"}
{"level":"debug","time":"2023-07-28T16:30:43.011","sender":"dataprovider_sqlite","message":"delayed quota update loop started, wait time: 0s"}
{"level":"debug","time":"2023-07-28T16:30:43.012","sender":"dataprovider_sqlite","message":"delayed quota update loop ended, wait time: 0s"}
{"level":"debug","time":"2023-07-28T16:30:43.013","sender":"plugins","message":"no match for plugin process /usr/local/bin/sftpgo-plugin-auth"}
{"level":"debug","time":"2023-07-28T16:30:43.013","sender":"plugins","message":"create new auth plugin \"/usr/local/bin/sftpgo-plugin-auth\""}
{"level":"debug","time":"2023-07-28T16:30:43.015","sender":"plugins.auth","path":"/usr/local/bin/sftpgo-plugin-auth","args":"[/usr/local/bin/sftpgo-plugin-auth --ldap-url=ldap://192.168.15.168:389 --ldap-base-dn=dc=mcdonalds,dc=local --ldap-bind-dn=CN=Wesley de Vree,CN=Users,DC=mcdonalds,DC=local --ldap-password=TestWindowsAD2023! --skip-tls-verify=1]","message":"starting plugin"}
{"level":"debug","time":"2023-07-28T16:30:43.015","sender":"plugins.auth","path":"/usr/local/bin/sftpgo-plugin-auth","pid":"12","message":"plugin started"}
{"level":"debug","time":"2023-07-28T16:30:43.015","sender":"plugins.auth","path":"/usr/local/bin/sftpgo-plugin-auth","message":"waiting for RPC address"}
{"level":"debug","time":"2023-07-28T16:30:43.019","sender":"plugins","message":"unable to get rpc client for kms plugin \"/usr/local/bin/sftpgo-plugin-auth\": Unrecognized remote plugin message: Incorrect Usage: flag provided but not defined: -ldap-url\nThis usually means\n  the plugin was not compiled for this architecture,\n  the plugin is missing dynamic-link libraries necessary to run,\n  the plugin is not executable by this process due to file permissions, or\n  the plugin failed to negotiate the initial go-plugin protocol handshake\n\nAdditional notes about plugin:\n  Path: /usr/local/bin/sftpgo-plugin-auth\n  Mode: -rwxr-xr-x\n  Owner: 0 [root] (current: 1000 [sftpgo])\n  Group: 0 [root] (current: 1000 [sftpgo])\n  ELF architecture: EM_X86_64 (current architecture: amd64)\n"}
{"level":"warn","time":"2023-07-28T16:30:43.024","sender":"plugins","message":"unable to create auth plugin: Unrecognized remote plugin message: Incorrect Usage: flag provided but not defined: -ldap-url\nThis usually means\n  the plugin was not compiled for this architecture,\n  the plugin is missing dynamic-link libraries necessary to run,\n  the plugin is not executable by this process due to file permissions, or\n  the plugin failed to negotiate the initial go-plugin protocol handshake\n\nAdditional notes about plugin:\n  Path: /usr/local/bin/sftpgo-plugin-auth\n  Mode: -rwxr-xr-x\n  Owner: 0 [root] (current: 1000 [sftpgo])\n  Group: 0 [root] (current: 1000 [sftpgo])\n  ELF architecture: EM_X86_64 (current architecture: amd64)\n, config {Type:auth NotifierOptions:{FsEvents:[] ProviderEvents:[] ProviderObjects:[] LogEvents:[] RetryMaxTime:0 RetryQueueMaxSize:0} KMSOptions:{Scheme: EncryptedStatus:} AuthOptions:{Scope:1} Cmd:/usr/local/bin/sftpgo-plugin-auth Args:[--ldap-url=ldap://192.168.15.168:389 --ldap-base-dn=dc=mcdonalds,dc=local --ldap-bind-dn=CN=Wesley de Vree,CN=Users,DC=mcdonalds,DC=local --ldap-password=TestWindowsAD2023! --skip-tls-verify=1] SHA256Sum: AutoMTLS:false kmsID:0}"}
{"level":"error","time":"2023-07-28T16:30:43.024","sender":"service","message":"unable to initialize plugin system: Unrecognized remote plugin message: Incorrect Usage: flag provided but not defined: -ldap-url\nThis usually means\n  the plugin was not compiled for this architecture,\n  the plugin is missing dynamic-link libraries necessary to run,\n  the plugin is not executable by this process due to file permissions, or\n  the plugin failed to negotiate the initial go-plugin protocol handshake\n\nAdditional notes about plugin:\n  Path: /usr/local/bin/sftpgo-plugin-auth\n  Mode: -rwxr-xr-x\n  Owner: 0 [root] (current: 1000 [sftpgo])\n  Group: 0 [root] (current: 1000 [sftpgo])\n  ELF architecture: EM_X86_64 (current architecture: amd64)\n"}

What are you using SFTPGo for?

Private user, home usecase (home backup/VPS), Medium business

Additional info

I'm running this local as a PoC and want to introduce it into the company. I've setup my own AD and CA for testing purposes locally. Edit-1: I noticed that there was a warning about the way i passed the args. I found out i had to change from: "--ldap-url ldap://192.168.15.168:389", to "--ldap-url=ldap://192.168.15.168:389", format.

drakkan commented 1 year ago

Hello,

this is not a bug, just a configuration issue, please check the plugin documentation again. I'm sorry but we no longer provide free support

Macleykun commented 1 year ago

Understandable! Is it possible to maybe provide an example for the plugin section in the full configuration example? Or that the issue can be left open in the hope someone else can tell me what mistake I’ve made in the config?

drakkan commented 1 year ago

Start a discussion then. But you should be able to fix your problem by simply reading the plugin doc

Macleykun commented 1 year ago 
{
  "data_provider": {
    "users_base_dir": "/srv/sftpgo/sftpgo/data"
  },
  "plugins": [
    {
      "cmd": "/usr/local/bin/sftpgo-plugin-auth",
      "type": "auth",
      "auth_options": {
        "scope": 1
      },
      "args": [
        "serve",
        "--ldap-url=ldap://192.168.15.168:389",
        "--ldap-base-dn=DC=mcdonalds,DC=local",
        "--ldap-bind-dn=CN=Wesley de Vree,CN=Users,DC=mcdonalds,DC=local",
        "--ldap-password=TestWindowsAD!2023",
        "--ldap-search-query=(&(objectClass=user)(sAMAccountType=805306368)(sAMAccountName=%username%))",
        "--skip-tls-verify=1"
      ],
      "auto_mtls": true
    }
  ]
}

Managed to find out how it worked, leaving this here (and a few other places) so others can copy paste the example and edit it to their liking.

Looking forward to play futher with sftpgo!