S3/NFSv4 - [19 Beta 2] User cannot login with quota enabled

SimplyCorbett commented 4 years ago

EDIT: This also happens with NFSv4 mounted locally, mounted on the data directory. So it's not just S3. NFSv4 error is a few posts down.

With an indexless primary S3 bucket if a user is assigned quota they are unable to login to nextcloud. However, if they have unlimited quota they can login fine.

Why does this matter? Nextcloud should be keeping track of everything on the bucket internally. It should not be querying the bucket at all for user information.

Steps to reproduce

Enable indexless S3 bucket
Add quota to user in the user menu
Login as user. Login fails.

Expected behaviour

Tell us what should happen

User can have quota enabled and login fine

Actual behaviour

Tell us what happens instead

User cannot login with quota enabled on their account

Server configuration

Operating system: Debian 10 Web server: Nginx Database: Postgresql PHP version: 7.4-fpm Nextcloud version: (see Nextcloud admin page) 19 beta 2 Updated from an older Nextcloud/ownCloud or fresh install: No Where did you install Nextcloud from: Nextcloud website

<
?php
$CONFIG = array (
  'objectstore' =>
  array (
    'class' => 'OC\\Files\\ObjectStore\\S3',
    'arguments' =>
    array (
      'bucket' => 'drive',
      'autocreate' => true,
      'key' => '',
      'secret' => '',
      'hostname' => 'sasfasfase',
      'port' => 443,
      'use_ssl' => true,
      'region' => 'default',
      'use_path_style' => false,
    ),
  ),
  'trusted_domains' =>
  array (
    0 => 'driasfasfe',
  ),
  'trusted_proxies' =>
  array (
    0 => '192.168.5.1',
    1 => '2asfas',
    2 => '205asfasfas6',
    3 => '205.1asfasf',
  ),
  'overwrite.cli.url' => 'httasfas',
  'overwritehost' => 'dasfas',
  'overwriteprotocol' => 'https',
  'overwritewebroot' => '/',
  'filelocking.enabled' => false,
  'instanceid' => '',
  'passwordsalt' => '',
  'secret' => '',
  'datadirectory' => '/var/www/drive/data',
  'dbtype' => 'pgsql',
  'version' => '19.0.0.3',
  'dbname' => 'drive',
  'dbhost' => '10.0.1.3',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'drive',
  'dbpassword' => 'e',
  'installed' => true,
  'loglevel' => 1,
  'preview_max_x' => '2048',
  'preview_max_y' => '2048',
  'jpeg_quality' => '80',
  'mail_smtpmode' => 'smtp',
  'mail_smtpsecure' => 'tls',
  'mail_sendmailmode' => 'smtp',
  'mail_from_address' => 'noreply',
  'mail_domain' => 'iasfas',
  'mail_smtpauth' => 1,
  'mail_smtphost' => 'smtp.sendgrid.net',
  'mail_smtpport' => '587',
  'mail_smtpname' => 'apikey',
  'mail_smtppassword' => ',
  'mail_smtpauthtype' => 'LOGIN',
  'quota_include_external_storage' => false,
  'filesystem_check_changes' => 0,
  'trashbin_retention_obligation' => '7, 14',
  'enabledPreviewProviders' =>
  array (
    0 => 'OC\\Preview\\PNG',
    1 => 'OC\\Preview\\PDF',
    2 => 'OC\\Preview\\JPEG',
    3 => 'OC\\Preview\\GIF',
    4 => 'OC\\Preview\\HEIC',
    5 => 'OC\\Preview\\BMP',
    6 => 'OC\\Preview\\XBitmap',
    7 => 'OC\\Preview\\MP3',
    8 => 'OC\\Preview\\TXT',
    9 => 'OC\\Preview\\MarkDown',
    10 => 'OC\\Preview\\Movie',
    11 => 'OC\\Preview\\TIFF',
    12 => 'OC\\Preview\\Photoshop',
  ),
  'app_install_overwrite' =>
  array (
    0 => 'passman',
    1 => 'camerarawpreviews',
    2 => 'files_photospheres',
    3 => 'documentserver_community',
  ),
);

Other settings

No encryption Fresh install Minimal apps (passman, calendar, contacts, metadata, photo sphere viewer, community document server) No external storage No LDAP Firefox Browser

Error:

Configuration: 19 beta 2 Primary S3 storage | path style false Debian 10 PHP 7.4-FPM HAProxy + Nginx Postgresql

kesselb commented 4 years ago

Issue Template?

SimplyCorbett commented 4 years ago

Issue Template?

I can, but from what I can see this is going to occur with any indexless S3 bucket. Indexless meaning you can't lookup all of the files in the bucket. There is no sharding. Nextcloud is in charge of everything, including orphaned files. There is no equivalent to ls -l

In exchange for this you can store as many files as you want on the bucket without much performance impact.

Since nextcloud is handling the primary S3 directly anyway bucket sharding/indexing is not needed and a waste of resources. I have confirmed file creation/viewing/deletion works fine, so this is specifically a quota bug.

Well, I can't verify the file is actually deleted on the backend. But there don't appear to be any errors stating otherwise.

Steps to reproduce

Enable indexless S3 bucket
Add quota to user in the user menu
Login as user. Login fails.

Expected behaviour

Tell us what should happen

User can have quota enabled and login fine

Actual behaviour

Tell us what happens instead

User cannot login with quota enabled on their account

Server configuration

Operating system: Debian 10 Web server: Nginx Database: Postgresql PHP version: 7.4-fpm Nextcloud version: (see Nextcloud admin page) 19 beta 2 Updated from an older Nextcloud/ownCloud or fresh install: No Where did you install Nextcloud from: Nextcloud website

Other settings

No encryption Fresh install Minimal apps (passman, calendar, contacts, metadata, photo sphere viewer, community document server) No external storage No LDAP Firefox Browser

kesselb commented 4 years ago

The template is not there because someone had fun creating it. That might be a regression of https://github.com/nextcloud/server/pull/19782 if 'quota_include_external_storage' => true,. That's the reason we usually ask for the configuration. It's not a big challenge to put together the requested information but makes triaging issues much easier.

SimplyCorbett commented 4 years ago

The template is not there because someone had fun creating it. That might be a regression of #19782 if 'quota_include_external_storage' => true,. That's the reason we usually ask for the configuration. It's not a big challenge to put together the requested information but makes triaging issues much easier.

I understand but I'm not sure what other information you need for this bug. I can provide more information on my S3 setup if needed. I listed the bug in the OP.

Sorry, not trying to be difficult here. Let me know what you need and I will provide it.

kesselb commented 4 years ago

Let me know what you need and I will provide it.

Please edit the first post and add: https://github.com/nextcloud/server/blob/master/.github/ISSUE_TEMPLATE/Bug_report.md

kesselb commented 4 years ago

For some reason you decided to not provide the configuration (that was actually the important thing). Is quota_include_external_storage true or false? If true is the same happening with false?

Add that information to the first post.

SimplyCorbett commented 4 years ago

It's not enabled. It's a fresh install without external storage.

SimplyCorbett commented 4 years ago

@kesselb It turns out this is not a S3 error. It happens with NFS v4 mounted locally (so nextcloud shouldnt be able to tell it's NFS) as well.

SimplyCorbett commented 4 years ago

NFSv4 Error:

SimplyCorbett commented 4 years ago

This is not fixed in the master branch.

SimplyCorbett commented 4 years ago

S3 is fixed in beta4.

nextcloud / server