patrickjahns commented 5 years ago

Changelog

All notable changes to this project will be documented in this file.

Items based on https://github.com/owncloud/core/blob/stable10/CHANGELOG.md

[Unreleased]

Added

[x] - occ decrypt-all command can now read password from an environment variable - #32252 #Pending issue, under investigation @davitol
[x] - Command for resetting password now supports sending reset email and outputting link - #32500 @davitol
N/A- Roave Security Advisories as a development dependency - #31818 @davitol
[x] - Store timestamp when ownCloud was first installed - #32000 @davitol
[x] :gear: - Symfony events for login action with token or Apache - #31985 @davitol (Contains unit tests)
[x] :gear: - Search API for files using Webdav REPORT and underlying search provider - #31946 #32328 @davitol (Unit Tests)
[x] Store user name in oc_preferences when provided by backend, use in external storage save in session mode #32579 - @cdamken
[x] Support JSON format for settings passed to occ system:config:set - #32524 @davitol
[x] :gear: - Add information whether user can share to capabilities API - #31824 @davitol
[x] - Reload the filelist view when accepting or rejecting a share - #31798 @davitol
[x] - Allow different language in public link share email - #31767 @davitol
[x] - Hook "loadAdditionalScripts" now also available for public link page - #31944 @davitol (Unit tests)
[x] - Command files:scan now outputs items per second - #32093 @davitol
[x] - New option to prevent users to share with specific system groups - #31740 #32533 #32501 @davitol
[x] - Add url parameter to files app which opens a specific sidebar tab - #32202 @davitol (Only works with files)
[x] - Retry chunks in web UI on stalled or timed out uploads - #32170 #32335 @davitol (Tested with IE uploading several files/sizes etc and it went fine. PR also contains debug code)
[x] - Add log entry for each migration that is run - #32461 @davitol
[x] - Ability to create users and send them an email for password creation - #32466 @davitol Fixed in https://github.com/owncloud/core/pull/32648 ###WIP the users are created, but when creating the users, the webUI looks like it got stuck instead of redirecting to login page
N/A - Added Phan static code analyzer to improve code quality - #32492
[x] ⚙️ - Added method in PHP share API to set password hashes directly - #32572 It contains Unit tests. It will be tested deeper when Data exporter with we ready. Checked that the hash changes in the DB @davitol
[x] - Experimental support for asynchronous MOVE operations - #32414, will be partly :robot: - @individual-it @pvince81

Changed

[x] - Handle SSL certificate verifications for others than Let's Encrypt - #31858 - @pvince81
[x] - Insufficient storage exception now logged with "debug" log level - #31978 - tested using quota - @pvince81
[x] - Skip filecache repair step for version greater than 10.0.4 - #31803 - @pvince81
[x] - Allow slashes in generated resource routes in app framework - #31939 @davitol
[x] - Email field is now default in user management page - #32466 @davitol
[x] - Bump swiftmailer/swiftmailer from 5.4.9 to 5.4.10 - #32200
[x] - Split of config.sample.php into two files for core and apps - #32554 new file is missing in RC1 build - @pvince81 => https://github.com/owncloud/core/pull/32634. :white_check_mark: now fixed in RC2

Karma PRs bumping versions (no need to be tested, just check the version)

[x] - Bump karma from 2.0.2 to 3.0.0 in /build - #31892 #32197 #32317
[x] - Bump karma from 2.0.4 to 2.0.5 in /build - #32197
[x] - Bump behat/behat from 3.4.3 to 3.5.0 - #32318
[x] - Bump sinon from 2.4.1 to 6.1.5 in /build - #32319

Symfony PRs bumping versions (no need to be tested, just check the version)

[x] - Bump paragonie/random_compat v2.0.15 to v2.0.17 - #32107
[x] - Bump symfony/event-dispatcher from 3.4.12 to 3.4.13 - #32199
[x] - Bump symfony/console from 3.4.12 to 3.4.13 - #32140
[x] - Bump symfony/routing from 3.4.12 to 3.4.13 - #32137
[x] - Bump symfony/process from 3.4.12 to 3.4.13 - #32135
[x] - Bump symfony/translation from 3.4.12 to 3.4.13 - #32198
[x] - Bump symfony polyfill 1.8.0 to 1.9.0 - #32255
[x] - Minor dependency bumps 2018-08-26 - #32439
[x] - Bump symfony 3.4.15 and zend-stdlib 3.2.1 - #32499

Removed

Fixed

Encryption fixes

[x] - Fix master key recreation - #32504 @PVince81
[x] - Use OC_DEFAULT_MODULE constant for encryption in core - #31838 => since encryption still works, this proves the constant worked @pvince81
[x] - Unset encrypted flag in file cache when running decrypt-all command - #32027 - checked oc_filecache before and after decryption, flag is reset to 0 correctly - @pvince81
[x] - Fix decrypt of single user in decrypt-all command - #32168 - tested with master key and user-key, works - @pvince81
[x] - Fix login exception in decrypt-all command - #31986 - @pvince81
[x] - Properly clean up encryption keys after file deletion - #31959 - tested user-key, deleting file or parent folder - @pvince81

Public link PRs

[x] - Lock public link share dialog while processing - #31928 - hit "Save" repeatedly, only one operation - @pvince81
[x] - Accept email addresses with subdomains with hyphens for public link emails - #32281 @davitol
[x] - Don't strip linebreaks in personal note of public link share - #32331 - @pvince81

Calendar PRs

[x] - Fix calendar or reminder insertion error via CalDAV on MacOS - #32024 - :robot: - @DeepDiver1975
[x] - Prevent share access to birthday calendar - #31882 - @pvince81 with steps from https://github.com/owncloud/core/pull/31833#issuecomment-420967736
[x] - Added space in display names of shared calendar/contact - #31877 - :robot: - @DeepDiver1975

Fixes for PUT method

[x] - Optimize file uploads with PUT method, with custom mtime, use storage instead of view - #31891 - :robot: covered by upload test in API tests - @pvince81
[x] - Optimize file uploads with PUT, don't fetch and update checksum again, reuse the one from part file - #31768 - :robot: covered by upload test in API tests - @pvince81

Fixes related to UIDs

[x] - Compare UIDs instead of objects when changing displayname - #32409 - @pvince81
[x] - Compare UIDs instead of objects when changing email address - #32391 - @pvince81

Fixes easy to test (estimation 0,5 md)

[x] - Improve text about logging in config.sample.php - #32049 - @pvince81
[x] - Fix not allowed to share message - #32429 - @pvince81
[x] - AppManager text typo and PHPdoc return tags - #31918
[x] - Do not throw an error when the same theme is enabled twice - #31783 - @pvince81
[x] - Fix repair step that removes duplicate sub shares - #31146 - tested with https://github.com/owncloud/core/issues/27990#issuecomment-357899190 - @pvince81
[x] - Adjust code to follow coding standard - #32116 - @pvince81
[x] - Fix overriding for gif images in themes for CLI scripts - #32131 - @pvince81
[x] - Fix wording on password change page - #32146 @davitol
[x] - Fixed mount config in frontend to only load once to avoid side effects - #32095 davitol - Tested Mounting and using Personal Mount and no issue appear and it worked fine
[x] - Don't urlencode group id to make it work with "/" and "%" - #31109 @davitol
[x] - Fix version previews to fall back to icon when no preview provider is available - #32474 @davitol
[x] - Fix typos in config.sample.php - #32496 - @pvince81

WIP other fixes

[x] - Return correct status when IMip email delivery fails - #32489 - :robot: - @DeepDiver1975
[x] - Don't check for avatar folder if not enabled - #32490 - @pvince81
[x] - Add missing ILogger declaration in MigrationService - #32473 #32475
[x] - Fix JS tests for future Sinon JS update - #32488 - the PR is only tests - @pvince81
[x] - Command to verify checksums is now more robust - #32360 - @pvince81
[x] - Update php doc to reflect proper return type - #32427 - nothing to test - @pvince81
[x] - Catch more errors in SMB storage - #32416
[x] - Don't crash on filescan where folder has symlink - #32408 - @pvince81
[x] - Fix issue with some special characters in queries - #32412 - @pvince81 - not testable, hardening, no regression
[x] - Use the core exception logger functionality in cron.php - #32404 @davitol
[x] - Improve performance when propagating size updates in file cache - #32304 - @pvince81
[x] - Prevent current chunk assembly failing by setting the exclusive file lock earlier - #32334 - @pvince81
[x] - Let files be overwritten by rename operations on local storage instead of pre-deleting - #32273 - @pvince81
[x] - Continue with upgrade even if the market app cannot be disabled - #32324 - confirmed by update-testing - @pvince81
[x] - Versions app now works also when comments app is disabled - #32208 - @pvince81
[x] - Fix two factor challenge page for when password has expired - #32058 - @pvince81
[x] - Scanner now properly resets checksum whenever a file has changed remotely - #32284 @davitol - Tested with test provided in https://github.com/owncloud/enterprise/issues/2561#issuecomment-392531205 and installing samba 4.8.2. provided via https://launchpad.net/~bnd-acc/+archive/ubuntu/samba-latest. Everything was synced fine.
[x] - Fix checksums not being updated on modifying shared file for objectstore - #32364 @individual-it
[x] - Properly set installed_version flag when enabling app via provisioning api - #32214 - @pvince81
[x] - Fix API response of pending shares when the state did not change - #32156 - @pvince81
[x] - Read mtime from both JS properties in web UI upload for browser compatibility - #32013 @davitol - Tested uploading in Chrome / Firefox and small file and a big file and checking mtime is preserved
[x] - Fix warning in logs while moving FutureFile after chunk assembly - #32166 - @pvince81
[x] - Allow null in "Origin" header for third party clients that send it with WebDAV - #32189 confirmed by community https://github.com/owncloud/core/issues/32090#issuecomment-419071636
[x] - Properly log failed message when token based authentication is enforced - #31948 - @pvince81 - raised unrelated https://github.com/owncloud/core/issues/32654
[ ] - Deleting a user now also properly deletes their external storages and storage assignations - #32069 :no_entry_sign: - @pvince81 raised https://github.com/owncloud/core/issues/32637
[x] - Remove sensitive shared_secret data from occ config:list output - #31997 @davitol
[x] - Fix file cache update function to properly handle empty string and nulls with Oracle - #31996 - :robot: - @pvince81 tested indirectly through the ScanTest of occ files:scan which triggers checksum resets (this is how the issue was found)
[x] - Fix bogus etag update when propagating etag for federated shares - #31992 - @pvince81
[x] 🚫 - Display all failed recipients when sending link share email - #31935 @davitol Fixed in RC2
[x] - Prevent logging LDAP password in case of failure - #32592 @davitol - Tested shutting down the already configured LDAP configuration and checking password is no longer shown in the stack trace
[x] - Prevent passwords to be set to empty strings - #32581 - @pvince81
[x] - Fix update issue related to oc_jobs when automatically enabling market app to assist for update in OC 10 - #32573 - @pvince81
[x] - Trigger missing migrations in files_sharing app, adds indices and can speed up some instances - #32562 - @pvince81
[x] - Fix issue with spam filters when sending public link emails - #32542 @davitol

Fixes related to notifications app

[x] Added translations - #233 - @pvince81
[x] Clear user notifications after user deletion - #223 - @pvince81
[x] Include the product name in the mail for the notifications - #211 - @pvince81
[x] Add "hello" in the mail templates - #209 - @pvince81
[x] Add common footer to notifications - #207 - @pvince81

RC2 items

changelog diff here: https://github.com/owncloud/core/pull/32679
[x] decrypt-all has default for "--continue" - #32677 - @pvince81
[x] - No more zsync in capabilities - #32603 - tested with curl on capabilities endpoint - @pvince81
[x] - Minimum desktop client version is 2.3.3 - #32657 @davitol - Tested with 2.3.0 and a warning is shown. 'Forbidden (Unsupported client version)'
[x] Fix PHP 7.2 issue with ini_set - #32538 :robot: - @pvince81
[x] OC redirect in password setting page - #32648 @davitol - Tested creating a user setting using a mail.
[x] Replace add-password with set_password - #32636 @davitol - ErrorException of Undefined index is no longer thrown
[x] Do not delete token if token mismatch happens - #32672 @davitol - Tested with the steps provided in the PR. Note: Step 'user sees that token is invalid', does not happen. It never shows invalid token page
[x] Concat all email addresses that had sending problems - #32633 @davitol - Tested reproducing dialog 'an error occured while sending email' and check that both mails are shown
[x] recheck failed items from RC1

RC3 items

[x] Config report now contains list of all migrations that have run, for easier debugging of update issues - configreport/#68 - @pvince81
[x] Update CA bundle - 2018-06-20 - #32688 @davitol checked in the code the changes
[x] Email field is now default in user management page, users receive an email with token to set initial password - #32466 [#32648] https://github.com/owncloud/core/pull/32690 @davitol
[x] recheck failed items from RC2

ownclouders commented 5 years ago

GitMate.io thinks possibly related issues are https://github.com/owncloud/core/issues/31579 (test), https://github.com/owncloud/core/issues/22492 (test), https://github.com/owncloud/core/issues/22485 (test), https://github.com/owncloud/core/issues/30152 (Testing 10.0.5RC3 Changelog), and https://github.com/owncloud/core/pull/32127 ([stable10] Update changelog for upcoming 10.0.10).

PVince81 commented 5 years ago

raised https://github.com/owncloud/encryption/issues/49 while testing encryption

PVince81 commented 5 years ago

I've tested "Fix master key recreation - #32504" and it worked fine.

While testing, raised https://github.com/owncloud/encryption/issues/50 regarding master key regeneration in maintenance mode.

PVince81 commented 5 years ago

Tested "Unset encrypted flag in file cache when running decrypt-all command - #32027"

Checked oc_filecache before and after decryption, flag is reset to 0 correctly.

While testing, I found https://github.com/owncloud/encryption/issues/51

PVince81 commented 5 years ago

Added changelog entries from notifications app.

PVince81 commented 5 years ago

Raised https://github.com/owncloud/core/issues/32619 related to the new "create by email" changes from https://github.com/owncloud/core/issues/32466.

It's about the toggle not working properly if you leave a value in the field.

PVince81 commented 5 years ago

I've tested all the notifications items

PVince81 commented 5 years ago

Don't strip linebreaks in personal note of public link share - #32331 => works

PVince81 commented 5 years ago

Improve performance when propagating size updates in file cache - #32304 => size and etag propagation still works as observed in filecache.

PVince81 commented 5 years ago

Let files be overwritten by rename operations on local storage instead of pre-deleting - #32273 - works
[x] tested by renaming a file to another existing one
[x] test overwrite a folder with a file
[x] test overwrite a file with a folder

PVince81 commented 5 years ago

Update php doc to reflect proper return type - #32427 - nothing to test
Fix typos in config.sample.php - #32496 - file from tarball looks the same

PVince81 commented 5 years ago

Improve text about logging in config.sample.php - #32049 => text is present in tarball

PVince81 commented 5 years ago

Fix not allowed to share message - #32429 - tested as recipient with a share that doesn't grant reshare permissions

PVince81 commented 5 years ago

Both already covered by upload API tests:

Optimize file uploads with PUT method, with custom mtime, use storage instead of view - #31891
- Optimize file uploads with PUT, don't fetch and update checksum again, reuse the one from part file - #31768

PVince81 commented 5 years ago

[x] TODO: add items from last changelog PR: https://github.com/owncloud/core/pull/32582 @davitol

PVince81 commented 5 years ago

Versions app now works also when comments app is disabled - #32208 => works

PVince81 commented 5 years ago

Properly set installed_version flag when enabling app via provisioning api - #32214 - confirmed that it did not work in 10.0.9 and works now

PVince81 commented 5 years ago

Split of config.sample.php into two files for core and apps :no_entry_sign:

I'll have a quick look, need to include the new file in the Makefile.

=> fix is here https://github.com/owncloud/core/pull/32634

phil-davis commented 5 years ago

Display all failed recipients when sending link share email - #31935 had regression in a later PR. Proposed fix in PR #32632

PVince81 commented 5 years ago

Deleting a user now also properly deletes their external storages and storage assignations - #32069 no_entry_sign - => raised #32637

PVince81 commented 5 years ago

Fix API response of pending shares when the state did not change - #32156 - works

PVince81 commented 5 years ago

Fix warning in logs while moving FutureFile after chunk assembly

Tested and confirmed that the warning log was there in v10.0.9 and gone in v10.0.10RC1 for a chunked upload initiated in the browser.

PVince81 commented 5 years ago

Do not throw an error when the same theme is enabled twice - #31783 => works

PVince81 commented 5 years ago

Fix repair step that removes duplicate sub shares - #31146 - tested with https://github.com/owncloud/core/issues/27990#issuecomment-357899190

PVince81 commented 5 years ago

Don't check for avatar folder if not enabled - #32490

I've disabled avatars and checked the files app, users page and also a public link page. All work fine. Tested this because the change is in the template layout code.

PVince81 commented 5 years ago

Properly log failed message when token based authentication is enforced - #31948

Works fine with "token_auth_enforced' => true" as expected in the ticket and PR.

However while testing I noticed that if you enable any two factor app without enforcing token auth in config.php, the two factor mode will automatically enforce tokens and the client will not be able to login. In this case, the "Login failed" message is missing in the log. I've raised this separately here https://github.com/owncloud/core/issues/32654

PVince81 commented 5 years ago

Fix two factor challenge page for when password has expired - #32058

Challenge page properly appears instead of white page.

cdamken commented 5 years ago

Store user name in oc_preferences when provided by backend, use in external storage save in session mode #32579

Checked -> works

PVince81 commented 5 years ago

Fix update issue related to oc_jobs when automatically enabling market app to assist for update in OC 10 - #32573

Confirmed working by update-testing and also rechecked manually.

PVince81 commented 5 years ago

Trigger missing migrations in files_sharing app, adds indices and can speed up some instances - #32562

Tested on my own server which was missing three migrations. They are now present after the upgrade:

| files_sharing         | 20170830112305 |
| files_sharing         | 20171115154900 |
| files_sharing         | 20171215103657 |

PVince81 commented 5 years ago

Command to verify checksums is now more robust - #32360

Tested by deleting a file from disk but leaving in file cache. Then ran occ files:checksums:verify: the command skipped the entry instead of crashing.

PVince81 commented 5 years ago

Fix bogus etag update when propagating etag for federated shares - #31992

Tested with steps from https://github.com/owncloud/core/pull/31988 and etags are stable for the root and for the one byte file as recipient in 10.0.10 RC1.

Note: I tried the same in 10.0.9 but could not reproduce the original issue. @butonic

PVince81 commented 5 years ago

Fix issue with some special characters in queries - #32412

Could not find a way to cause an issue. Seems not testable, hardening.

PVince81 commented 5 years ago

Fix overriding for gif images in themes for CLI scripts - #32131

Tested with steps from https://github.com/owncloud/core/issues/31804. Logo in email is correct.

PVince81 commented 5 years ago

Compare UIDs instead of objects when changing displayname - #32409

Setting display name in personal page works and users page works.

Compare UIDs instead of objects when changing email address - #32391

Setting email in personal page works after using link. Also works when set from users page without link.

PVince81 commented 5 years ago

I've added the RC2 items in the original post. Please note that most aren't listed directly in the changelog bu are part of existing entries as these bugs only exist since the addition of said entries, so they belong there. I've expanded that list into checkboxes for verification.

PVince81 commented 5 years ago

Skip filecache repair step for version greater than 10.0.4

Increased fourth digit in version.php and ran occ upgrade -vvvvvv. The filecache repair step did not appear in the list of executed steps.

Also tested that occ files:scan --repair still runs as it uses the repair step code path.

individual-it commented 5 years ago

"Fix checksums not being updated on modifying shared file for objectstore" - #32364
1. start server with OWNCLOUD_FOLDER=~/www/owncloud-10.0.10RC2 OWNCLOUD_LICENSE_KEY=xxxxxxx OWNCLOUD_MARKETPLACE_APIKEY=xxxxx OWNCLOUD_APPS_INSTALL=enterprise_key,objectstore docker-compose -f owncloud-base.yml -f owncloud-mount-core.yml -f owncloud-exported-ports.yml -f storage/scality.yml -f storage/scality-primary-objectstore.yml up
2. as user1 create a txt file and check checksum
3. as user1 share file to user2
4. as user2 verify checksum is still the same
5. as user2 overwrite file with same content, verify checksum didn't change
6. as user2 overwrite file with new content
7. as user1 and as user2 check the checksum, verify checksum changed and is the same for both users

PVince81 commented 5 years ago

Experimental support for asynchronous MOVE operations

did a quick test to see if the web UI uses the job status endpoint and reports back for upload and overwrite+upload.

the automated tests were already merged in https://github.com/owncloud/core/pull/32594

PVince81 commented 5 years ago

Prevent share access to birthday calendar

Tested with steps from https://github.com/owncloud/core/pull/31833#issuecomment-420967736. Works, no event is added to birthday calendar.

PVince81 commented 5 years ago

Handle SSL certificate verifications for others than Let's Encrypt

I don't have the setup to test this but as I can see the directives in the htaccess file are present in my test setup.

davitol commented 5 years ago

Catch more errors in SMB storage - #32416 To take into account https://github.com/owncloud/core/issues/32552#issuecomment-420973273 . I was not able to get StorageNotAvailable exception but worked for other people.(to be investigated) I guess I should get a 503 and the files should not be deleted

PVince81 commented 5 years ago

Config report now contains list of all migrations that have run, for easier debugging of update issues - configreport/#68

Tested on RC3, the migrations appear in the report.

davitol commented 5 years ago

I've added the RC3 items in the original post.

davitol commented 5 years ago

Email field is now default in user management page, users receive an email with token to set initial password - https://github.com/owncloud/core/issues/32684

Tested on RC3, invalid token page appears and no infinity loop occurs

PVince81 commented 5 years ago

Prevent current chunk assembly failing by setting the exclusive file lock earlier - #32334

Tested by adding sleep(300) in both locations specified in https://github.com/owncloud/core/pull/32226#discussion_r216366209, then uploading a big file. Then wait for 10+ minutes. In the end the file finishes the upload.

Observing the access log: the first MOVE has the client time out. Subsequent MOVE all fail with 423 Locked, which is expected as the first MOVE is still working. Once the first MOVE finishes, the file appears on the server.

owncloud / core

10.0.10 - Changelog Test Results #32566

Changelog

[Unreleased]

Added

Changed

Removed

Fixed