pulsejet / memories

Fast, modern and advanced photo management suite. Runs as a Nextcloud app.
https://memories.gallery
GNU Affero General Public License v3.0
3.13k stars 83 forks source link

Unhandled exception in migrate-google-takeout #1053

Open hindsholm opened 7 months ago

hindsholm commented 7 months ago

Describe the bug

I have made a Google takeout and placed the photos and .json files on a drive which is mounted as "external storage" at /MntPhotos in Nextcloud. All photos have been scanned and are visible in the Memories app. Now I want to use the metadata in the Google .json files, so I run:

sudo docker exec --user www-data -it nextcloud-aio-nextcloud php occ memories:migrate-google-takeout --override --user=morten --folder=/MntPhotos/Bratten

The command starts scanning but fails on the first file with an unhandled exception.

I have tried different combinations of the --user and --folder options, but I get the same error.

To Reproduce I don't know what to say here, but the bug is easily reproducable on my setup. The command fails consistently.

Screenshots Output:

sudo docker exec --user www-data -it nextcloud-aio-nextcloud php occ memories:migrate-google-takeout --override --user=morten --folder=/MntPhotos/Bratten
This command will migrate JSON metadata from Google Takeout to EXIF metadata.
Only metadata that is missing from EXIF will be migrated, unless --override is specified.
It will also update the JSON files to mark them as migrated.
Make sure you have a backup of your originals before running this command.
Also make sure exiftool is working beforehand by running memories:index on some files.
Are you sure you want to continue? (y/N): y
Migrating user morten
Scanning folder /morten/files/MntPhotos/Bratten
Writing EXIF metadata for /morten/files/MntPhotos/Bratten/IMG_6663.JPG (AllDates)
An unhandled exception has been thrown:
TypeError: OC\Files\View::basicOperation(): Argument #2 ($path) must be of type string, null given, called in /var/www/html/lib/private/Files/View.php on line 528 and defined in /var/www/html/lib/private/Files/View.php:1128
Stack trace:
#0 /var/www/html/lib/private/Files/View.php(528): OC\Files\View->basicOperation('file_exists', NULL)
#1 /var/www/html/lib/private/Files/Filesystem.php(545): OC\Files\View->file_exists(NULL)
#2 /var/www/html/apps/files_versions/lib/Storage.php(189): OC\Files\Filesystem::file_exists(NULL)
#3 /var/www/html/apps/files_versions/lib/Listener/FileEventsListener.php(196): OCA\Files_Versions\Storage::store(NULL)
#4 /var/www/html/apps/files_versions/lib/Listener/FileEventsListener.php(102): OCA\Files_Versions\Listener\FileEventsListener->write_hook(Object(OC\Files\Node\File))
#5 /var/www/html/lib/private/EventDispatcher/ServiceEventListener.php(86): OCA\Files_Versions\Listener\FileEventsListener->handle(Object(OCP\Files\Events\Node\BeforeNodeWrittenEvent))
#6 /var/www/html/3rdparty/symfony/event-dispatcher/EventDispatcher.php(251): OC\EventDispatcher\ServiceEventListener->__invoke(Object(OCP\Files\Events\Node\BeforeNodeWrittenEvent), 'OCP\\Files\\Event...', Object(Symfony\Component\EventDispatcher\EventDispatcher))
#7 /var/www/html/3rdparty/symfony/event-dispatcher/EventDispatcher.php(73): Symfony\Component\EventDispatcher\EventDispatcher->callListeners(Array, 'OCP\\Files\\Event...', Object(OCP\Files\Events\Node\BeforeNodeWrittenEvent))
#8 /var/www/html/lib/private/EventDispatcher/EventDispatcher.php(94): Symfony\Component\EventDispatcher\EventDispatcher->dispatch(Object(OCP\Files\Events\Node\BeforeNodeWrittenEvent), 'OCP\\Files\\Event...')
#9 /var/www/html/lib/private/EventDispatcher/EventDispatcher.php(106): OC\EventDispatcher\EventDispatcher->dispatch('OCP\\Files\\Event...', Object(OCP\Files\Events\Node\BeforeNodeWrittenEvent))
#10 /var/www/html/lib/private/Files/Node/HookConnector.php(112): OC\EventDispatcher\EventDispatcher->dispatchTyped(Object(OCP\Files\Events\Node\BeforeNodeWrittenEvent))
#11 /var/www/html/lib/private/legacy/OC_Hook.php(105): OC\Files\Node\HookConnector->write(Array)
#12 /var/www/html/lib/private/Files/View.php(1275): OC_Hook::emit('OC_Filesystem', 'write', Array)
#13 /var/www/html/lib/private/Files/View.php(1144): OC\Files\View->runHooks(Array, '/MntPhotos/Brat...')
#14 /var/www/html/lib/private/Files/View.php(680): OC\Files\View->basicOperation('file_put_conten...', '/morten/files/M...', Array, '{\n    "title": ...')
#15 /var/www/html/lib/private/Files/Node/File.php(73): OC\Files\View->file_put_contents('/morten/files/M...', '{\n    "title": ...')
#16 /var/www/html/custom_apps/memories/lib/Command/MigrateGoogleTakeout.php(279): OC\Files\Node\File->putContent('{\n    "title": ...')
#17 /var/www/html/custom_apps/memories/lib/Command/MigrateGoogleTakeout.php(170): OCA\Memories\Command\MigrateGoogleTakeout->migrateFile(Object(OC\Files\Node\File))
#18 /var/www/html/custom_apps/memories/lib/Command/MigrateGoogleTakeout.php(150): OCA\Memories\Command\MigrateGoogleTakeout->migrateFolder(Object(OC\Files\Node\Folder))
#19 /var/www/html/custom_apps/memories/lib/Command/MigrateGoogleTakeout.php(112): OCA\Memories\Command\MigrateGoogleTakeout->migrateUser(Object(OC\User\User))
#20 /var/www/html/3rdparty/symfony/console/Command/Command.php(298): OCA\Memories\Command\MigrateGoogleTakeout->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#21 /var/www/html/3rdparty/symfony/console/Application.php(1040): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#22 /var/www/html/3rdparty/symfony/console/Application.php(301): Symfony\Component\Console\Application->doRunCommand(Object(OCA\Memories\Command\MigrateGoogleTakeout), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#23 /var/www/html/3rdparty/symfony/console/Application.php(171): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#24 /var/www/html/lib/private/Console/Application.php(211): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#25 /var/www/html/console.php(100): OC\Console\Application->run()
#26 /var/www/html/occ(11): require_once('/var/www/html/c...')
#27 {main}

Contents of Bratten/IMG_6663.JPG.json:

{
  "title": "IMG_6663.JPG",
  "description": "",
  "imageViews": "70",
  "creationTime": {
    "timestamp": "1281650148",
    "formatted": "Aug 12, 2010, 9:55:48 PM UTC"
  },
  "photoTakenTime": {
    "timestamp": "1280572001",
    "formatted": "Jul 31, 2010, 10:26:41 AM UTC"
  },
  "geoData": {
    "latitude": 0.0,
    "longitude": 0.0,
    "altitude": 0.0,
    "latitudeSpan": 0.0,
    "longitudeSpan": 0.0
  },
  "geoDataExif": {
    "latitude": 0.0,
    "longitude": 0.0,
    "altitude": 0.0,
    "latitudeSpan": 0.0,
    "longitudeSpan": 0.0
  },
  "people": [{
    "name": "George",
    "email": "george@xxxxx.dk",
    "x": 0.28125,
    "y": 0.2221181,
    "width": 0.09375,
    "height": 0.16869728
  }, {
    "name": "George"
  }, {
    "name": "Lise"
  }],
  "url": "https://photos.google.com/photo/AF1QipN_C-zeVtzWzYtSgb6q5gU1-0_hASvrKmRYfqaR",
  "googlePhotosOrigin": {
    "picasa": {
    }
  }
}

Platform:

Additional context Please let me know if I should run some further tests.

pulsejet commented 7 months ago

This is a bug in Nextcloud, not Memories

pulsejet commented 7 months ago

If you want a temporary workaround, you can chagne line 280 of MigrateGoogleTakeout.php to catch \Throwable instead of \Exception. It will still show some errors but continue to the next file.

hindsholm commented 7 months ago

This is a bug in Nextcloud, not Memories

Do you have a reference to the bug in Nextcloud?

pulsejet commented 7 months ago

No I've no idea. But looking at the trace it seems to be in a hook in the files versions app.

pulsejet commented 7 months ago

https://github.com/nextcloud/server/pull/44066