Closed darrepac closed 2 years ago
The latest one should be "2022-06-02.00-39-36.0.c8d7547", you can run /opt/lomorage/bin/lomoc import [username] [password] [import_dir] -n
, given that the import_dir is accessible on the host where lomod running.
lomoware@lomorage:~$ /opt/lomorage/bin/lomoc -h
NAME:
lomoc - A new cli application
USAGE:
lomoc [global options] command [command options] [arguments...]
VERSION:
2022-06-02.00-39-36.0.c8d7547
COMMANDS:
reset, r reset password, or home directory
discover, d Discover lomo backend
import, i Import all photos from given directory into lomo backend with given username and password
consistency, c consistent check and fix btw db and local file system
frame, f lomo-frame related commands
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--help, -h show help
--version, -v print the version
lomoware@lomorage:~$ /opt/lomorage/bin/lomoc import -h
NAME:
lomoc import - Import all photos from given directory into lomo backend with given username and password
USAGE:
lomoc import [command options] [username] [password] [directory]
OPTIONS:
--port value, -p value (default: 8000)
--no-move, -n not moving original photos/videos, and only insert record in db
Thanks for the reply. I am using image: lomorage/amd64-lomorage:latest, I pulled the latest image but I have still a lomoc version of May 1st 2022... Is there a beta image that is more recent?
Could you please try update again, just pushed new image
Ok thanks I have the right version now
Here is what I did... I have 40k photos and it finished in few seconds, so obviously something was going wrong:
root@b89920c0943c:/# opt/lomorage/bin/lomoc import -p 8001 -n user password /PHOTOS
Login localhost successfully
import photos/videos from /PHOTOS
please check import log /opt/lomorage/var/log/import_PHOTOS.log
Unfortunately, there is not such log file. /opt/lomorage/var doesn't contain log directory
root@b89920c0943c:/# cat /opt/lomorage/var/log/import_PHOTOS.log
cat: /opt/lomorage/var/log/import_PHOTOS.log: No such file or directory
@darrepac sorry to hear that, can you please find logs around that period in /opt/lomorage/var/log/lomod.log
and /opt/lomorage/var/log/lomod_access.log
, and post here? or you can send log from mobile app.
In fact, the import was running but it crashes after a while:
asset(user_id, hash, year, month, day, ext_id, device_id, latitude, longitude, size, status, create_time, upload_time)
values(1, 'f2453c38f80152f'/*+25 bytes*/, 2011, 6, 17, 1, 14, 0.0, 0.0, 947690, 1, '2011-06-17 20:4'/*+10 bytes*/, '2022-06-24 11:0'/*+20 bytes*/)
time="2022-06-24T11:01:14Z" level=info msg="created asset 20110617_8193.jpg: f2453c38f80152f6d391cdf39a6b21fcd68f5e22 @ 2011-6-17 from /PHOTOS/Photos_canon/2011/IMG_0689.JPG"
DB [1:14]: insert into asset_album (asset_id, album_id, orig_filename, create_time) values (8193,31,'img_0689.jpg', '2022-06-24 11:0'/*+20 bytes*/)
time="2022-06-24T11:01:15Z" level=warning msg="User 's /media/primary/pascal/Photos/master/2018/01/22/20180122_7005.mov is not found in file system, remove from DB"
DB [1:15]: delete from metadata_similarity where asset_id = 7005
DB [1:15]: delete from metadata_tag where asset_id = 7005
DB [1:15]: delete from metadata_encrypt where asset_id = 7005
DB [1:15]: delete from metadata_geo where asset_id = 7005
DB [1:15]: delete from metadata_scene where asset_id = 7005
DB [1:15]: delete from metadata_face where asset_id = 7005
DB [1:16]: delete from metadata_text where asset_id = 7005
DB [1:18]: delete from metadata_human where asset_id = 7005
DB [1:18]: delete from asset_album_geo where asset_id=7005
DB [1:18]: delete from asset_album_label where asset_id=7005
DB [1:18]: delete from asset_album_camera_make where asset_id=7005
DB [1:18]: delete from asset where id = 7005
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0xb30a65]
goroutine 7182 [running]:
bitbucket.org/lomoware/lomo-backend/handler.(*Handler).deleteAsset1.func1({0x17d4180, 0xc003da95c0}, 0xc0000aa000)
/media/KingFast_e672fee1-90d2-4b78-966c-7cda33684599/Jenkins/workspace/build-lomod-docker-amd64/handler/asset.go:632 +0x225
bitbucket.org/lomoware/lomo-backend/common/dbx.runQuery({0x17d4180, 0xc003da95c0}, 0x203000, 0xc00084f7c8)
/media/KingFast_e672fee1-90d2-4b78-966c-7cda33684599/Jenkins/workspace/build-lomod-docker-amd64/common/dbx/db.go:209 +0xd7
bitbucket.org/lomoware/lomo-backend/common/dbx.InQuery(0x0, 0xc00084f790)
/media/KingFast_e672fee1-90d2-4b78-966c-7cda33684599/Jenkins/workspace/build-lomod-docker-amd64/common/dbx/db.go:188 +0x9c
bitbucket.org/lomoware/lomo-backend/handler.(*Handler).deleteAsset1(0xc000348000, 0x1, {0xc000331240, 0x8}, 0x0, 0x2, 0x0)
/media/KingFast_e672fee1-90d2-4b78-966c-7cda33684599/Jenkins/workspace/build-lomod-docker-amd64/handler/asset.go:620 +0xca
bitbucket.org/lomoware/lomo-backend/handler.(*Handler).fixMissMasterPreviewAssets(0xc000348000, {0xc000874000, 0x4, 0x1})
/media/KingFast_e672fee1-90d2-4b78-966c-7cda33684599/Jenkins/workspace/build-lomod-docker-amd64/handler/system.go:704 +0xb8b
bitbucket.org/lomoware/lomo-backend/handler.(*Handler).maintenanceScan(0xc000348000, 0x0, 0x0)
/media/KingFast_e672fee1-90d2-4b78-966c-7cda33684599/Jenkins/workspace/build-lomod-docker-amd64/handler/system.go:655 +0x751
bitbucket.org/lomoware/lomo-backend/handler.(*Handler).maintenance(0xc000348000, 0xc0001ea500, 0xa, 0x0, 0xc0006ae660, 0xc0006ae6c0, 0x0)
/media/KingFast_e672fee1-90d2-4b78-966c-7cda33684599/Jenkins/workspace/build-lomod-docker-amd64/handler/system.go:569 +0xb1
created by bitbucket.org/lomoware/lomo-backend/handler.NewHandler
/media/KingFast_e672fee1-90d2-4b78-966c-7cda33684599/Jenkins/workspace/build-lomod-docker-amd64/handler/handler.go:369 +0xd9d
sorry, it is one bug. Will fix. Thanks for reporting!
sorry, it is one bug. Will fix. Thanks for reporting!
Tell me when you have an updated container so I can give a new try
@darrepac , could you try update docker image? already upload the fixed version.
Thanks, it is runing now. Is there a way to see the progress. I have 40k photos to scan and would like to see a least a counter 434 / 40 000 or something on the logs (or UI) to know where I am.
It crashed again, here is the end of the logs:
2022-06-27T07:15:12.180307618Z asset(user_id, hash, year, month, day, ext_id, device_id, latitude, longitude, size, status, create_time, upload_time)
2022-06-27T07:15:12.180324127Z values(1, 'c5504a13c38eb2d'/*+25 bytes*/, 2005, 7, 8, 1, 14, 0.0, 0.0, 2836560, 1, '2005-07-08 14:1'/*+10 bytes*/, '2022-06-27 07:1'/*+20 bytes*/)
2022-06-27T07:15:12.220228156Z time="2022-06-27T07:15:12Z" level=info msg="created asset 20050708_9032.jpg: c5504a13c38eb2d2080d10eac0a3d77c7f5d27ac @ 2005-7-8 from /PHOTOS/Photos Pyrenees/Best Off/DSC_5729.JPG"
2022-06-27T07:15:12.402887850Z DB [15:12]: insert into asset_album (asset_id, album_id, orig_filename, create_time) values (9032,18,'dsc_5729.jpg', '2022-06-27 07:1'/*+20 bytes*/)
2022-06-27T07:15:14.202971849Z time="2022-06-27T07:15:14Z" level=info msg="start create asset: /PHOTOS/Photos Pyrenees/DVD/Etape6/DSC_5087.JPG (jpg) for user: 1 from 14"
2022-06-27T07:15:14.203680371Z DB [15:14]: insert into
2022-06-27T07:15:14.203713635Z asset(user_id, hash, year, month, day, ext_id, device_id, latitude, longitude, size, status, create_time, upload_time)
2022-06-27T07:15:14.203728074Z values(1, '1e27033626717f7'/*+25 bytes*/, 2005, 7, 2, 1, 14, 0.0, 0.0, 2815933, 1, '2005-07-02 07:0'/*+10 bytes*/, '2022-06-27 07:1'/*+20 bytes*/)
2022-06-27T07:15:14.204341226Z time="2022-06-27T07:15:14Z" level=info msg="created asset 20050702_9033.jpg: 1e27033626717f788c2b63dc8cbe67a7836cf1d3 @ 2005-7-2 from /PHOTOS/Photos Pyrenees/DVD/Etape6/DSC_5087.JPG"
2022-06-27T07:15:14.417016599Z DB [15:14]: insert into asset_album (asset_id, album_id, orig_filename, create_time) values (9033,46,'dsc_5087.jpg', '2022-06-27 07:1'/*+20 bytes*/)
2022-06-27T07:15:14.678267783Z time="2022-06-27T07:15:14Z" level=info msg="finish scan from 2022-06-27T06:15:14Z to 0001-01-01T00:00:00Z"
2022-06-27T07:15:14.679479355Z time="2022-06-27T07:15:14Z" level=info msg="FINISH: totally scanned 23 directories, and imported 840 media files"
2022-06-27T07:15:15.170915500Z panic: send on closed channel
2022-06-27T07:15:15.171144732Z
2022-06-27T07:15:15.172295963Z goroutine 34467 [running]:
2022-06-27T07:15:15.225105601Z bitbucket.org/lomoware/lomo-backend/common/scan.(*Runner).reportStats(0xc000494000, 0x34630b8a000, 0xc000fdf6e0, 0x1873df0, 0xc00103c5c0)
2022-06-27T07:15:15.233939546Z /media/KingFast_e672fee1-90d2-4b78-966c-7cda33684599/Jenkins/workspace/build-lomod-docker-amd64/common/scan/scan.go:180 +0x58e
2022-06-27T07:15:15.233988555Z created by bitbucket.org/lomoware/lomo-backend/common/scan.(*Runner).Start
2022-06-27T07:15:15.234003641Z /media/KingFast_e672fee1-90d2-4b78-966c-7cda33684599/Jenkins/workspace/build-lomod-docker-amd64/common/scan/scan.go:121 +0x325
@darrepac sorry to hear that. It is caused by one race condition btw asynchronous channel communication. I'll provide a fix this week. Thanks for all feedback and support!
@darrepac, we pushed the fix , could you try again with the latest version?
ok So far the log is full of such message, I don't know why (tried to restart but was the same). I did not even try to import anything...I will wait and see....
2022-07-04T06:57:25.681990598Z time="2022-07-04T06:57:25Z" level=warning msg="[/media/primary/pascal/Photos/master/2005/07/08/20050708_8577.jpg /media/primary/pascal/Photos/preview/2005/07/08/20050708_8577] - /media/primary/pascal/Photos/preview/2005/07/08/20050708_8577_640_0.webp exif analysis got: exiftool analysis /media/primary/pascal/Photos/preview/2005/07/08/20050708_8577_640_0.webp got Error: File not found - /media/primary/pascal/Photos/preview/2005/07/08/20050708_8577_640_0.webp\n 1 files could not be read\n 0 output files created\n: exit status 1"
Seems import failed:
/# /opt/lomorage/bin/lomoc import -p 8001 -n user password /PHOTOS/
2022/07/04 09:56:38 {"id": "0", "text": "sql: no rows in result set"}
@darrepac thanks for trying. If possible, could you please send log at mobile app's own setting page? We can analysis and find the root cause.
Don't know why but now it imports photos ok...let see if it crashes after a while
thanks for update. Please let us know if you see any problems.
Hi
Seems the doc is outdated. Contrary to what is written here , when I press lomoc -h, here is what I have:
So I don't find the "no-move" option... and as I don't want to move the files that I import, I don't know what to do. thanks for your help