Database corruption?

[LebedevRI]

raw-camera-samples$ rsync -rLtW --preallocate --delete --compress --compress-level=1 --stats --progress rsync://raw.pixls.us/data/ ~/raw-camera-samples/raw.pixls.us
receiving incremental file list
symlink has no referent: "Fujifilm/X100VI/DSCF0025.RAF" (in data)
IO error encountered -- skipping file deletion
symlink has no referent: "Panasonic/DMC-FZ28/P1030025.RW2" (in data)
symlink has no referent: "Panasonic/DMC-FZ28/P1320048.RW2" (in data)
symlink has no referent: "Panasonic/DMC-FZ28/P1030021.RW2" (in data)
symlink has no referent: "Sigma/SIGMA fp/SDIM0094.DNG" (in data)

Number of files: 2,862 (reg: 1,817, dir: 1,045)
Number of created files: 0
Number of deleted files: 0
Number of regular files transferred: 0
Total file size: 52,303,235,513 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 81,784
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 1,179
Total bytes received: 87,966

sent 1,179 bytes  received 87,966 bytes  35,658.00 bytes/sec
total size is 52,303,235,513  speedup is 586,720.91
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1865) [generator=3.2.7]

https://raw.pixls.us/data/filelist.sha1 contains, e.g.,

a2aabbfff4a275e73d0697f7bc5c102fecbea855 *Fujifilm/X100VI/DSCF0025.RAF

but https://raw.pixls.us/data/Fujifilm/X100VI/ is empty, and searching for DSCF0025.RAF in either admin iface or user iface says No matching records found.

But for Sigma/SIGMA fp, there's:

fe9d52cba16566a4289527971d99ce5a39c6c48c *Sigma/SIGMA fp/A002_647_20240229_000001_4k_12bit.DNG
f4dad05053a012b403c4114e89cf5668a8fee349 *Sigma/SIGMA fp/A002_648_20240229_000001_4k_10bit.DNG
b4c9846aefeb243ca9f670f2bfbecbc5d01622c4 *Sigma/SIGMA fp/A002_650_20240229_000001_4k_8bit.DNG
847fa43e585e8d0bae1670d918274aadd52e560a *Sigma/SIGMA fp/A002_651_20240229_000001_2k_12bit.DNG
ee0fb4ff6e9f678aaa93e78d76167be540c3f21b *Sigma/SIGMA fp/A002_652_20240229_000001_2k_10bit.DNG
6d00c034c8a14f9c4fca3dc1a8e14300eb866ec7 *Sigma/SIGMA fp/A002_653_20240229_000001_2k_8bit.DNG
71a0ee5c3cec766cd2f2b215708356c486fae652 *Sigma/SIGMA fp/SDIM0094.DNG

and in /data/, SDIM0094.DNG no longer exists as it was replaced with the other 6 samples.

Likewise with Panasonic/DMC-FZ28, the newly-reviewed samples are visible in the admin, but the user-facing side is out-of-sync...

cc @andabata @patdavid

[LebedevRI]

@andabata @patdavid @kmilos And now the 9 Sony - ILCE-9M3 newly-reviewed (many hours ago) samples simply do not show up in /data/...

[kmilos]

And now the 9 Sony - ILCE-9M3 newly-reviewed (many hours ago) samples simply do not show up in /data/...

I did download most, but not all of them some 18h ago.

[LebedevRI]

The user-facing web view (https://raw.pixls.us/) has them, and may be up to date, but /data/ is out-of-sync. This may be a new cron issue, as a result of fixing #82.

[andabata]

Might it be that you looked to fast ? The cronjob for building the building the datadir only runs once an hour (41 mins past the hour)

[LebedevRI]

I mean, it's been 19 hours since that comment, and they are still not in /data/.

[andabata]

mm. strange.. if i look for the Sony ILCE-9M3 i see the following on the server root@namib:/srv/www/raw.pixls.us/www/data/Sony/ILCE-9M3# ls -la total 8 drwxr-xr-x 2 www-data www-data 4096 Mar 27 20:26 . drwxr-xr-x 101 www-data www-data 4096 Mar 27 20:26 .. lrwxrwxrwx 1 www-data www-data 50 Mar 27 20:26 DSC00110.ARW -> /storage/raw.pixls.us/data/3/6/0/7360/DSC00110.ARW lrwxrwxrwx 1 www-data www-data 50 Mar 27 20:26 DSC00111.ARW -> /storage/raw.pixls.us/data/3/6/1/7361/DSC00111.ARW lrwxrwxrwx 1 www-data www-data 50 Mar 27 20:26 DSC00112.ARW -> /storage/raw.pixls.us/data/3/6/2/7362/DSC00112.ARW lrwxrwxrwx 1 www-data www-data 50 Mar 27 20:26 DSC00113.ARW -> /storage/raw.pixls.us/data/3/6/3/7363/DSC00113.ARW lrwxrwxrwx 1 www-data www-data 50 Mar 27 20:26 DSC00114.ARW -> /storage/raw.pixls.us/data/3/6/4/7364/DSC00114.ARW lrwxrwxrwx 1 www-data www-data 50 Mar 27 20:26 DSC00115.ARW -> /storage/raw.pixls.us/data/3/6/6/7366/DSC00115.ARW lrwxrwxrwx 1 www-data www-data 50 Mar 27 20:26 DSC00116.ARW -> /storage/raw.pixls.us/data/3/6/7/7367/DSC00116.ARW lrwxrwxrwx 1 www-data www-data 50 Mar 27 20:26 DSC00117.ARW -> /storage/raw.pixls.us/data/3/6/8/7368/DSC00117.ARW lrwxrwxrwx 1 www-data www-data 50 Mar 27 20:26 DSC00118.ARW -> /storage/raw.pixls.us/data/3/6/9/7369/DSC00118.ARW

I ran the cronjob forced at 20:26. So it seems to be working. And if i browse to https://raw.pixls.us/data/Sony/ILCE-9M3/ i see them aswell. Question is where goes wrong for you, if you don't see them.

[LebedevRI]

I ran the cronjob forced at 20:26. So it seems to be working. And if i browse to https://raw.pixls.us/data/Sony/ILCE-9M3/ i see them aswell.

They weren't there before you did that, i'm 100% sure of that. I've just rerun rsync, and now it's fetching stuff. (it looks like it's trying to re-fetch all files, even though they are locally here. file timestamp changed?) So that cronjob wasn't running, or was failing.

And https://raw.pixls.us/data/filelist.sha1 just updated, too. It wasn't getting updated before.

@andabata thank you! But it really looks like something was/is going wrong on the server side...

[andabata]

Oh that's quite possible, i'm just trying to figure out if it breaks, when it breaks. I didn't doubt what you where seeing. Only ran the cronjob manually to see if it was still working. Hadden't checked before hand.

[LebedevRI]

@andabata once again, thank you! For the moment i'm going to consider the situation resolved, but if i notice that there are new discrepancies, i'll reopen...