search

deajan / osync

A robust two way (bidirectional) file sync script based on rsync with fault tolerance, POSIX ACL support, time control and near realtime sync
http://www.netpower.fr/osync
BSD 3-Clause "New" or "Revised" License
934 stars 101 forks source link

_getFileCtimeMtimeLocal using undefined variable $retval #224

Open johnwc opened 3 years ago

johnwc commented 3 years ago

The method _getFileCtimeMtimeLocal is using a var that is not defined, maybe from a copy/paste from another?

deajan commented 3 years ago

Which one please? The file n_osync.sh uses definitions from ofunctions.sh.

johnwc commented 3 years ago

We get the following error that we are trying to figure out the root cause. Those files that it is complaining about actually do not exists anymore. We expected it to pick that up and mark them as deleted.

TIME: 0 - This is an unstable dev build [2019052201]. Please use with caution.
TIME: 0 - Tue Feb 23 19:31:47 UTC 2021 - osync 1.3.0-beta3 script begin.
TIME: 1 - Could not get file attributes for [/var/www/vhosts/site-local/wwwroot/sync_maintenance.html].
TIME: 5 - Could not get file attributes for [/var/www/vhosts/site-local/wwwroot/wp-admin/includes/.class-ftp.php.Tgz287].
TIME: 8 - Getting file time attributes failed [] on target. Stopping execution.
TIME: 8 - _ExecTasksPidsCheck called by [Sync_timestampListBefore] finished monitoring pid [30305] with exitcode [1].
TIME: 160 - Could not get file attributes for [/home/site/wwwroot/wp-content/uploads/bb-theme/skin-60344a51b079a.css].
TIME: 174 - Getting file time attributes failed [] on initiator. Stopping execution.
TIME: 174 - _ExecTasksPidsCheck called by [Sync_timestampListBefore] finished monitoring pid [30304] with exitcode [1].
TIME: 174 - osync finished with errors.
deajan commented 3 years ago

Could you come up with a minimum example (SSCCE) ? Ie a script creating test files, running osync, deleting the files, run osync again and get the above errors?

Without any further info, it's hard to know what's going on on your system.

johnwc commented 3 years ago

I can get you access to a current running docker container that is giving that error. I don't know what is causing it, hence why my OP was pointing out that the error message wasn't giving much info to go on. Otherwise I would be able to produce reproducible steps to share.

deajan commented 3 years ago

I'd rather have reproductible steps (including file creation) so I can investigate without the risk of breaking something.

deajan commented 3 years ago

Thinking of the problem, I might actually think of a possible issue with ACL transfer. I guess you're using osync because it handles ACLs ?

johnwc commented 3 years ago

What would cause the error "Could not get file attributes"

We are using osync because we need a way to bidirectional sync two folders on a container.

In azure, if you use a Linux hosted App Service, it hosts your site in a container that is mapped to a storage blob. Well, that storage blob has very slow IO. So hosting a large site like WordPress causes the site to run very poorly when running from the storage blob. We got around that by modifying the container so that the wwwroot is running off container storage, and it is extremely faster. When you FTP files to the App Service, you're actually FTPing to the storage blob, not the container. So once you upload your site, we must "sync" the wwwroot directory on the mounted storage to the local container's wwwroot that the site is running from. And also the container can be rebuilt at anytime by Azure to migrate it around hosts. This means upon start we much run a sync so we move the blob storage files to the local storage so the site will have it's files to run. You can view the (project here)[https://github.com/johnwc/azure-app-service-openlitespeed-php-container].

The osync is to keep the two directories in sync, if someone updates WP via WP-Admin or if someone FTP's files up.

deajan commented 3 years ago

Again, do you need to transfer ACLs ?

johnwc commented 3 years ago

No, ACLs are not required

deajan commented 3 years ago

Okay, I have a rough idea of what's going on. Could you disable PRESERVE_ACL and PRESERVE_XATTR please ? This should resvole your issue. If so, please report back so I can fix the ACL issue.