File deleted on host still exists in app container

[konrad-o]

First of all thanks for this great piece of software. It really gives performance boost to rails app. We are trying to dockerise our codebase & everything seemed to work fine, but then we've realised that files deleted on host are still present in the app container.

Error

File deletion on host is not propagated into container. File creation on host and in app container is synchronised, same is for deletion of the file from within app container. When inside docker-sync container file deleted on host disappears in /host_sync, but is still visible in/app_sync. Unison log looks like file deletion has not been fired/detected.

What we've tried so far:

• docker-sync stop + clean
• delete all volumes + containers
• add monit_enable: true in docker-sync.yml
• update docker
• enable/disable VirtioFS accelerated directory sharing & Use the new Virtualization framework

Logs/details below.

Docker Driver

Docker Desktop 4.6.1 & 4.9.0 docker-sync 0.7.2

Sync strategy

native_osx

your docker-sync.yml

version: "2"
options:
  verbose: true
syncs:
  sync_test:
    src: './app'
    sync_strategy: 'native_osx'

• docker-compose.yml

  
  version: "3"

services: ds_test: container_name: ds_test image: ubuntu:impish working_dir: /app volumes:

• sync_test:/app:nocopy command: tail -f /dev/null

volumes: sync_test: external: true

### OS
macOS Monterey 12.4 (21F79)

**Output from `docker-sync start`**

      ok  Starting native_osx for sync sync_test
      ok  sync_test container not running
      ok  creating sync_test container
      ok  Starting precopy
 precopy  docker run --rm -v "sync_test:/app_sync" -v "/Users/*****/projects/ds-test/app:/host_sync" -e HOST_VOLUME=/host_sync -e APP_VOLUME=/app_sync -e TZ=$(basename $(dirname `readlink /etc/localtime`))/$(basename `readlink /etc/localtime`) -e UNISON_SRC="/host_sync" -e UNISON_DEST="/app_sync" -e UNISON_ARGS="  -prefer /host_sync -numericids -auto -batch" -e UNISON_WATCH_ARGS="-repeat watch"  --name sync_test eugenmayer/unison:2.51.3-4.12.0-AMD64 /usr/local/bin/precopy_appsync

doing initial sync with unison Unison 2.51.3 (ocaml 4.12.0): Contacting server... Looking for changes Reconciling changes Propagating updates UNISON 2.51.3 (OCAML 4.12.0) started propagating changes at 16:28:19.28 on 03 Jun 2022 [BGN] Copying .DS_Store from /host_sync to /app_sync [END] Copying .DS_Store [BGN] Copying a.txt from /host_sync to /app_sync [END] Copying a.txt [BGN] Copying b.js from /host_sync to /app_sync [END] Copying b.js UNISON 2.51.3 (OCAML 4.12.0) finished propagating changes at 16:28:19.28 on 03 Jun 2022 Saving synchronizer state Synchronization complete at 16:28:19 (3 items transferred, 0 skipped, 0 failed) real 0m 0.14s user 0m 0.09s sys 0m 0.04s chown ing file to uid 0 initial sync done using unison ok Starting container command docker run -d -v "sync_test:/app_sync" -v "/Users/*****/projects/ds-test/app:/host_sync" -e HOST_VOLUME=/host_sync -e APP_VOLUME=/app_sync -e TZ=$(basename $(dirname readlink /etc/localtime))/$(basename readlink /etc/localtime) -e UNISON_SRC="/host_sync" -e UNISON_DEST="/app_sync" -e UNISON_ARGS=" -prefer /host_sync -numericids -auto -batch" -e UNISON_WATCH_ARGS="-repeat watch" --name sync_test eugenmayer/unison:2.51.3-4.12.0-AMD64 bec0872123731d732f9a89ac01234c7a0c707fdc36c941837da8a137e91cd50c ok starting initial sync of sync_test success Sync container started success Starting Docker-Sync in the background

**docker sync container logs**
- file being created on host ✅ 
- file being created in app container ✅ 
- file being deleted in app container ✅ 
- file being deleted on host ❌

user with uid 0 already exist and its root /entrypoint.sh: ignoring /docker-entrypoint.d/* 2022-06-03 16:28:20,722 CRIT Supervisor is running as root. Privileges were not dropped because no user is specified in the config file. If you intend to run as root, you can set user=root in the config file to avoid this message. 2022-06-03 16:28:20,722 INFO Included extra file "/etc/supervisor.conf.d/supervisor.daemon.conf" during parsing 2022-06-03 16:28:20,732 INFO RPC interface 'supervisor' initialized 2022-06-03 16:28:20,732 CRIT Server 'unix_http_server' running without any HTTP authentication checking 2022-06-03 16:28:20,735 INFO supervisord started with pid 1 2022-06-03 16:28:21,744 INFO spawned: 'unison' with pid 24 Unison 2.51.3 (ocaml 4.12.0): Contacting server... Looking for changes Warning: No archive files were found for these roots, whose canonical names are: /host_sync /app_sync This can happen either because this is the first time you have synchronized these roots, or because you have upgraded Unison to a new version with a different archive format.

Update detection may take a while on this run if the replicas are large.

Unison will assume that the 'last synchronized state' of both replicas was completely empty. This means that any files that are different will be reported as conflicts, and any files that exist only on one replica will be judged as new and propagated to the other replica. If the two replicas are identical, then no changes will be reported.

If you see this message repeatedly, it may be because one of your machines is getting its address from DHCP, which is causing its host name to change between synchronizations. See the documentation for the UNISONLOCALHOSTNAME environment variable for advice on how to correct this.

Donations to the Unison project are gratefully accepted: http://www.cis.upenn.edu/~bcpierce/unison

Reconciling changes Nothing to do: replicas have been changed only in identical ways since last sync. 2022-06-03 16:28:22,791 INFO success: unison entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) Looking for changes Reconciling changes file ----> a.aa
Propagating updates UNISON 2.51.3 (OCAML 4.12.0) started propagating changes at 16:30:47.96 on 03 Jun 2022 [BGN] Copying a.aa from /host_sync to /app_sync [END] Copying a.aa UNISON 2.51.3 (OCAML 4.12.0) finished propagating changes at 16:30:47.96 on 03 Jun 2022 Saving synchronizer state Synchronization complete at 16:30:47 (1 item transferred, 0 skipped, 0 failed) Looking for changes Reconciling changes Nothing to do: replicas have not changed since last sync. Looking for changes Reconciling changes <---- file test.x
Propagating updates UNISON 2.51.3 (OCAML 4.12.0) started propagating changes at 16:30:56.96 on 03 Jun 2022 [BGN] Copying test.x from /app_sync to /host_sync [END] Copying test.x UNISON 2.51.3 (OCAML 4.12.0) finished propagating changes at 16:30:56.97 on 03 Jun 2022 Saving synchronizer state Synchronization complete at 16:30:56 (1 item transferred, 0 skipped, 0 failed) Looking for changes Reconciling changes Nothing to do: replicas have not changed since last sync. Looking for changes Reconciling changes <---- deleted a.aa
Propagating updates UNISON 2.51.3 (OCAML 4.12.0) started propagating changes at 16:32:04.90 on 03 Jun 2022 [BGN] Deleting a.aa from /host_sync [END] Deleting a.aa UNISON 2.51.3 (OCAML 4.12.0) finished propagating changes at 16:32:04.91 on 03 Jun 2022 Saving synchronizer state Synchronization complete at 16:32:04 (1 item transferred, 0 skipped, 0 failed) Looking for changes Reconciling changes Nothing to do: replicas have not changed since last sync. Looking for changes Reconciling changes <---- deleted a.txt
Propagating updates UNISON 2.51.3 (OCAML 4.12.0) started propagating changes at 16:32:11.38 on 03 Jun 2022 [BGN] Deleting a.txt from /host_sync [END] Deleting a.txt UNISON 2.51.3 (OCAML 4.12.0) finished propagating changes at 16:32:11.38 on 03 Jun 2022 Saving synchronizer state Synchronization complete at 16:32:11 (1 item transferred, 0 skipped, 0 failed) Looking for changes Reconciling changes Nothing to do: replicas have not changed since last sync.

[konrad-o]

Small update - removing file from /host_sync folder while being inside docker-sync container seems to trigger deletion

Reconciling changes
deleted  ---->            aa.xx  
Propagating updates
UNISON 2.51.3 (OCAML 4.12.0) started propagating changes at 10:33:23.22 on 06 Jun 2022
[BGN] Deleting aa.xx from /app_sync
[END] Deleting aa.xx
UNISON 2.51.3 (OCAML 4.12.0) finished propagating changes at 10:33:23.22 on 06 Jun 2022
Saving synchronizer state
Synchronization complete at 10:33:23  (1 item transferred, 0 skipped, 0 failed)
Looking for changes
Reconciling changes
Nothing to do: replicas have not changed since last sync.

[makproductions]

OS: MAC OS Monterey v12.3.1 Docker Sync Version: 0.7.2

Possible Cause:

The way i experienced this issue was to use git and checkout a different branch where some files in had been deleted. The deleted files did not get reflected in the container. So files which did not exist on host were still in the container.

Workaround:

I simply deleted the folder in PHPSTORM and then undo my delete. This forced docker-sync to delete the entire directory from the container and sync it again. This might not be ideal for big projects as resync takes a while but if you know which folder is affected you can do targeted delete and undo.

Hope this helps someone out there until a fix is out.

[NikitaKharkov]

I can say that in older (0.6.* I guess) version of docker-sync there was no problem with deleting files. Now I faced it again and again. For example, I rename the file and obverses 2 file in app folder: old and new one. And, of course, one in host folder

I tried to restart docker-sync, sync command - nothing happen until I manually delete the file.

My version is 0.7.2, image: eugenmayer/unison:2.51.3-4.12.0-AMD64

[NikitaKharkov]

@EugenMayer Can you investigate this issue? Just check simple renaming and observe the issue. I don't think it adds some effect but I did it Symfony project.

[EugenMayer]

@NikitaKharkov i'am sorry, i cannot invest my time here. You will need to jump on this yourself.

[jasonandres]

@NikitaKharkov Did you figure out a resolution for this?

[Housik]

I have the same issue, latest Monterey, latest docker-sync, latest Docker for Mac... as it starts to happen after Docker for Mac update or Monterey update, I suppose, it is related to one of this software... Trying to figure out, what exactly is causing the problem... because it is really difficult to work now...

[Gu3]

Same issue here : resolved if I choose gRPC FUSE on docker desktop not VirtioFS

[Klizzy]

Same issue here : resolved if I choose gRPC FUSE on docker desktop not VirtioFS

Can confirm that the switch to gRPC fixes the file deletion detection issue. thanks for the tip @Gu3!

[TheNapse]

Same issue here : resolved if I choose gRPC FUSE on docker desktop not VirtioFS

Can confirm that the switch to gRPC fixes the file deletion detection issue. thanks for the tip @Gu3!

Can also confirm! Solved the issue for me, too. Thanks alot, pointing this out @Gu3

[demandre]

I still have the issue, on MAC OS 12.6.3 with docker-sync v1.0.5, with gRPC FUSE on docker desktop 4.17.0