syncthing / syncthing

Open Source Continuous File Synchronization
https://syncthing.net/
Mozilla Public License 2.0
65.26k stars 4.3k forks source link

.stignore * behaves differently on mac and linux #9628

Closed adriangalilea closed 3 months ago

adriangalilea commented 3 months ago

What happened?

I have 3 hosts:

My ~ is a git repo.

Same exact .gitignore on the raspberry pi and on the mac.

*
.stignore

Same exact .stignore on the raspberry pi and on the mac

!.env
*

Same exact config.xml on the raspberry pi and on the mac

<configuration version="37">
    <folder id="..." label="foo" path="~" type="sendreceive" fsWatcherEnabled="true" fsWatcherDelayS="10">
        <device id="..."/>
        <minDiskFree unit="%">1</minDiskFree>
        <versioning>
            <cleanupIntervalS>3600</cleanupIntervalS>
            <fsType>basic</fsType>
        </versioning>
        <ignoreDelete>false</ignoreDelete>
        <disableSparseFiles>false</disableSparseFiles>
        <disableTempIndexes>false</disableTempIndexes>
        <paused>false</paused>
        <weakHashThresholdPct>25</weakHashThresholdPct>
        <markerName>.stfolder</markerName>
        <copyOwnershipFromParent>false</copyOwnershipFromParent>
        <modTimeWindowS>0</modTimeWindowS>
        <maxConcurrentWrites>2</maxConcurrentWrites>
        <disableFsync>false</disableFsync>
        <blockPullOrder>standard</blockPullOrder>
        <copyRangeMethod>standard</copyRangeMethod>
        <caseSensitiveFS>false</caseSensitiveFS>
        <junctionsAsDirs>false</junctionsAsDirs>
        <syncOwnership>false</syncOwnership>
        <sendOwnership>false</sendOwnership>
        <syncXattrs>false</syncXattrs>
        <sendXattrs>false</sendXattrs>
        <xattrFilter>
            <maxSingleEntrySize>1024</maxSingleEntrySize>
            <maxTotalSize>4096</maxTotalSize>
        </xattrFilter>
    </folder>

    <device id="..." name="syno.local" compression="metadata" introducer="foo">
        <address>dynamic</address>
        <paused>false</paused>
        <autoAcceptFolders>false</autoAcceptFolders>
        <maxSendKbps>0</maxSendKbps>
        <maxRecvKbps>0</maxRecvKbps>
    </device>

    <gui enabled="false" tls="false">
        <address>127.0.0.1:8384</address>
    </gui>

    <options>
        <listenAddress>tcp://0.0.0.0:22000</listenAddress>
        <globalAnnounceEnabled>false</globalAnnounceEnabled>
        <localAnnounceEnabled>true</localAnnounceEnabled>
        <relaysEnabled>false</relaysEnabled>
        <natEnabled>false</natEnabled>
        <minHomeDiskFree unit="%">1</minHomeDiskFree>
        <urURL>https://data.syncthing.net/newdata</urURL>
        <keepTemporariesH>24</keepTemporariesH>
        <progressUpdateIntervalS>5</progressUpdateIntervalS>
        <crashReportingEnabled>true</crashReportingEnabled>
        <stunServer>default</stunServer>
        <databaseTuning>auto</databaseTuning>
        <announceLANAddresses>true</announceLANAddresses>
        <maxFolderConcurrency>0</maxFolderConcurrency>
    </options>
</configuration>

When I add Linux onto the synology, it'll work great, the .env file is synced.

When I add the mac, it just starts adding everything he has on ~

It doesn't matter how many things I tried. Raspberry pi .stignore works differently than on the mac.

I just can't use it.

Syncthing version

v1.27.9

Platform & operating system

Linux aarch64 / Mac Sonoma 14.5

Browser version

No response

Relevant log output

No response

tomasz1986 commented 3 months ago

Are you editing the .stignore files using the Web GUI or directly (e.g. with a text editor)? If it's the latter, please make sure that there are no funky invisible characters inserted into the file (e.g. BOM) that could possibly hinder with the ignore patterns, making them invalid in the process.

Normally, there should be no difference between how ignore patterns behave depending on the OS (except for some limitations on Windows).

adriangalilea commented 3 months ago

Are you editing the .stignore files using the Web GUI or directly (e.g. with a text editor)? If it's the latter, please make sure that there are no funky invisible characters inserted into the file (e.g. BOM) that could possibly hinder with the ignore patterns, making them invalid in the process.

Normally, there should be no difference between how ignore patterns behave depending on the OS (except for some limitations on Windows).

I'm using an editor, but a cli one(helix), I re-created the very same file several times, copy-pasting some times to avoid the possibility of typos, nothing.

It is particularly hard to diagnose, because when it decides to pick the whole ~ it takes solid several minutes to realize and 4 folders appear at once on my syncthing folder on the rest of the devices which is incredibly annoying, I have 0 indication that it's going to happen.

99.9% certainty.

I replicated it just once again, this time through the web UI, same result, completely ignores my .stignore, oh the irony :)

Could it be because I'm using ~?

Could it be because is a git repo?

Why would it behave differently on the 2 machines... I have no idea, but is literally the same setup, copy-pasted >x5 times through different means.

acolomb commented 3 months ago

This is a support request which should be handled on our forum. Please move the discussion there, until we are certain that an actual bug is identified.

Using a tilde for the folder path does not affect ignores in any way, they are always relative to the folder root. Therefore you shouldn't use it inside the ignore file of course.

For easier testing, un-share the folder for the other devices, or create a test folder at the same path, with different ID and no sharing. Also note that syncing the home folder is usually not encouraged. Although your special case of ignoring all but some specific files should work alright. As long as you don't mess up the ignore patterns.

adriangalilea commented 3 months ago

This is very odd,

Should this:

*
!.env

Be the same as

!.env
*

??

Because it ignores everything that comes after the * in my testing.

Also !.env and !/.env does not behave the same, even tho .env is literally on the root folder.

Perhaps it's because I've been hitting my head with this wall for too many hours...

I think doing this:

!.env
*

On both Linux and mac worked, and I can't quite put my finger on what is going on.

tomasz1986 commented 3 months ago

Should this:

*
!.env

Be the same as

!.env
*

??

No, the order does matter, so in this case * needs to come last.

adriangalilea commented 3 months ago

No, the order does matter, so in this case * needs to come last.

That's unexpected, as .gitignore doesn't behave the same(I know is not the same, but seems inspired on it).And is also undocumented :/

I still think I haven't arrived at the bottom of my issues, but I've been doing this for too many hours so will take a break now.

Well, maybe this counts as documented:

The .stignore file contains a list of file or path patterns. The first pattern that matches will decide the fate of a given file.

But a explicit example may be best, ideally in a warn style that says "order matters" it does not behave like .gitingore,

calmh commented 3 months ago
Screenshot 2024-08-05 at 11 00 27
adriangalilea commented 3 months ago
Screenshot 2024-08-05 at 11 00 27

I did literally quoted that in the message above, I would suggest making that text bold at the very least.

Also, as I mentioned, I don't think that was all I was observing, but will keep testing and notify if that's indeed the case, many thanks to everyone regardless.