Impossible to include a sub-subdirectory via griveignore

[alanmilinovic]

I used latest version of Grive (0.5.1-1+git20180727-webupd8-bionic0) today taken from launchpad.net and it crippled so many things. I am using partial sync on two folders and with latest master version, my google root folder with all files is deleted, and new folder is created but empty. Currently I have 10 empty folders. Also api problem with limit reached is still there as a HUGE limitation.

Very bad work I must say, now I switched to some older version (0.5.1-1+git20170322-webupd8-bionic0). With old version things are fine, ignore file doesn't work in addition.

[vitalif]

Hi, I've only merged some pull requests before all of this happened to you :) It seems there was a problematic change coming from PR #201 Everything else looks really innocent - can you retry with a build from master?

[alanmilinovic]

Hi, thank you for fast reply. Yes I will try to build original code from master and let you know.

[alanmilinovic]

Ok, so this is what I did:

1. Purging of "grive" package.

2. Building from master.

3. Put all files in folder /home/amilino/Google Drive/amilino/Private/Programs/Aimp/ from google drive cloud (exactly this remote location on my google drive cloud server: amilino/Private/Programs/Aimp), to confirm that grive will not find anything to sync.

4. cd "/home/amilino/Google Drive"

5. Tried dry run: amilino@Jumper:/home/amilino/Google Drive$ grive -s "/amilino/Private/Programs/Aimp/" -P --dry-run Reading local directories Reading remote server file list Synchronizing files (dry-run) sync "./amilino/Private" deleted in remote. deleting local Finished!

Folder amilino/Private/Programs/Aimp exist on my google drive cloud, checked one more time.

5. Deleted file .grive on my google drive root folder locally.

6. amilino@Jumper:/home/amilino/Google Drive$ grive -s "/amilino/Private/Programs/Aimp/" -P --dry-run -a ----------------------- Please go to this URL and get an authentication code:

https://accounts.google.com/o/oauth2/auth?[REMOVED CONTENT]...

----------------------- Please input the authentication code here: 4/AACYOG[REMOVED CONTENT]... Reading local directories Reading remote server file list Synchronizing files (dry-run) sync "./amilino/Private" doesn't exist in server, uploading sync "./amilino/Private/Programs" doesn't exist in server, uploading sync "./amilino/Private/Programs/Aimp" doesn't exist in server, uploading sync "./amilino/Private/Programs/Aimp/AIMP.exe" doesn't exist in server, uploading sync "./amilino/Private/Programs/Aimp/AIMP.ini" doesn't exist in server, uploading sync "./amilino/Private/Programs/Aimp/AIMP.Runtime.dll" doesn't exist in server, uploading sync "./amilino/Private/Programs/Aimp/AIMP.Shared.dll" doesn't exist in server, uploading sync "./amilino/Private/Programs/Aimp/AIMP.url" doesn't exist in server, uploading sync "./amilino/Private/Programs/Aimp/AIMPac.exe" doesn't exist in server, uploading sync "./amilino/Private/Programs/Aimp/AIMPate.exe" doesn't exist in server, uploading sync "./amilino/Private/Programs/Aimp/bass.dll" doesn't exist in server, uploading ... sync "./amilino/Private/Programs/Aimp/Plugins" doesn't exist in server, uploading sync "./amilino/Private/Programs/Aimp/Plugins/aimp_AnalogMeter" doesn't exist in server, uploading sync "./amilino/Private/Programs/Aimp/Plugins/aimp_AnalogMeter/aimp_AnalogMeter.dll" doesn't exist in server, uploading sync "./amilino/Private/Programs/Aimp/Plugins/aimp_AnalogMeter/desktop.ini" doesn't exist in server, uploading ... sync "./amilino/Private/Programs/Aimp/Uninstall.dat" doesn't exist in server, uploading sync "./amilino/Private/Programs/Aimp/Uninstall.exe" doesn't exist in server, uploading Finished!

All this exist on my google drive cloud server. Also it exist locally, so there shouldn't be anything to sync. With previously mentioned version it is working fine. Don't know why on first run it tries to remove local folder.

[alanmilinovic]

And this is my .griveignore file located in root folder (/home/amilino/Google Drive) which doesn't work with latest version: *Thumbs.db *desktop.ini **/Thumbs.db **/desktop.ini *.odt# *.ods# *.odp# *.odg# **/*.odt# **/*.ods# **/*.odp# **/*.odg# *.gsheet *.gdoc *.gmap *.gform *.gslides *.gdraw *.gtable **/*.gsheet **/*.gdoc **/*.gmap **/*.gform **/*.gslides **/*.gdraw **/*.gtable

[alanmilinovic]

I did re-installation one more time and now things are getting better, but there are still some things to be fixed:

1. .griveignore is not taken into consideration: sync "./amilino/Private/Programs/Aimp/Help/desktop.ini" doesn't exist in server, uploading

2. I removed one folder with one file inside on google drive web, and grive thinks that this file was created locally and tries to upload it: sync "./amilino/Private/Programs/Aimp/!Backup" doesn't exist in server, uploading sync "./amilino/Private/Programs/Aimp/!Backup/desktop.ini" doesn't exist in server, uploading

[alanmilinovic]

Any help, what am I doing wrong?

[alanmilinovic]

Cannot believe there isn't a stable and good working version of google drive synchronization for Linux out there. I just lost 2 days trying out so many solutions and none of them works. :(

[vitalif]

Local deletion works, remote deletion works too - just checked both. However, for deletions to work correctly you need to have a synced state prior to the deletion. If you manipulate file timestamps grive may think the file is created locally, not deleted in remote. Sometimes you can also try to remove .grive_state and start over. What for .griveignore: are you talking about it being ignored when used with -s?

[vitalif]

(and if so, do you REALLY need both?)

[alanmilinovic]

I removed .grive_state and it is not recreated, maybe because I used --dry-run all the time. Yes I need both, I am syncing only one folder and I don't want to have desktop.ini uploaded on my google drive server.

[vitalif]

why can't you setup your .griveignore to include settings equivalent to -s?

[alanmilinovic]

Can you give me example how to include sync only for this location amilino/Private/Programs/Aimp on my google cloud?

[alanmilinovic]

Hm, tried so many combinations to exclude all and include single subdir but no luck. Any help would be much appreciated.

[vitalif]

*
!amilino/Private/Programs/Aimp

and then all other stuff with extension exclusions

[alanmilinovic]

Tnx, but now nothing is getting synced. Sync is finished successfully but no changes detected and I did one change localy and one on google cloude server. Tried without --dry-run also.

[alanmilinovic]

I also tried to add !.griveignore and !*.griveignore

in addition. Same result, running out of ideas.

[vitalif]

Strange, I'll recheck it later

[alanmilinovic]

Ok, tnx.

[vitalif]

I think second line doesn't work because "amilino" 1st level folder is already excluded by *. So we need something like

*
!amilino
!amilino/Private
!amilino/Private/Programs
!amilino/Private/Programs/Aimp

[vitalif]

In fact git works the same... it's even stated in man gitignore :-)

[alanmilinovic]

Yes, I know, I think I already tried that, doesn't work. Now it is syncing everything.

What if I have files inside amilino/Private should't be like this then

*
!amilino
amilino/Private/**
!amilino/Private

...

[alanmilinovic]

https://gist.github.com/hieblmedia/9318457

[alanmilinovic]

I tried this also and nothing synced (with and without last line).

*
!.griveignore
!amilino
amilino/**
!amilino/Private
amilino/Private/**
!amilino/Private/Programs
amilino/Private/Programs/**
!amilino/Private/Programs/Aimp
!amilino/Private/Programs/Aimp/**

[vitalif]

Okaaay, I'll try it myself when I get back to PC :-)

[vitalif]

Yes, problem is that current conversion of griveignores to regexes ignores pattern order. Yeah, that's a fail that you can't include a single (sub-sub)directory via griveignore...) I'll definitely fix it.

[vitalif]

In fact that's probably the best that is possible with regexes... soooo I should either change matching engine to something else or incorporate some other kind of hack :)

[alanmilinovic]

Ok, tnx for the fix, but what is the syntax for this to work on my example? I tried so many examples now and syncing is not limited to one folder.

[alanmilinovic]

So this is how it should work, if you ask me. Previously removed .grive and .grive_state folders locally. I did testing on this.

# Ignore everything
*

!.grive
!.grive_state
!.griveignore

!/amilino
/amilino/**
!/amilino/Private/
/amilino/Private/**
!/amilino/Private/Programs/
/amilino/Private/Programs/**
!/amilino/Private/Programs/Aimp/
!/amilino/Private/Programs/Aimp/**

# Ignore Windows system files
*Thumbs.db
*desktop.ini
**/Thumbs.db
**/desktop.ini

After running grive --dry-run, this is what I get.

Reading local directories
Reading remote server file list
Synchronizing files (dry-run)
sync "./.griveignore" changed in remote. downloading
Finished!

Wrong matching:

• .griveignore has newer date locally then remote file, it shouldn't be candidate for downloading from remote.
• Some files in remote Aimp folder are newer on remote, they are not recognized as downloading candidates at all. Some files are deleted on remote side, also not discovered by sync.
• Some files are new in local Aimp folder but not discovered they should be candidates for uploading.

[alanmilinovic]

One more question, why is .grive_state not generated if --dry-run used?

[vitalif]

Now it should work with just:

**
!a/b/c/d

[vitalif]

.grive_state contains information about the state after sync is finished, so if it's written, next sync will do nothing

[alanmilinovic]

And how to add desktop.ini to be ignored in addition? :) Currently it is not considered in my testing. I added this lines at the end.

*desktop.ini
**/desktop.ini

[vitalif]

Yes, that's not possible - order is still ignored at the moment. Re-exclusion needs another matching engine...

[alanmilinovic]

Ok, tnx for the fix. Looking forward to the new releases.