search

bcedu / VGrive

Google Drive client for linux
GNU General Public License v3.0
313 stars 33 forks source link

VGrive is downloading and excluding the same files over and over #70

Open RudahXimenes opened 4 years ago

RudahXimenes commented 4 years ago

VGrive downloaded all my files from Google Drive, but everytime it checks my local files and then erase some of them. After, it identify that are missing files localy, then download from Google Drive same file it erased minutes ago. And it keep going everytime I change anything localy or remotely.

Happened once the VGrive uploaded same files twice to my Google Drive, so I had 2 same files with the same name in my online account.

Thank you in advance!

RudahXimenes commented 4 years ago

Today I found out that all the files VGrive bugs have some special character, like ~, #, !, + or others.

Today buged specifically with a file named .~lock.Curriculo - loja.odt#

And I don't know about accents, cause I didn't tried it, but as my language uses lots of accents, soon I'll be trying it.

As suggestion, maybe VGrive should ignore hidden files and backup files (started with ~).

bcedu commented 4 years ago

Actually in the lastest release this should be already fixed: https://github.com/bcedu/VGrive/releases/tag/1.4.0

This release is pending to be published by elementary at the app center.

Which version do you have installed?

RudahXimenes commented 4 years ago

@bcedu I'm using Flatpak version. I cannot see in the app which version it is, but according to App Centre (Flathub), it is 1.4.0

RudahXimenes commented 4 years ago

Today I could copy the log to you understand better:

Everything is up to date!
CHANGE IN LOCAL FILE: Senhas.kdbx uploaded ✓
CHANGE IN LOCAL FILE: Senhas.kdbx uploading newest version...
NEW REMOTE FILE: Trabalho de Gestalt-Terapia.pdf downloaded ✓
NEW REMOTE FILE: Trabalho de Gestalt-Terapia.pdf downloading...
NEW REMOTE FILE: Vinculação da gestante e apego materno fetal.pdf downloaded ✓
NEW REMOTE FILE: Vinculação da gestante e apego materno fetal.pdf downloading...
NEW REMOTE FILE: 15311370_1793974630861549_408519936_o.jpg downloaded ✓
NEW REMOTE FILE: 15311370_1793974630861549_408519936_o.jpg downloading...
NEW REMOTE FILE: 1509286_10202822622983581_1927100839_n.jpg downloaded ✓
NEW REMOTE FILE: 1509286_10202822622983581_1927100839_n.jpg downloading...
NEW REMOTE FILE: 12407551_1043541459018507_1201894677_n.jpg downloaded ✓
NEW REMOTE FILE: 12407551_1043541459018507_1201894677_n.jpg downloading...
NEW REMOTE FILE: Questionário de Estilos de Apego Adulto (EAA).odt downloaded ✓
NEW REMOTE FILE: Questionário de Estilos de Apego Adulto (EAA).odt downloading...
NEW REMOTE FILE: Formulação de caso - Lucas.ods downloaded ✓
NEW REMOTE FILE: Formulação de caso - Lucas.ods downloading...
NEW REMOTE FILE: Registro de Experimento.odt downloaded ✓
NEW REMOTE FILE: Registro de Experimento.odt downloading...
NEW REMOTE FILE: Resumos ausentes downloaded ✓
NEW REMOTE FILE: Resumos ausentes downloading...
NEW REMOTE FILE: Anais da ATC - Com superviões.odt downloaded ✓
NEW REMOTE FILE: Anais da ATC - Com superviões.odt downloading...
NEW REMOTE FILE: config tools.exe downloaded ✓
NEW REMOTE FILE: config tools.exe downloading...
NEW REMOTE FILE: kernel32 downloaded ✓
NEW REMOTE FILE: kernel32 downloading...
NEW REMOTE FILE: microdrum.exe.manifest downloaded ✓
NEW REMOTE FILE: microdrum.exe.manifest downloading...
NEW REMOTE FILE: microdrum.ui downloaded ✓
NEW REMOTE FILE: microdrum.ui downloading...
NEW REMOTE FILE: Microsoft.VC90.CRT.manifest downloaded ✓
NEW REMOTE FILE: Microsoft.VC90.CRT.manifest downloading...
NEW REMOTE FILE: msvcm90.dll downloaded ✓
NEW REMOTE FILE: msvcm90.dll downloading...
NEW REMOTE FILE: msvcp90.dll downloaded ✓
NEW REMOTE FILE: msvcp90.dll downloading...
NEW REMOTE FILE: msvcr90.dll downloaded ✓
NEW REMOTE FILE: msvcr90.dll downloading...
NEW REMOTE FILE: pins.ini.txt downloaded ✓
NEW REMOTE FILE: pins.ini.txt downloading...
NEW REMOTE FILE: qcncodecs4.dll downloaded ✓
NEW REMOTE FILE: qcncodecs4.dll downloading...
NEW REMOTE FILE: qjpcodecs4.dll downloaded ✓
NEW REMOTE FILE: qjpcodecs4.dll downloading...
NEW REMOTE FILE: qkrcodecs4.dll downloaded ✓
NEW REMOTE FILE: qkrcodecs4.dll downloading...
NEW REMOTE FILE: qglgraphicssystem4.dll downloaded ✓
NEW REMOTE FILE: qglgraphicssystem4.dll downloading...
NEW REMOTE FILE: qsvgicon4.dll downloaded ✓
NEW REMOTE FILE: qsvgicon4.dll downloading...
NEW REMOTE FILE: QtCore4.dll downloaded ✓
NEW REMOTE FILE: QtCore4.dll downloading...
NEW REMOTE FILE: QtOpenGL4.dll downloaded ✓
NEW REMOTE FILE: QtOpenGL4.dll downloading...
NEW REMOTE FILE: QtSvg4.dll downloaded ✓
NEW REMOTE FILE: QtSvg4.dll downloading...
NEW REMOTE FILE: QtXml4.dll downloaded ✓
NEW REMOTE FILE: QtXml4.dll downloading...
NEW REMOTE FILE: rtmidi_python.pyd downloaded ✓
NEW REMOTE FILE: rtmidi_python.pyd downloading...
NEW REMOTE FILE: select.pyd downloaded ✓
NEW REMOTE FILE: select.pyd downloading...
NEW REMOTE FILE: _ctypes.pyd downloaded ✓
NEW REMOTE FILE: _ctypes.pyd downloading...
NEW REMOTE FILE: _hashlib.pyd downloaded ✓
NEW REMOTE FILE: _hashlib.pyd downloading...
NEW REMOTE FILE: _ssl.pyd downloaded ✓
NEW REMOTE FILE: _ssl.pyd downloading...
NEW REMOTE FILE: lanceiro.png downloaded ✓
NEW REMOTE FILE: lanceiro.png downloading...
NEW REMOTE FILE: obj_mesa.png downloaded ✓
NEW REMOTE FILE: obj_mesa.png downloading...
NEW REMOTE FILE: argenteo.png downloaded ✓
NEW REMOTE FILE: argenteo.png downloading...
NEW REMOTE FILE: guerreiro1.png downloaded ✓
NEW REMOTE FILE: guerreiro1.png downloading...
NEW REMOTE FILE: Identidade_frente.png downloaded ✓
NEW REMOTE FILE: Identidade_frente.png downloading...
DELETE REMOTE FILE: python27.dll
DELETE REMOTE FILE: qtwcodecs4.dll
DELETE REMOTE FILE: qtaccessiblewidgets4.dll
DELETE REMOTE FILE: pyexpat.pyd
DELETE REMOTE FILE: _socket.pyd
DELETE REMOTE FILE: sip.pyd
DELETE REMOTE FILE: PyQt4.QtGui.pyd
DELETE REMOTE FILE: PyQt4.QtCore.pyd
DELETE REMOTE FILE: pins.ini
DELETE REMOTE FILE: imageformats
DELETE REMOTE FILE: bz2.pyd
DELETE REMOTE FILE: QtGui4.dll
DELETE REMOTE FILE: qgif4.dll**
DELETE LOCAL FILE: 12407551_1043541459018507_1201894677_n.jpg
DELETE LOCAL FILE: Anais da ATC - Com superviões.odt
DELETE REMOTE FILE: qtiff4.dll
DELETE REMOTE FILE: unicodedata.pyd**
DELETE LOCAL FILE: Config Tools
DELETE LOCAL FILE: lanceiro.png
DELETE LOCAL FILE: argenteo.png
DELETE LOCAL FILE: obj_mesa.png
DELETE LOCAL FILE: select.pyd
DELETE LOCAL FILE: qsvg4.dll
DELETE LOCAL FILE: Identidade_frente.png
DELETE LOCAL FILE: Vinculação da gestante e apego materno fetal.pdf
DELETE LOCAL FILE: 15311370_1793974630861549_408519936_o.jpg
DELETE LOCAL FILE: Formulação de caso - Lucas.ods
DELETE LOCAL FILE: Trabalho de Gestalt-Terapia.pdf
DELETE LOCAL FILE: Registro de Experimento.odt
DELETE LOCAL FILE: guerreiro1.png
DELETE LOCAL FILE: Resumos ausentes
DELETE LOCAL FILE: 1509286_10202822622983581_1927100839_n.jpg
DELETE LOCAL FILE: Questionário de Estilos de Apego Adulto (EAA).odt
Change detected. Updating files...
Everything is up to date!

As you can see, randomly it chooses to delete and download some random files. I'm worried because some of this files are erased in the Drive, as show in the log, so I could lose it for ever.

I'm sorry if I'm being annoying, but some of these data cannot be erased. :s

bcedu commented 4 years ago

Im workimg ok it By now stop synking until ita solved

Sent from Outlook Email App for Android divendres, 03 de gener 2020, 02:18a. m. +01:00 from RudahXimenes notifications@github.com :

Today I could copy the log to you understand better: Everything is up to date! CHANGE IN LOCAL FILE: Senhas.kdbx uploaded ✓ CHANGE IN LOCAL FILE: Senhas.kdbx uploading newest version... NEW REMOTE FILE: Trabalho de Gestalt-Terapia.pdf downloaded ✓ NEW REMOTE FILE: Trabalho de Gestalt-Terapia.pdf downloading... NEW REMOTE FILE: Vinculação da gestante e apego materno fetal.pdf downloaded ✓ NEW REMOTE FILE: Vinculação da gestante e apego materno fetal.pdf downloading... NEW REMOTE FILE: 15311370_1793974630861549_408519936_o.jpg downloaded ✓ NEW REMOTE FILE: 15311370_1793974630861549_408519936_o.jpg downloading... NEW REMOTE FILE: 1509286_10202822622983581_1927100839_n.jpg downloaded ✓ NEW REMOTE FILE: 1509286_10202822622983581_1927100839_n.jpg downloading... NEW REMOTE FILE: 12407551_1043541459018507_1201894677_n.jpg downloaded ✓ NEW REMOTE FILE: 12407551_1043541459018507_1201894677_n.jpg downloading... NEW REMOTE FILE: Questionário de Estilos de Apego Adulto (EAA).odt downloaded ✓ NEW REMOTE FILE: Questionário de Estilos de Apego Adulto (EAA).odt downloading... NEW REMOTE FILE: Formulação de caso - Lucas.ods downloaded ✓ NEW REMOTE FILE: Formulação de caso - Lucas.ods downloading... NEW REMOTE FILE: Registro de Experimento.odt downloaded ✓ NEW REMOTE FILE: Registro de Experimento.odt downloading... NEW REMOTE FILE: Resumos ausentes downloaded ✓ NEW REMOTE FILE: Resumos ausentes downloading... NEW REMOTE FILE: Anais da ATC - Com superviões.odt downloaded ✓ NEW REMOTE FILE: Anais da ATC - Com superviões.odt downloading... NEW REMOTE FILE: config tools.exe downloaded ✓ NEW REMOTE FILE: config tools.exe downloading... NEW REMOTE FILE: kernel32 downloaded ✓ NEW REMOTE FILE: kernel32 downloading... NEW REMOTE FILE: microdrum.exe.manifest downloaded ✓ NEW REMOTE FILE: microdrum.exe.manifest downloading... NEW REMOTE FILE: microdrum.ui downloaded ✓ NEW REMOTE FILE: microdrum.ui downloading... NEW REMOTE FILE: Microsoft.VC90.CRT.manifest downloaded ✓ NEW REMOTE FILE: Microsoft.VC90.CRT.manifest downloading... NEW REMOTE FILE: msvcm90.dll downloaded ✓ NEW REMOTE FILE: msvcm90.dll downloading... NEW REMOTE FILE: msvcp90.dll downloaded ✓ NEW REMOTE FILE: msvcp90.dll downloading... NEW REMOTE FILE: msvcr90.dll downloaded ✓ NEW REMOTE FILE: msvcr90.dll downloading... NEW REMOTE FILE: pins.ini.txt downloaded ✓ NEW REMOTE FILE: pins.ini.txt downloading... NEW REMOTE FILE: qcncodecs4.dll downloaded ✓ NEW REMOTE FILE: qcncodecs4.dll downloading... NEW REMOTE FILE: qjpcodecs4.dll downloaded ✓ NEW REMOTE FILE: qjpcodecs4.dll downloading... NEW REMOTE FILE: qkrcodecs4.dll downloaded ✓ NEW REMOTE FILE: qkrcodecs4.dll downloading... NEW REMOTE FILE: qglgraphicssystem4.dll downloaded ✓ NEW REMOTE FILE: qglgraphicssystem4.dll downloading... NEW REMOTE FILE: qsvgicon4.dll downloaded ✓ NEW REMOTE FILE: qsvgicon4.dll downloading... NEW REMOTE FILE: QtCore4.dll downloaded ✓ NEW REMOTE FILE: QtCore4.dll downloading... NEW REMOTE FILE: QtOpenGL4.dll downloaded ✓ NEW REMOTE FILE: QtOpenGL4.dll downloading... NEW REMOTE FILE: QtSvg4.dll downloaded ✓ NEW REMOTE FILE: QtSvg4.dll downloading... NEW REMOTE FILE: QtXml4.dll downloaded ✓ NEW REMOTE FILE: QtXml4.dll downloading... NEW REMOTE FILE: rtmidi_python.pyd downloaded ✓ NEW REMOTE FILE: rtmidi_python.pyd downloading... NEW REMOTE FILE: select.pyd downloaded ✓ NEW REMOTE FILE: select.pyd downloading... NEW REMOTE FILE: _ctypes.pyd downloaded ✓ NEW REMOTE FILE: _ctypes.pyd downloading... NEW REMOTE FILE: _hashlib.pyd downloaded ✓ NEW REMOTE FILE: _hashlib.pyd downloading... NEW REMOTE FILE: _ssl.pyd downloaded ✓ NEW REMOTE FILE: _ssl.pyd downloading... NEW REMOTE FILE: lanceiro.png downloaded ✓ NEW REMOTE FILE: lanceiro.png downloading... NEW REMOTE FILE: obj_mesa.png downloaded ✓ NEW REMOTE FILE: obj_mesa.png downloading... NEW REMOTE FILE: argenteo.png downloaded ✓ NEW REMOTE FILE: argenteo.png downloading... NEW REMOTE FILE: guerreiro1.png downloaded ✓ NEW REMOTE FILE: guerreiro1.png downloading... NEW REMOTE FILE: Identidade_frente.png downloaded ✓ NEW REMOTE FILE: Identidade_frente.png downloading... DELETE REMOTE FILE: python27.dll DELETE REMOTE FILE: qtwcodecs4.dll DELETE REMOTE FILE: qtaccessiblewidgets4.dll DELETE REMOTE FILE: pyexpat.pyd DELETE REMOTE FILE: _socket.pyd DELETE REMOTE FILE: sip.pyd DELETE REMOTE FILE: PyQt4.QtGui.pyd DELETE REMOTE FILE: PyQt4.QtCore.pyd DELETE REMOTE FILE: pins.ini DELETE REMOTE FILE: imageformats DELETE REMOTE FILE: bz2.pyd DELETE REMOTE FILE: QtGui4.dll DELETE REMOTE FILE: qgif4.dll DELETE LOCAL FILE: 12407551_1043541459018507_1201894677_n.jpg DELETE LOCAL FILE: Anais da ATC - Com superviões.odt DELETE REMOTE FILE: qtiff4.dll DELETE REMOTE FILE: unicodedata.pyd DELETE LOCAL FILE: Config Tools DELETE LOCAL FILE: lanceiro.png DELETE LOCAL FILE: argenteo.png DELETE LOCAL FILE: obj_mesa.png DELETE LOCAL FILE: select.pyd DELETE LOCAL FILE: qsvg4.dll DELETE LOCAL FILE: Identidade_frente.png DELETE LOCAL FILE: Vinculação da gestante e apego materno fetal.pdf DELETE LOCAL FILE: 15311370_1793974630861549_408519936_o.jpg DELETE LOCAL FILE: Formulação de caso - Lucas.ods DELETE LOCAL FILE: Trabalho de Gestalt-Terapia.pdf DELETE LOCAL FILE: Registro de Experimento.odt DELETE LOCAL FILE: guerreiro1.png DELETE LOCAL FILE: Resumos ausentes DELETE LOCAL FILE: 1509286_10202822622983581_1927100839_n.jpg DELETE LOCAL FILE: Questionário de Estilos de Apego Adulto (EAA).odt Change detected. Updating files... Everything is up to date!

As you can see, randomly it chooses to delete and download some random files. I'm worried because some of this files are erased in the Drive, as show in the log, so I could lose it for ever. I'm sorry if I'm being annoying, but some of these data cannot be erased. :s — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub , or unsubscribe .

RudahXimenes commented 4 years ago

Even if I'm not a great programmer, I was taking a look at the code today and found some interesting things.

The function sync_files checks if the app is syncing, check for files remotelly and locally and watch for changes as well.

Regarding the watching step, I noticed that a while loop making the code sleep for this.changes_check_period*1000000, and this.changes_check_period is set to 10, which makes the code sleep for almost 3 hours.

But I noticed, as well, that in the function watch_remote_changes () we have to wait this same time, which looks that adds with the sync_files() and we have to wait for almost 6 hours to conclude this processes. In watch_local_changes () I don't see any delay, by the way.

Looks like in the compiled version, the var changes_check_period is set to value 1, so we should wait for about half an hour instead of 6 hours.

Maybe I'm not understanding well what's happening, but it looks to me like this two usleeps are adding with it other.

I'm telling you this, cause I went through a delay after changes were detected.

Regarding the function check_deleted_files (), it was harder to me, but I tried to understand anyway.

In this var must_delete = !exist_local || !exist_remote; if any of the files is missing, the var must_delete will became TRUE, so this function will delete the other anyway. It should be fine if this function is called after file changes detection, but probably need some changes in the code to prevent this function to be called from nowhere and delete something.

Still in this function, looks like the remote file checking is not properly set. I don't know what the var DriveFile remote_file stands for, but you called and don't used it.

Also needs to verify if the connection with the Drive is properly established, cause if the connection is not established, the request could return NULL and then delete a local file.

Maybe a interesting thing to do could be split this function into 2: one to check the files and the other to delete the files, getting 2 args, "filename" and "remote/local". This way should be easier to track any issue.

I'll try to do a Pull Request with some changes as soon as possible, but I can't compile and test, unfortunatly.

bcedu commented 4 years ago

Hi! thanks for all the interest on VGrive. I've been on holidays but now I0m back to work, so some comments about what you've said:

Regarding the watching step, I noticed that a while loop making the code sleep for this.changes_check_period*1000000, and this.changes_check_period is set to 10, which makes the code sleep for almost 3 hours.

Thread.usleep works with micorseconds (https://valadoc.org/glib-2.0/GLib.Thread.usleep.html), so:

VGrive should check changes eacht 10 seconds. I'm planning to add an option to the preferences menu to configure this inteerval.

But I noticed, as well, that in the function watch_remote_changes () we have to wait this same time, which looks that adds with the sync_files() and we have to wait for almost 6 hours to conclude this processes. In watch_local_changes () I don't see any delay, by the way.

Same than in last comment: it should be only 10 seconds. In the worst case, you will have to wait 20 seconds to process changes. All this Thread.sleep are used to avoid spending all the CPU of the computer. They are two separated threads, so there must be two separated sleeps.

In this var must_delete = !exist_local || !exist_remote; if any of the files is missing, the var must_delete will became TRUE, so this function will delete the other anyway. It should be fine if this function is called after file changes detection, but probably need some changes in the code to prevent this function to be called from nowhere and delete something.

I know it's a bit confusing :see_no_evil: . The method check_deleted_files it's always the first method called. I need to process the deleted files at the begining because this method is the one which deketes the files from the VGrive library, so when 'check_remote_files' or check_local_files are called I'm sure that the files that exist on teh computer or on Google Drive are the ones that have to be and must be uploaded/downloaded.

Still in this function, looks like the remote file checking is not properly set. I don't know what the var DriveFile remote_file stands for, but you called and don't used it.

Legacy code from an innitial aproach I made to delete files. It could be removed.

Also needs to verify if the connection with the Drive is properly established, cause if the connection is not established, the request could return NULL and then delete a local file.

Yes, I know. I have a large list of TODO things to improve the code. Bu by know I've stopped making new improvements until I've finished the tests of the current code.

Maybe a interesting thing to do could be split this function into 2: one to check the files and the other to delete the files, getting 2 args, "filename" and "remote/local". This way should be easier to track any issue. I'll try to do a Pull Request with some changes as soon as possible, but I can't compile and test, unfortunatly.

It could be interessting. I will check the PR

RudahXimenes commented 4 years ago

Hello! I'm glad you make this program! So I want, at least, contribute with you! :smile: As I said before, I'm not a great programmer and don't know Vala (only know basics of C and C++), so I'll probably do some mistakes, but I want to help.

Thread.usleep works with micorseconds (https://valadoc.org/glib-2.0/GLib.Thread.usleep.html), so:

1000000 10 = 10000000 microseconds 10000000 10^-6 = 10 seconds VGrive should check changes eacht 10 seconds. I'm planning to add an option to the preferences menu to configure this inteerval.

My bad! I should've checked the function!

I know it's a bit confusing see_no_evil . The method check_deleted_files it's always the first method called. I need to process the deleted files at the begining because this method is the one which deketes the files from the VGrive library, so when 'check_remote_files' or check_local_files are called I'm sure that the files that exist on teh computer or on Google Drive are the ones that have to be and must be uploaded/downloaded.

I see. It makes me think some thing, maybe we should do it in 2 steps:

This way we could do all the checkings before make any change and, this way, make sure that the changes are real. What you think?

Yes, I know. I have a large list of TODO things to improve the code. Bu by know I've stopped making new improvements until I've finished the tests of the current code.

Alright! Count on me if necessary.