Problem accessing all notes from old Tomboy repo

cericson46 commented 5 years ago

I copied the repo from old Tomboy to a new directory structure. When I do the original sync only some of the notes are handled correctly. A large number are rejected because of problems in the title. Running tomboy-ng with debug options says it can't find notes in 0/100, 0/200 etc. The notes are actually in 1/100, 2/200 etc. I think this is the problem but not sure. What logs/data can I get for you?

cericson46 commented 5 years ago

Here is the debug log

debug.txt

davidbannon commented 5 years ago

cericson, did you close accidentally ?

I'm sure worried about your report, hope you don't mind me reopening ?

I'm traveling at present, about to get on a plane. But will look a bit more closly at your issue as time permits, well be a day or so I'm afraid. But will definitely get back to you. Davo

cericson46 commented 5 years ago

Glad you did. Wondered where it went. Not a critical problem. Check in when you return

On Sun, May 5, 2019, 11:50 PM davidbannon notifications@github.com wrote:

cericson, did you close accidentally ?

I'm sure worried about your report, hope you don't mind me reopening ?

I'm traveling at present, about to get on a plane. But will look a bit more closly at your issue as time permits, well be a day or so I'm afraid. But will definitely get back to you. Davo

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/tomboy-notes/tomboy-ng/issues/94#issuecomment-489493883, or mute the thread https://github.com/notifications/unsubscribe-auth/AFRXYT3TFJ3CCKLNKF4DEMLPT6TILANCNFSM4HKVEVHQ .

davidbannon commented 5 years ago

Hi Cericson, yep, I have looked at your log and your explanation above and you are right ! I have been guilty of a bad oversight I am afraid. The Tomboy sync model has not been described anywhere, pretty sad, so my implementation of it was based on reverse engineeding own installation. I never set an auto sync so a lot less sync events took place. I noted the directory structure had a leading entry, '0' in my case but as the directory below that could contain 32 characters on any system it was to be used, I reasoned that the upper level '0' would never have been indexed. Sounds like I was seriously wrong ! If your sync directory still contains a 0\something I'd dearly like to know how many digits the highest 'something' has. That way, I can replicate the Tomboy behaviour. I suspect it might relate to the max size of particular integers used in Mono ...

(Windows) a command like - dir "C:\Users\ChuckE\Dropbox\Tomboy Notes\0*" Will (hopefully) generate quite a long output but in there would be something I need. But maybe its been pruned.......

Davo

cericson46 commented 5 years ago

Welcome back. Attached is the directory listing of the Tomboy notes directory. It looks like a forked directory so that each sub-directory contains a limited number of files. It looks like each sub-directory contains a manifest, maybe to keep track of deleted files? Hope this helps.

notes.txt

davidbannon commented 5 years ago

Wow, there is some very strange sequences in there. But apparently using no more than three digits for the actual working directory. That I really don't understand. And lots of skipped blocks. I really don't want to be looking back through the Tomboy mono code, there there be dragons !

But maybe thats all I can do to be sure I don't break some obscure Tomboy rule.

If you are still syncing on one or more machines using Tomboy, then you will need to wait until I have gotten to the bottom of this thing. If you are using tomboy-ng on all clients, then what you could do is start a new sync run, perhaps -

One one machine, with a directory full of notes, perhaps last synced with Tomboy, copy all those notes to the appropriate tomboy-ng directory. Don't have a windows machine with me (in Zurich at present) but if you open tomboy-ng, settings, it will display where its default notes directory is.
Make a new, empty sync directory and sync that tomboy-ng to it.
Sync the other machines to the same directory.

As a short term fix, I will make tomboy-ng look for intermediate sync directories greater than '0', thats relatively easy. But making it behave in the same manner as Tomboy will be harder. So, until done properly, there will always be a risk that tomboy-ng makes something Tomboy cannot handle. bad for people using a mixed env.

Davo

cericson46 commented 5 years ago

I have been using Tomboy for years so I am not surprised there are dead ends in the forked directory structure. It has been the "scrap pieces of paper" collecting notes from phone calls and online activity. Lots of adds, changes and deletes.

Presently I am keeping Tomboy and -ng in separate sync files. I am manually using copy/paste to add more notes from the original to -ng, which appears to be stable. I have not investigated the evolving directory structure.

I have -ng going on a Windows 10 desktop and a Linux Lubuntu laptop sharing a sync file in a Dropbox folder. Eventually I plan to add Android access from my Fire 7 and Pixel 3

On May 10, 2019, at 6:27 AM, davidbannon notifications@github.com wrote:

Wow, there is some very strange sequences in there. But apparently using no more than three digits for the actual working directory. That I really don't understand. And lots of skipped blocks. I really don't want to be looking back through the Tomboy mono code, there there be dragons !

But maybe thats all I can do to be sure I don't break some obscure Tomboy rule.

If you are still syncing on one or more machines using Tomboy, then you will need to wait until I have gotten to the bottom of this thing. If you are using tomboy-ng on all clients, then what you could do is start a new sync run, perhaps -

One one machine, with a directory full of notes, perhaps last synced with Tomboy, copy all those notes to the appropriate tomboy-ng directory. Don't have a windows machine with me (in Zurich at present) but if you open tomboy-ng, settings, it will display where its default notes directory is.

Make a new, empty sync directory and sync that tomboy-ng to it.

Sync the other machines to the same directory.

As a short term fix, I will make tomboy-ng look for intermediate sync directories greater than '0', thats relatively easy. But making it behave in the same manner as Tomboy will be harder. So, until done properly, there will always be a risk that tomboy-ng makes something Tomboy cannot handle. bad for people using a mixed env.

Davo

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or mute the thread.

[ { "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": "https://github.com/tomboy-notes/tomboy-ng/issues/94#issuecomment-491241224", "url": "https://github.com/tomboy-notes/tomboy-ng/issues/94#issuecomment-491241224", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

davidbannon commented 5 years ago

OK, line #492 of FileSystemSyncServer.cs the the origional Tomboy source has a line function -

        private string GetRevisionDirPath (int rev)
        {
            return Path.Combine (
                           Path.Combine (serverPath, (rev/100).ToString ()),
                           rev.ToString ());
        }

The rule seems to be as the revision number exceeds (eg) 100, move its synced dir to ~/1/100. And at revno=200, we'll move its synced dir to ~/2/200 and so on. Is this to prevent too many synced dirs in one upper directory ? All modern filesystems can handle at least 32K entries, maybe early windows systems struggled ? Implies we should expect trouble beyond revision number =100*100 ?

And, all this means a sync repo written to by -ng with a revno greater than 100 is going to be incompatible with Tomboy - Oh, wow, I have stuffed up badly here.

Solution seems to be to write any new synced dir in the Tomboy manner and when reading a sync dir, try (eg at rev number 741) looking for file in both ~/0/741 and ~/7/741 . Not hard to code up but will require some serious testing before release.

And a tool that rewrites the -ng sync dir in Tomboy style ?

Not happy !

davidbannon commented 5 years ago

OK, fix for this will be in next release, v0.23, "a couple of days". Will seemlessly read either dir structure, will write Tomboy's way. Davo

davidbannon commented 5 years ago

OK, 0.23 released, fixes this problem.

cericson46 commented 5 years ago

Thanks for your efforts. Just getting back to this. Will let you know if any problems.

tomboy-notes / tomboy-ng

Problem accessing all notes from old Tomboy repo #94