Notes from clipboard still goes over 300 limit, causing sync to be stopped

[klorinczi]

Using Clipto v4.7.3 for Windows, Clipto v4.7.11 for Android.

"Notes from clipboard" category still goes over 300 limit, causing sync to be stopped.

Unfortunately this is still not fixed nor on Windows, nor on Android.

[atrashler]

could you please make a shirt video how it looks? Probably there is some specific issue, but I can not reproduce it:(

[atrashler]

Could you also check the toggle “Do not display changed notes” in settings and tell me if the number of notes were changed or not for the category “Notes from Clipboard”

[klorinczi]

Clipto on Windows 7 Pro (Notes over limit):

Options of "Notes from clipboard" category:

Clipto on Windows 7 Pro (with “Do not display changed notes” option turned on):

I think it should work as FILO buffer (First In, Last out). So should delete the oldest unstarred note, when reaches the 300 note category limit.

It seems buggy now, as currently the oldest notes are not deleted from the category when it reaches the limit.

May be useful a user option to allow user confirmation of deleting the oldest note.

Note: I use multiple clients at the same time: 1) home computer, Windows 7 Pro 2) work computer, Windows 7 Pro 3) notebook, Windows 10 4) Android, Samsung Note 3 5) Android, Samsung Note 4

[atrashler]

Got it! With “Do not display changed notes” the app does not show in this category any notes that were marked with “Star”, “Tags” or “Title”, because such notes are treated by the app as “important for you” and don’t counted by functionality “Limit the Clipboard”. For your scenarios you need to delete such “important notes” automatically?

[atrashler]

Now with the option enabled “Do not display changed notes” the app show you correct counter of notes that can be deleted (133). If you decrease the limit to 100 then the app will replace the oldest note with the newest one if the newest note did not get “auto tags” when it was created. But it will not delete all notes exceeding this limit (in our example, 33 notes), just the oldest one. It is current behaviour that can be discussed:)

[klorinczi]

Got it! With “Do not display changed notes” the app does not show in this category any notes that were marked with “Star”, “Tags” or “Title”, because such notes are treated by the app as “important for you” and don’t counted by functionality “Limit the Clipboard”. For your scenarios you need to delete such “important notes” automatically?

No, I do NOT want to delete the starred notes!

[klorinczi]

Now with the option enabled “Do not display changed notes” the app show you correct counter of notes that can be deleted (133). If you decrease the limit to 100 then the app will replace the oldest note with the newest one if the newest note did not get “auto tags” when it was created. But it will not delete all notes exceeding this limit (in our example, 33 notes), just the oldest one. It is current behaviour that can be discussed:)

Why should be the counter of notes that can be deleted 133? Isn't too much?

If "Notes from clipboard" category has 335 notes, the limit is 300, then should delete only the oldest 35 one. If there are 13 starred ones, 6 labeled, 2 snippets, then should delete the oldest 35 notes (which should not be any of starred, labeled, or snippet).

[atrashler]

Now with the option enabled “Do not display changed notes” the app show you correct counter of notes that can be deleted (133). If you decrease the limit to 100 then the app will replace the oldest note with the newest one if the newest note did not get “auto tags” when it was created. But it will not delete all notes exceeding this limit (in our example, 33 notes), just the oldest one. It is current behaviour that can be discussed:)

Why should be the counter of notes that can be deleted 133? Isn't too much?

If "Notes from clipboard" category has 335 notes, the limit is 300, then should delete only the oldest 35 one. If there are 13 starred ones, 6 labeled, 2 snippets, then should delete the oldest 35 notes (which should not be any of starred, labeled, or snippet).

You are right. I wrote it wrongly. If limit is set to 100 and the category has 133 notes then only the oldest 33 could be deleted. But the logic with auto delete works like “for the new note exceeding the limit delete the only one oldest note”. For example: 1) you had a limit 300 and 133 notes 2) you set a limit to 100 3) you created a new note from clipboard 4) the app will delete the one oldest note and the number of notes in the category will be 133 (not 134)

[klorinczi]

Ok. So can be implemented the logic I wrote?

[klorinczi]

My programming logic for this problem is (simplified):

$notes_from_clipboard_num = 335
$category_limit = 300
$starred = 13
$labeled = 6
$snippets = 2
$notes_array = [ filled_with_notes() ]
$reversed_notes_array = reverse_notes_array()
while ( $reversed_notes_array as $note ) {
  if ( $notes_from_clipboard_num >  $category_limit) {
    if ( is_starred($note) or is_labeled($note) or is_snippet($note) ) {
      next
    } else {
      delete_this_note()
      $notes_from_clipboard_num = $notes_from_clipboard_num - 1
    } 
  } else {
    end
  } 
} 

What do you think?

[atrashler]

My programming logic for this problem is (simplified):

$notes_from_clipboard_num = 335
$category_limit = 300
$starred = 13
$labeled = 6
$snippets = 2
$notes_array = [ filled_with_notes() ]
$reversed_notes_array = reverse_notes_array()
while ( $reversed_notes_array as $note ) {
  if ( $notes_from_clipboard_num >  $category_limit) {
    if ( is_starred($note) or is_labeled($note) or is_snippet($note) ) {
      next
    } else {
      delete_this_note()
      $notes_from_clipboard_num = $notes_from_clipboard_num - 1
    } 
  } 
} 

What do you think?

Now the "while loop" is finished when the only one oldest note is deleted. I had thought about the logic you described and decided not to do it because of a possible user typo: for example, user stores 10000 notes and decided to decrease the limit to 9000, but set it to 100 by mistake and lost 9900 notes as a result.

Usually, with the logic that is implemented now (for each new note exceeding the limit delete the only one oldest) it should not be possible to exceed the limit. But if the user decides to decrease the limit, at first he has to manually delete all notes exceeding the new one. Actually, this logic requires investigation for possible improvement, because now it seems not obvious:)

[klorinczi]

How about confirm dialog window (maybe optionally could be turned off), when more than 1 note should be deleted to keep the limit?

My logic + confirm window.

[atrashler]

How about confirm dialog window (maybe optionally could be turned off), when more than 1 note should be deleted to keep the limit?

My logic + confirm window.

When this dialog will be displayed? Each time the app detects that there are more than one note over the limit and can be deleted or at the first time user applied new limit?

[klorinczi]

How about confirm dialog window (maybe optionally could be turned off), when more than 1 note should be deleted to keep the limit?

My logic + confirm window.

When this dialog will be displayed? Each time the app detects that there are more than one note over the limit and can be deleted or at the first time user applied new limit?

Currently, if count is over limit, sync is stopped silently, even if only 1 is over.

This behavior is not worser, than asking confirmation to delete multiple notes over limit immediately (an option might be good idea to allow deleting X number of notes in one step).

My suggestion:

Purge notes in a category:

1) Stop sync, if note count is over limit. (keep this as default?).

2)

• Confirm to delete multiple note count over limit, each time it happens. Option values: a) Always. b) When X or less note needs to be deleted. c) Never ask confirmation to delete over limit notes.
• Delete number option: number of notes or less allowed to be deleted in one step, when note count is over category limit. Default: 0 means disabled.

When a) or c) option is selected, then "Delete number option" is disabled and grayed.

Opinion?

[atrashler]

Currently, if count is over limit, sync is stopped silently, even if only 1 is over.

It stops syncing only if the total number of all notes exceed the limit. But (as example) if you activate 'Do not display changed notes' and configure the limit to the value that let you to stay in the total limit, such situation should not happen. As I understand now, the problem exists because:

1) you configured the limit as 300 2) you have only 133 notes in the category 'Notes from Clipboard' and 340 - 133 = 207 notes in other categories

If (as example) you delete 83 notes from the category 'Notes from Clipboard' (so the total number can be <= 300) and change the limit to 50, then the app should correctly delete the oldest note each time you copy something. But this will work until some of the copied notes get 'auto-tags' or 'starred' and will be excluded from the logic of 'auto-delete'... (maybe it could me useful to delete tagged notes also...)

than asking confirmation to delete multiple notes over limit immediately Usually the app is in background when user actively Copy texts outside of the app. He will not see this dialog and the expected behaviour of continuous synchronisation will stop working.

My suggestion:

Purge notes in a category:

Stop sync, if note count is over limit. (keep this as default?).

• Confirm to delete multiple note count over limit, each time it happens. Option values: a) Always. b) When X or less note needs to be deleted. c) Never ask confirmation to delete over limit notes.
• Delete number option: number of notes or less allowed to be deleted in one step, when note count is over category limit. Default: 0 means disabled.

When a) or c) option is selected, then "Delete number option" is disabled and grayed.

Opinion?

I need to think about it.

(As a workaround) If you are using 'Clipboard' functionality actively and you need more notes to keep in sync for free you could ask me, send me your user_uid (Settings -> scroll down -> email -> field 'user_uid' from the email), required number of notes and I can upgrade your account with the given number (it is mentioned on site, that the limit is put to prevent system from abuse and it is not a problem to extend it by request). The only condition of this deal is that if you have performance problems with the application with large amounts of data, any bugs detected, you let me know so that I can fix it.

[klorinczi]

you have only 133 notes in the category 'Notes from Clipboard' and 340 - 133 = 207 notes in other categories

I don't know how. Currently I have

All 311
Starred 13
Untagged 310
Notes from Clipboard 152/300  (with "Don't display changed notes" turned ON), otherwise 306/300
Snippets 0
Tags 5, but only 1 note is affected.
Recycle bin 0

Why I have only 152 / 300 in Notes from Clipboard, but I haver only 13 starred + 1 tagged note.

But this will work until some of the copied notes get 'auto-tags' or 'starred' and will be excluded from the logic of 'auto-delete'... (maybe it could me useful to delete tagged notes also...)

My logic handles this situation, too.

Also I can not filter the inverse of "Don't display changed notes" from Notes from Clipboard. Can not investigate, which was is changed, so doesn't list in Notes from Clipboard. Could you add a tagged, favorites flag, unstarred, with title, without title, snippet tag, unsynchronized, synchronized labels to Advanced Search?

PS: "favorites flag" is same as "Starred"? PS: I don't agree to delete tagged notes!

[atrashler]

Why I have only 152 / 300 in Notes from Clipboard, but I haver only 13 starred + 1 tagged note.

Strange. 306 - 152 = 154 notes that the app excluded from auto-delete.. 1) Do these notes have a title? 2) Do you have the same counters on different devices? (or on the same, but try to sign in Web Editor)

[atrashler]

Could you add a tagged, favorites flag, unstarred, with title, without title, snippet tag, unsynchronized, synchronized labels to Advanced Search

yes. It will be supported in advanced search

[atrashler]

PS: "favorites flag" is same as "Starred"?

yes. lack of the translation.

[atrashler]

PS: I don't agree to delete tagged notes!

you do not need do it:)

[klorinczi]

2. Do you have the same counters on different devices?

No, at least currently: Windows: 316 (has unsynced items) Android1: 312 Android2: 319 (has unsynced items)

[atrashler]

2. Do you have the same counters on different devices?

No, at least currently: Windows: 316 (has unsynced items) Android1: 312 Android2: 319 (has unsynced items)

Do these notes (which do not displayed when you check “do not display changed notes”) have title?

[klorinczi]

Now it changed:

• Windows: 323 (has unsynced items): Notes from Clipboard: 198 /300
• Android1: 304: Notes from Clipboard: 288 /300
• Android2: 304 (has unsynced items): Notes from Clipboard: 288 /300

Strange, because I did not change anything (except typing & using clipboard) since our last message. I have check “do not display changed notes” for all clients.

Android ones still complaining about over limit.

Usually I do not set title for any of the notes. Rarely for those, which are tagged or starred.

[Abragus]

I have a similar issue. I use Clipto on 3 devices: Android 11 phone, Windows laptop, Windows desktop. Copied items are synced when copied on the laptop and the phone, and show up on all three devices. But when copying on the computer, the items are not synced. It also says that "Sync is off due to the reached limit", and shows 454 Notes From Clipboard ("Do not display changed notes"-switch is off, so all notes are shown), even though the limit is set to 200. The laptop and phone say there are 283 and 273 notes, respectively (I recently changed the limit from 300 to 200, so them showing more than 200 is okay.)

This is the same issue that I mentioned in #88 .

I don't think the problem is in the logic, as it works perfectly on two of my devices.

[atrashler]

I need to reproduce it myself. I have few hundred of notes from clipboard and limit works correctly. Probably I am doing something wrong...

[Abragus]

I mean, you're the one who has it working, so if anyone, I'm the one doing something wrong :D

I tried deleting a bunch of notes, my desktop is now at 283 total, about 100 higher than my other devices (guess these were copied on my desktop, and didn't get synced). It still won't sync, even though I'm not above the limit. I also reinstalled the app a bunch of times, just to try something.

[atrashler]

I mean, you're the one who has it working, so if anyone, I'm the one doing something wrong :D

I tried deleting a bunch of notes, my desktop is now at 283 total, about 100 higher than my other devices (guess these were copied on my desktop, and didn't get synced). It still won't sync, even though I'm not above the limit. I also reinstalled the app a bunch of times, just to try something.

By the way, could you click on the red banner you see and send me the screenshot of opened dialog (if such is available). Or, could you send me current statistics of notes in all categories (Starred, Clipboard, Untagged, Recycle Bin, etc.). Probably it has some clue for me...:)

[Abragus]

Nothing happens when I click the red banner. Currently, these are the statistics on my non-working desktop: All: 302 Starred: 1 Untagged: 302 Notes from Clipboard: 299 Snippets: 0 Recycle bin: 55

On my laptop: All: 184 Starred: 1 Untagged: 184 Notes from Clipboard: 181 Snippets: 0 Recycle bin: 57

On my phone: All: 174 Starred: 1 Untagged: 174 Notes from Clipboard: 171 Snippets: 0 Recycle bin: 51

[atrashler]

The number of notes in recycle bin is also important. Could you please write it also?

[Abragus]

The number of notes in recycle bin is also important. Could you please write it also?

Yes, of course. I forgot that somehow. I'll add it to the last comment as well.

Desktop recycle bin: 55 Laptop recycle bin: 57 Phone recycle bin: 51

The limit is set to 100.

[Abragus]

Okay, so I'm not sure if I should have done this because I no longer have the statistics available, but I cleared all notes from the clipboard, only leaving my one starred note, and syncing now works from the desktop as well. I saw that there were some notes on my laptop which hadn't been synced either (about 15). I'm guessing this isn't true, but is it possible that the "desync" between the devices is caused by copying and syncing notes while one of the devices are turned off? And when the device turns on again, it doesn't get the note for some reason, leaving the 2 other devices with more notes, making them reach the limit earlier. And it only deletes notes if the limit has been reached on one specific device, instead of checking each device individually.

As I said, I don't think this is the case, but if the problem isn't with the auto deletion logic itself, this is one of few plausible explanations I see.

[Abragus]

I set the limit to 50 and copied a bunch of stuff, and the auto deletion now works as it should, deleting the oldest note when a new one is created.

I did notice though, that I have 51 notes, 1 above the limit, when also showing my one starred notes. Without it, it's 50. What would happen if the limit was att 300? Would I get a total off 301 notes, disabling cloud sync?

I think klorinczi was onto something with his auto-delete logic and this:

How about confirm dialog window (maybe optionally could be turned off), when more than 1 note should be deleted to keep the limit?

My logic + confirm window.

Changing the limit should reduce the amount of Notes from Clipboard immediately by deleting the oldest non-starred/tagged/changed notes until the limit is reached. If the change will delete more than 1 note (or something), display a confirmation message:

"Are you sure? XX notes are going to be deleted."

I had thought about the logic you described and decided not to do it because of a possible user typo: for example, user stores 10000 notes and decided to decrease the limit to 9000, but set it to 100 by mistake and lost 9900 notes as a result.

Shouldn't recently deleted notes go into the Recycle Bin for a while, allowing the user to restore accidentally deleted notes? I don't know how it works right now, but let's say notes have to be in the Recycle Bin for 24 hours before being permanently deleted automatically. If you accidentally delete 10000 notes, you will have 24 hours to restore them, even if your Recycle Bin limit is set much lower. The notes may be deleted from the cloud as soon as they go into the Recycle Bin, only being stored locally for the last 24 hours, which would free up the cloud space immediately.

On a side note, there should be a "Restore All" option in the Recycle Bin, which would simplify a lot of things in the above mentioned scenario. There might be one already, but I haven't seen it.

Sorry for the long reply. I just wanted to share these ideas and inputs with you.

[klorinczi]

Using Clipto v5.08 on Android, but still goes over the 300 limit, I have to delete manually to go below the limit.

Why the oldest (non-starred, non-labeled) note is not purged over the limit?

[Abragus]

Using Clipto v5.08 on Android, but still goes over the 300 limit, I have to delete manually to go below the limit.

Why the oldest (non-starred, non-labeled) note is not purged over the limit?

Did you try deleting all non-starred, non-labeled notes? I was able to on the computer I had problems, and that seems to have solved it for me. I now have the limit set to 200, just to make sure I absolutely don't go over the limit.

[klorinczi]

Using Clipto v5.08 on Android, but still goes over the 300 limit, I have to delete manually to go below the limit.

Why the oldest (non-starred, non-labeled) note is not purged over the limit?

Did you try deleting all non-starred, non-labeled notes? I was able to on the computer I had problems, and that seems to have solved it for me. I now have the limit set to 200, just to make sure I absolutely don't go over the limit.

I don't want to delete the starred or labeled notes. It is a simple bug, which should be fixed.

[Abragus]

Using Clipto v5.08 on Android, but still goes over the 300 limit, I have to delete manually to go below the limit.

Why the oldest (non-starred, non-labeled) note is not purged over the limit?

Did you try deleting all non-starred, non-labeled notes? I was able to on the computer I had problems, and that seems to have solved it for me. I now have the limit set to 200, just to make sure I absolutely don't go over the limit.

I don't want to delete the starred or labeled notes. It is a simple bug, which should be fixed.

You don't have to. I only deleted the non-starred and non-labeled notes, and that fixed it for me.

[klorinczi]

It would be just a temporary workaround to delete all non-starred, non-labeled notes.

Maybe the bug comes from the edited, titled notes. As far as I know, the edited, titled notes are also not deleted. The algorithm might not count with them.

Although my opinion is, that the titled or edited notes can be also purged out.

Might be a good idea to make it optional, which type of notes should be automatically purged, when limit is reached, then purge SELECTED specific types, in the following order: 1) Recycle bin notes 2) edited notes 3) notes with title 4) normal notes, copied from clipboard.

[Abragus]

Well, I don't know about that, because I have very few notes that are changed in any way. The problem for me though, where the notes would not sync, seemed to be caused by some kind of desync between the devices. On two of my devices the filter worked fine, and I had less than 300 notes. These also synced as they should. On my third device, I for some reason had more than 300 notes, so they didn't sync. Deleting all (but not starred, labeled, or others) "re-synced" the devices again, and they have all been syncing as they should ever since.

[klorinczi]

Unfortunately this bug is still alive on Windows, using Clipto v5.0.3. The note number still goes over 300, and they are not purged automatically.

[Abragus]

I'm still getting this as well, from time to time. It should be an easy fix, even if the actual root problem may be harder to find. A simple check that makes sure there are the same number of notes on all devices, and if there aren't, find the ones that differ, and try to sync them all again. If the limit is reached, delete some notes on all devices.

Fixing the desync itself would of course be better, but I understand if the cause is difficult to point out.

[atrashler]

I'm still getting this as well, from time to time. It should be an easy fix, even if the actual root problem may be harder to find. A simple check that makes sure there are the same number of notes on all devices, and if there aren't, find the ones that differ, and try to sync them all again. If the limit is reached, delete some notes on all devices.

Fixing the desync itself would of course be better, but I understand if the cause is difficult to point out.

The problem is that I can not reproduce the problem:) It works correctly on my devices, all operations are atomic. Just need to revise it again, probably I miss something.

Could you describe your environment. Is it a one device where you create all these notes or two or more?

[Abragus]

Three, one android phone, two windows computers.

The problem is that I can not reproduce the problem:)

That's why I think building a "work around" like the one I mentioned could be a good option, until you manage to find the problem, if you do.

Whenever desync is found, where one device has more or less than the others, the difference is nullified by syncing the problem-notes, and making sure they are successfully synced to all devices.

[klorinczi]

Could you describe your environment. Is it a one device where you create all these notes or two or more?

I actually use 3 Android phones + 1 Windows (v5.0.3). However I can reproduce the bug just by copy texts on Windows, it simple steps over the 300 limit, without deleting the notes over 300 limit.

Let me show my calculation algorithm:

If you look at the screenshot I posted above, All: 308 - should be 300 Starred: 46 - correct, we don't delete from here Untagged: 307 - should be 299, because 1 note is tagged Snippets: 0 -
Recycle bin: 0, max 50. Notes from clipboard: 255 / 300 - should be 253 / 253 (all over this limit should be deleted or optionally moved to Recycle bin).

Example calculation 1: Notes from clipboard = All (300) - Starred (46) - Tagged (1) - Snipppets (0) - Recycle bin (0) = 253

Example calculation 2: In other case, it should calculate like this: Notes from clipboard = All (300) - Starred (46) - Tagged (12) - Snippets (23) - Recycle bin (50) = 169

[atrashler]

Could you describe your environment. Is it a one device where you create all these notes or two or more?

I actually use 3 Android phones + 1 Windows (v5.0.3). However I can reproduce the bug just by copy texts on Windows, it simple steps over the 300 limit, without deleting the notes over 300 limit.

Let me show my calculation algorithm:

If you look at the screenshot I posted above, All: 308 - should be 300 Starred: 46 - correct, we don't delete from here Untagged: 307 - should be 299, because 1 note is tagged Snippets: 0 - Recycle bin: 0, max 50. Notes from clipboard: 255 / 300 - should be 253 / 253 (all over this limit should be deleted or optionally moved to Recycle bin).

Example calculation 1: Notes from clipboard = All (300) - Starred (46) - Tagged (1) - Snipppets (0) - Recycle bin (0) = 253

Example calculation 2: In other case, it should calculate like this: Notes from clipboard = All (300) - Starred (46) - Tagged (12) - Snippets (23) - Recycle bin (50) = 169

But the total number of notes and the number of notes in each category is the same as on other devices?

[klorinczi]

Well, after updating all 3 Android phones to latest Clipto v5.6.0, all 3 mobiles had 299 notes. Windows Clipto v5.0.3 had 302 total notes.

After copying some texts on Android phone 1: total 307 (all newly copied is unsynced) Android phone 2: total 297, Android phone 3: total 297, Windows: total 302

[klorinczi]

Strange, that if I delete Clipto notes on my 3 Android phones, to be below the 300 limit, they are still not synced.

Maybe the phones does not sync until the Windows Clipto is not below the 300 limit?

[atrashler]

Strange, that if I delete Clipto notes on my 3 Android phones, to be below the 300 limit, they are still not synced.

Maybe the phones does not sync until the Windows Clipto is not below the 300 limit?

the logic to stop/resume is local for the device. It should be resumed if the total number including the Recycle Bin does not exeed the limit. Could you tell the total number of notes in categories All + Recycle Bin ?

[atrashler]

should be fixed in 7.2.0

[klorinczi]

Waiting for the release of Windows binaries of v7.2.

[atrashler]

Waiting for the release of Windows binaries of v7.2.

7.2.3 - https://github.com/clipto-pro/Desktop/releases/tag/v7.2.3