More automated Sync/Backup options

[Goldendawn69]

Is your feature request related to a problem? Please describe.

Syncing between multiple systems/browsers is not really what interests me here. I am interested in backup capabilities, which I feel are lacking.

I wish there were a way to use the Sync/backup functionality, but I don't want to sync my data to your server due to security concerns. Sorry to say, but being a long-term technology person, I know there are too many situations where data has been leaked and used in situations where it should not be happening.

I am perfectly happy to fund you every month through your Patreon page, but there need to be other options for syncing/backing up data.

Describe the solution you'd like

It would be good if we could sync/backup automatically to Google, Dropbox, or other prime cloud-based drives. We could, in fact, do this now (by backing up local to Google Drive), but there is no automatic backup process.

What would be good is to offer an automatic backup process to backup the content of the GSD (it doesn't have to include Thumbnails) and allow them to be either synced or backed up locally. Again, automatic backup could be offered as part of your monthly Pro fee if you felt the need.

[Juraj-Masiar]

Hello, I would love to integrate the popular storage services, but it's not easy, at all. It requires OAuth implementation on the client (in the addon), and I would have to bundle also some google library in my addon that handles their API, which is not great either. Additionally, using OAuth means regular updates (usually not breaking though) and various checkups by the provider service (usually only once a year).

So, I would rather focus on some "alternative solutions", maybe some out of box ideas. This is what I can think of:

1. addons can create a folder with files inside, in your default downloads folder (this depends where your browser downloads files, so either it's the "user/Downloads" folder, or "user/Desktop". For example, I could create some kind of automatic backup in this path: <default downloads folder>/_backups/export_2024_07_09___15_18_01.group_speed_dial. You could then let your favorite storage provider to backup data in this folder (if it's supported). And if the extra folder bothers you, I think you can mark is as hidden (or as "system"), and it would "disappear". Sadly, in Linux this requires folder name starting with a ".", which is forbidden for addons.

2. I could build a "End to end" encrypted synchronization, so you would set a strong and long password locally, then it would be saved in the browser so that you don't have enter it each time browser starts, and all data send to my server would be encrypted, so not readable to me or any attacker. The main issue is, that if you would lost your device (or if your SSD dies) and you can't remember your password, you would loose your dials data forever.

UPDATE: There is a new "File System API" that may allow me to store backups in any user-selected folder. If this works (also after restarting PC), I will use it for optional regular backups. This will allow full backups including thumbnails. I could then create a full backup once a week or so... And since this targets non-server-using users, I will have to make it free :) I'll put it on my todo list, somewhere up.

[Goldendawn69]

I understand the OAuth conversation - though I know many applications that support that model.

But as stated, if there is a local capability to save your file, then people can put it into their local Dropbox, Google folder, etc. They (or like me) could backup the file offsite (I do my backups encrypted into Google Drive and Dropbox) myself, with my own encryption.

I like your approach, and if you implement it, I would support your service anyway because then the solution you are presenting would do everything I need. FVD saves locally, but they haven't been interested in making it automated for Chrome (though I do believe they did have it in Firefox) and are planning on changing that behaviour (at some stage) to force people onto their Eversync service.

Also, I liked version 2 of your suggestion—this would add that level of encryption and safety I feel is missing right now. I know you have encryption, but honestly, because of other sites' behaviours, our data (our history) is important. In all situations where I am sending data to external sites, I have a private token that encrypts the data. And if I "lost my token," that is my problem, as is everything else I have. I am responsible for storing my token and keeping it safe - otherwise, I will lose it all.

(isn't that what that piece of paper is that you store under your keyboard... with your secret information and all your passwords for :D)

Which would be easier to implement - or - would both of them be good?

Because of your response, because of the way you have already thought on this, I have decided to join your Patreon because this is the type of awesome developer behaviour we should support! So thanks.

[Juraj-Masiar]

Oh, thank you for supporting my work! That's very generous of you. Now I have to move it to the top of my todo list :)

I will be implementing both options, but I will start with the easier one, so locally created backup files. I've already read the documentation so I'll start implementing it now, so maybe next week it could be released.

There seems to be one issue though, if I'm seeing it right, only Chromium browsers currently support selecting a directory outside the downloads folder. But we will see soon, I'm writing proof of concept now.

[Goldendawn69]

That's not a problem. Right now, I use Chrome myself, so it's selfish. It's great! :D BUT, Firefox should not be ignored :D https://stackoverflow.com/questions/77129471/any-workarounds-for-using-window-showdirectorypicker-in-firefox

However, if the path location is a problem - that doesn't have to be right away; saving to the default location on a timed basis (like once a day/per hour/etc.) is the biggest benefit. The user can then write a Powershell script that moves or even use my Sync application to sync it to Google Drive.

The MVP could not include the Path selection if that proves to be a problem.

Also, there isn't a way to sync - syncing should be through your system - and for me, that would be best - if we could add additional encryption because it makes it more secure. This is for local backup. When you have as many Speed Dails as me - that is so important :D

(I use Speed Dail as my bookmark tool; I like the visual representation of bookmarks far better. So all the important stuff - goes on there :D LOTS :D) - though probably there are people out there with a lot more :D

I want to say again that supporting your work was a no-brainer. Too many developers think it is just about coding. However, engagement with your business stakeholders is just as important. Code Monkey's are a dime a dozen, but developers who are engaged - those are unique and special :D

(sorry, being a Technical Lead/Solution Arch in my company, when I engage with developers who are engaged, it is one of my awesome moments :D)

On Tue, Jul 16, 2024 at 11:36 PM Juraj Mäsiar @.***> wrote:

Oh, thank you for supporting my work! That's very generous of you. Now I have to move it to the top of my todo list :)

I will be implementing both options, but I will start with the easier one, so locally created backup files. I've already read the documentation so I'll start implementing it now, so maybe next week it could be released.

There seems to be one issue though, if I'm seeing it right, only Chromium browsers currently support selecting a directory https://developer.mozilla.org/en-US/docs/Web/API/Window/showDirectoryPicker outside the downloads folder. But we will see soon, I'm writing proof of concept now.

— Reply to this email directly, view it on GitHub https://github.com/fastaddons/GroupSpeedDial/issues/338#issuecomment-2230673005, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQH4K2FKJWI45ZWRN3ADNVTZMUAUZAVCNFSM6AAAAABK3X7YK2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZQGY3TGMBQGU . You are receiving this because you authored the thread.Message ID: @.***>

[Juraj-Masiar]

Thanks for the stackoverflow link! Sadly, it won't help, the polyfill is missing the crucial feature:

Storing a file handle in IndexedDB or sharing it with postMessage isn't currently possible unless you use native.

You see, once the user selects the folder, the app will get access to it ONLY while it holds the "approval" object (the handler). So in order to be able to do a an automatic backup to that folder, I need to store the handler and then load it when needed.

To be honest, this sounds too good to be true :), and I'm giving it 80% chance it's actually gonna work as expected :). Addons/extensions environment is sometimes too special for new features like this.

In any case, the automatic storing to the Downloads folder (sub-folder) should work, so this is what I'll use for Firefox. And if it works fine in Chrome, I'll request the missing API in the Firefox bugzilla page.

I still didn't do the proof of concept, I've discovered some random crazy huge memory leak (40GB of RAM!) that requires my attention, as it's likely related to my addon.

Thank you for the words of encouragement :), I'll do my best to make you a happy customer!

[Goldendawn69]

Just as a matter of interest have you tried turning off your extension to make sure it does not happen Also turn off automated thumbnails? The reason being when I was using FVD I noticed on Firefox it had issues with memory behaviour - which didn't happen in Chrome. I fixed it by turning off the updating of thumbnails.

On Thu, Jul 18, 2024 at 2:42 AM Juraj Mäsiar @.***> wrote:

Thanks for the stackoverflow link! Sadly, it won't help, the polyfill is missing the crucial feature:

Storing a file handle in IndexedDB or sharing it with postMessage isn't currently possible unless you use native.

You see, once the user selects the folder, the app will get access to it ONLY while it holds the "approval" object (the handler). So in order to be able to do a an automatic backup to that folder, I need to store the handler and then load it when needed.

To be honest, this sounds too good to be true :), and I'm giving it 80% chance it's actually gonna work as expected :). Addons/extensions environment is sometimes too special for new features like this.

In any case, the automatic storing to the Downloads folder (sub-folder) should work, so this is what I'll use for Firefox. And if it works fine in Chrome, I'll request the missing API in the Firefox bugzilla page.

I still didn't do the proof of concept, I've discovered some random crazy huge memory leak https://bugzilla.mozilla.org/show_bug.cgi?id=1906755(40GB of RAM!) that requires my attention, as it's likely related to my addon.

Thank you for the words of encouragement :), I'll do my best to make you a happy customer!

— Reply to this email directly, view it on GitHub https://github.com/fastaddons/GroupSpeedDial/issues/338#issuecomment-2233497446, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQH4K2ENLH2RRAUZN3LGTJ3ZMZ7GRAVCNFSM6AAAAABK3X7YK2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZTGQ4TONBUGY . You are receiving this because you authored the thread.Message ID: @.***>

[Juraj-Masiar]

Actually one of the Firefox engineers is helping me identify the issue, since we were not sure about the cause of the leak. So far it looks like my addon is trying to store something huge into "in-memory" storage and Firefox should not allow it (it should have a limit of 10MB). Firefox is also very slow releasing used memory as we discovered.

Anyway, I've spend almost whole day yesterday with checking my code so today I'll continue with normal work and once I see the leak again (it's a bit rare right now, I've seen it only on my home PC and only like 5 times), I'll check the store manually to see what monstrosity is there.

[Juraj-Masiar]

Hello, I've just finished implementing a proof of concept, and as I feared, it doesn't work as smooth as expected. The "permission" (the directory handle) is valid only for the current "session", so after restarting browser, the handle stored in IndexedDB is "expired": DOMException: The request is not allowed by the user agent or the platform in the current context.

It seems to be known issue: https://stackoverflow.com/questions/66500836/domexception-the-request-is-not-allowed-by-the-user-agent-or-the-platform-in-th

HOWEVER, as was writing this, and further testing it, suddenly Chrome asked me, whether I want to permanently allow access to the backup folder! I think I just need to re-request it twice and it will trigger the extra permission popup:

And that seems to fix the issue! I can now create backup, asynchronously, from background script, without any user interaction. So good news! :)

[Goldendawn69]

That is AWESOME! Great work - did your issue with Firefox get resolved?

Do you need someone to test your release before you release it?

Becs

On Tue, Jul 23, 2024 at 12:26 AM Juraj Mäsiar @.***> wrote:

Hello, I've just finished implementing a proof of concept, and as I feared, it doesn't work as smooth as expected. The "permission" (the directory handle) is valid only for the current "session", so after restarting browser, the handle stored in IndexedDB is "expired": DOMException: The request is not allowed by the user agent or the platform in the current context.

It seems to be known issue:

https://stackoverflow.com/questions/66500836/domexception-the-request-is-not-allowed-by-the-user-agent-or-the-platform-in-th

HOWEVER, as was writing this, and further testing it, suddenly Chrome asked me, whether I want to permanently allow access to the backup folder! I think I just need to re-request it twice and it will trigger the extra permission popup: image.png (view on web) https://github.com/user-attachments/assets/b6b6cfc4-3a7d-4440-b156-dab80157cd33

And that seems to fix the issue! I can now create backup, asynchronously, from background script, without any user interaction. So good news! :)

— Reply to this email directly, view it on GitHub https://github.com/fastaddons/GroupSpeedDial/issues/338#issuecomment-2242833059, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQH4K2ALM5AMIZ44HHABFETZNT26ZAVCNFSM6AAAAABK3X7YK2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBSHAZTGMBVHE . You are receiving this because you authored the thread.Message ID: @.***>

[Juraj-Masiar]

I'm working on the Firefox workaround right now, it will be able to create backups in the default downloads folder, in a specified subfolder. Chrome version is already finished:

This will be creating a full backups, including thumbnails in a specific location (wherever on the disk).

[Juraj-Masiar]

Done, new version 25.9 is waiting for review and brings folder specific offline backups in Chrome. In Firefox, it can create backups in the user specified subfolder in the Downloads folder.

The permissions are super strict though (in Chrome). It will ask you to allow access to the folder when you first select it, but then also while making the first backup (after restarting Chrome, week or two later) - at that time it will give you option to always allow access to the folder.

I'm sorry it took me so long, I've been sick for more than a week plus I wanted to fix some issues in this release.

[Goldendawn69]

This sounds awesome - can not wait to see it :D Will let you know how it goes - have you done any screenshots doe tutorials - if not I can do some as I set it up?

On Tue, Aug 20, 2024 at 1:52 AM Juraj Mäsiar @.***> wrote:

Done, new version 25.9 is waiting for review and brings folder specific offline backups in Chrome. In Firefox, it can create backups in the user specified subfolder in the Downloads folder.

The permissions are super strict though (in Chrome). It will ask you to allow access to the folder when you first select it, but then also while making the first backup (after restarting Chrome, week or two later) - at that time it will give you option to always allow access to the folder.

I'm sorry it took me so long, I've been sick for more than a week plus I wanted to fix some issues in this release.

— Reply to this email directly, view it on GitHub https://github.com/fastaddons/GroupSpeedDial/issues/338#issuecomment-2296635863, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQH4K2HIOW4UFBWZLH5PH3LZSH2DHAVCNFSM6AAAAABK3X7YK2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJWGYZTKOBWGM . You are receiving this because you authored the thread.Message ID: @.***>

[Juraj-Masiar]

Well, I'm preparing a post about new features for my Patreon page. And I did wanted to make a short videos for the new features too, I've totally forgot about that. But it should be intuitive enough even without the "manual", I wish I could watch you how you use it, it's always super useful to see how my "intuitive design" is actually a total mess :D.

In any case, the new version is now available for Chrome, if you didn't get it yet, you can force the update: https://github.com/fastaddons/GroupSpeedDial/wiki/How-to-update-to-latest-version

[Goldendawn69]

Just wanted to let you know - already used it to recover and it worked wonderfully.

It is a little bit of a pain that it will come up and ask for permissions again every so often - BUT - that isn't really in your control (sigh security is something we have to live with now days)

Thank you so much for doing this ...!!

On Wed, Aug 21, 2024 at 7:32 PM Juraj Mäsiar @.***> wrote:

Well, I'm preparing a post about new features for my Patreon page https://www.patreon.com/fastaddons. And I did wanted to make a short videos for the new features too, I've totally forgot about that. But it should be intuitive enough even without the "manual", I wish I could watch you how you use it, it's always super useful to see how my "intuitive design" is actually a total mess :D.

In any case, the new version is now available for Chrome, if you didn't get it yet, you can force the update:

https://github.com/fastaddons/GroupSpeedDial/wiki/How-to-update-to-latest-version

— Reply to this email directly, view it on GitHub https://github.com/fastaddons/GroupSpeedDial/issues/338#issuecomment-2301341841, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQH4K2EMZLUYTFTBAMGSWV3ZSQ7BHAVCNFSM6AAAAABK3X7YK2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBRGM2DCOBUGE . You are receiving this because you authored the thread.Message ID: @.***>

[Juraj-Masiar]

Thanks for the feedback! Regarding the security question - it should have asked you during the second backup if you want to "Allow on every visit", but if you clicked "Allow this time", it will not ask you to allow it forever again. At least that's what I've seen in my tests.

[Goldendawn69]

Yep which it appears to work for a long time - but then - suddenly it will lose access again. What I think is that Chrome times it out - not sure why - or updates - etc etc etc

But that is okay - it is just something that comes up and I click it :D That is a LOT better than remembering to do it myself :D

On Mon, Sep 23, 2024 at 11:35 PM Juraj Mäsiar @.***> wrote:

Thanks for the feedback! Regarding the security question - it should have asked you during the second backup if you want to "Allow on every visit", but if you clicked "Allow this time", it will not ask you to allow it forever again. At least that's what I've seen in my tests.

— Reply to this email directly, view it on GitHub https://github.com/fastaddons/GroupSpeedDial/issues/338#issuecomment-2367949449, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQH4K2HJTWJV3IJNJPODIKDZX74HHAVCNFSM6AAAAABK3X7YK2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRXHE2DSNBUHE . You are receiving this because you authored the thread.Message ID: @.***>

[Juraj-Masiar]

Oh, you are right, I've just tested it and Chrome indeed removed the handler. They add all these security features but they don't mention them in the documentation :(

[Goldendawn69]

Yep this is typically is an issue with Chrome (think Firefox is just as bad)

I suspect this might be the cause? [image: image.png] chrome://flags/#deprecate-unload

On Tue, Sep 24, 2024 at 7:00 PM Juraj Mäsiar @.***> wrote:

Oh, you are right, I've just tested it and Chrome indeed removed the handler. They add all these security features but they don't mention them in the documentation :(

— Reply to this email directly, view it on GitHub https://github.com/fastaddons/GroupSpeedDial/issues/338#issuecomment-2370355606, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQH4K2GSZG4IN36SOC2OMBDZYEEYJAVCNFSM6AAAAABK3X7YK2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNZQGM2TKNRQGY . You are receiving this because you authored the thread.Message ID: @.***>

[Juraj-Masiar]

Well, Firefox doesn't support the file system API (due to security concerns), but the Downloads API I've used there as a backup works pretty fine, it doesn't ask anything, the only issue is that now I have a "gsd" folder in my Downloads folder :).

Regarding the flag - that's something else, it's for an "unload" event that fires when you navigate away from a page, at least I think. Also, note that you can't send images through the email reply into the Github issue. This email is just a github notification, if you want to upload image you need to do it on the web: https://github.com/fastaddons/GroupSpeedDial/issues/338