Use local.storage API instead of bookmarks

Riajyuu commented 6 years ago

pros:

possibilities on Android since Firefox for Android doesn't have bookmark APIs.
does not interfere with activity stream & bookmark sync

cons: maybe you need unlimited storage permission and break files into several onto sync

ReDEnergy commented 6 years ago

Thanks for the feedback. Firefox Android and Firefox Desktop share the exactly same API... yes some things (mostly behaviors) might work a little different because on mobile you don't have that much flexibility but other than that the same API and storage model is also available on Firefox mobile.

It's possible to use bookmark APIs on mobile to without any problem. I just have to completely rewrite the addon UI for session management and that's the only reason I was not able to provide a working version on mobile. So even if it's local.storage or bookmarks storage I have the same problem, need a good working UI for interacting with saved sessions.
I don't think bookmarks interfere with activity stream - If I'm not wrong those should only be based on local browsing history.

Regarding interference with bookmarks sync. That's a little bit complicated I would say. I built this addon specifically to do that, to be able to save sessions as bookmarks because I think that It's a great storage model and you have a lot of benefits from doing this. It's not easy to change the whole purpose of the addon. The point is... bookmarked sessions should be only those sessions that you want to have them available forever or for a longer period of time. Some examples:

you want to buy something for your home (but not really right away, maybe in the near feature or whenever you have time/money) and do some research and then saved all that thing as a session
for students it's a great research storage - sometimes you have projects that span over months and all your research links can be saved into these kinds of sessions and whenever you want to go back to them their right there
same for any working scenarios or whatever important links you might want to store for a longer time and not exactly come back to them the following day but even after a few months... they will still be there

That's why I use bookmarks storage. Everything you specifically saved will be there for as long as you need them without worrying that they might be deleted. Once saved as bookmarks you have complete 100% control over them and only you decide when it's time to delete them. Also, the addon is not really required to access them, you can do that using the internal Bookmark Manager. No matter what addons you may use in the future you're bookmarks should be there and even if you change the browser you can import them elsewhere. All browsers already have a method of backup/exporting/importing of boomarks.

Now... auto-saved sessions - those that are stored in the History Tab. These actually use the local.storage API because these are saved automatically and you don't have complete control over them. When the limits (configured by you) are hit the oldest sessions will be automatically deleted.

Of course, you can use whichever you want. I don't really use the auto-saved history at all. Some users might not use the bookmarks saving feature at all but only rely on the auto-saved sessions and so on.

So... in a few words, that's why I don't really want to use local.storage API that much. It's very good for some things, but I consider the bookmarks to be the real critical difference between what other sessions managers do and what mine does.

I'm all for options, configuration, features and I will try to provide as much as possible withing next updates but removing bookmarks storage might definitely not be an option. Maybe a complete new addon, If I ever have time to write it.

Riajyuu commented 6 years ago

@ReDEnergy Thx for your patience in advance. Well, bookmarks will ouccrs in highlights by default even if you use other bookmarks. So requesting local database overid this. Plus, I agree this maybe better for a new addon because I was supposing to keep sessions in different devices synchronized. Thx again for your patience.

rayman89 commented 6 years ago

Isnt indexdb better then localstorage?

ReDEnergy commented 6 years ago

Not really. IndexDB is local to a certain page and not at browser level. I use WebExtensions storage that is guaranteed to be fine even if Firefox crashes while IndexDB or page local storage can be corrupted in such cases, So sadly not really.

RobertAb1 commented 6 years ago

@ReDEnergy Will these bugs be useful to you?

Implement local filesystem read/write access. https://bugzilla.mozilla.org/show_bug.cgi?id=1246236

Use indexedDB as the backend for storage.local https://bugzilla.mozilla.org/show_bug.cgi?id=1406181 https://bugzilla.mozilla.org/show_bug.cgi?id=944918

Convert content-sessionStore to C++ https://bugzilla.mozilla.org/show_bug.cgi?id=1474130

ReDEnergy commented 6 years ago

Hi @RobertAb1, thanks for the resources but sadly I don't plan to make any change to the way the storage works. Bookmarks are supported virtually anywhere and I have no real reason to change things right now (Not even discussing the complexity and challenge of doing that right now. So that's a clear no)

That being said, I do use local.storage where it makes sense. Browser IndexDB is not safe (any non webextensions storage can be corrupted inside an addon so I clearly don't want to use it). LocalStorage will transition to an indexDB backend in a few months but that will be transparent to most addons. There were some concerns that some data may be lost when this transition will be made but at least in this case (SessionSync) it should be safe even on the worst possible case because the main storage is bookmarks.

Right now I kinda wait for this change to see how well the transition will go because If I implement some other critical functionality and that would be lost it will be really bad.

RobertAb1 commented 6 years ago

@ReDEnergy

Probably you are aware that this bug has just been fixed: https://bugzilla.mozilla.org/show_bug.cgi?id=1378647 I am not sure if your extension is using it to restore sessions with lazy-loading.

I have also compiled longer bug list for session managers: https://www.reddit.com/r/firefox/comments/7m8nvx/can_session_manager_tab_session_manager_coexist/drslt43/ Are any Firefox bugs important/useful for your extension are not mentioned on this list?

Happy coding :)

ReDEnergy commented 6 years ago

wow... that's really awesome! Thanks a lot for point out. I had to do a really stupid hack to offer the lazy-loading functionality. I will definitely switch to the internal API with the next update (hope everything goes according to the plan and that a discarded tab still has the data I need)

Thanks for the bug list. Pretty sure many of those apply to this addon as well. Session Sync does not use the internal session-history that Firefox provides... the History Tab I implemented is similar but it's addon specific implementation and has nothing in common with the default Firefox session tracking (various reasons such as the "clear history" functionality)

Sadly what I'm missing the most and probably won't ever happen is the ability to add custom annotations to bookmarks like it was possible prior to WebExtensions. That would've solved so many issues like saving pinned state, mute state as well as any other custom data required for each tab. Sadly I will have to spit the data between bookmarks and something in localStorage (and that's a pretty bad design solution)

There are a little to many bugs to check all of them right now but I will clearly go through all of them at some point. It's really nice to have all of them in the same location.

RobertAb1 commented 6 years ago

(UPDATED)

Other developers had to prepare their own software workarounds with lazy-loading like you did. See example (three the newest videos): https://www.youtube.com/channel/UC6xH2pxt9wJS2XqUOaOfxyA It is simple waste of time. People from Mozilla do not get that. Instead of writing one script by Firefox software engineer before removing XUL addon support, many WE developers had to do this work independently.

You can try to file a bug(s) at Bugzilla for what you need. And let me know about that so I will add your bug(s) to the voting lists.

There is a meta-bug for session managers - all bugs for session managers are in the section "Depends on": https://bugzilla.mozilla.org/show_bug.cgi?id=1427928 Check also fixed bugs (some of them can be found in section "Depends on" of meta-bug.

See discussion here: https://github.com/sienori/Tab-Session-Manager/issues/131 https://bugzilla.mozilla.org/show_bug.cgi?id=1413525 https://bugzilla.mozilla.org/show_bug.cgi?id=1427928

Voting list for session managers is now on 3 pages: https://www.reddit.com/r/firefox/comments/7m8nvx/can_session_manager_tab_session_manager_coexist/drslt43/ https://github.com/sienori/Tab-Session-Manager/issues/150 https://github.com/ReDEnergy/SessionSync/issues/50 I have just created similar thread on your Github profile (the last link). Probably is better to show Mozilla that many session manager developers need these APIs.

See section "About this extension" on these pages: https://addons.mozilla.org/en-US/firefox/addon/tab-session-manager/ https://addons.mozilla.org/en-US/firefox/addon/my-sessions/ https://addons.mozilla.org/en-US/firefox/addon/tab-mix-plus/ https://addons.mozilla.org/en-US/firefox/addon/tab-mix-plus-webextension/ Developers of these extensions added links to voting lists (TMP has longer list with many bugs not related to session managers). Can you do the same? The correct link for session managers would be: https://www.reddit.com/r/firefox/comments/7m8nvx/can_session_manager_tab_session_manager_coexist/drslt43/ or https://github.com/ReDEnergy/SessionSync/issues/50

Riajyuu commented 6 years ago

@ReDEnergy Hi Sorry for my long delay on this. The main reason for my proposal is exactly that bookmarks are interfered with Activity Stream so your addon add some unwanted noise to it. See screenshots below

ReDEnergy commented 6 years ago

Well... I suppose that's a perfectly valid point. Sadly, it's almost impossible to offer such an option right now while removing the bookmark storage would be a really stupid idea (I personally don't want it) because any core feature changes to a product can upset 90% (or 99%) of the user base (I have to assume since I really don't know how users actually use it) and that's why it's hard to change things. The only option for any new feature is to be integrated as an option and if it's something that's generally expected to be also activated. For example I had to decide if I was activating lazy-loading in the previous update. I decided to do so because I thought that most users will prefer faster loading speed and those few that I would not like it to change the option.

Sadly... application storage (in this case addon storage) decisions are usually core aspects of the whole architecture. I kind of built everything around bookmarks because that's what the was the intention. It is possible to offer this bookmark/local-storage replacement? Well... yes but the effort that goes into it makes no sense (at least at this very moment) because there are other features way more important to make the product better.

I can only say that I will consider this option, but I can't promise that I will ever be able to deliver it mostly because I really have very little free time to support the addon and big new features take a lot of careful planing and time.

RobertAb1 commented 6 years ago

@ReDEnergy Have you already checked links I gave you? Was something useful?

ReDEnergy commented 6 years ago

@RobertAb1 Yes I looked over most of them. A few are useful but I would say that because of the way I built the extension most of them are not really. The thing is that Session Sync doesn't rely on any kind of internal Firefox session management features. Everything is build from scratch, so most of those are not really related to the addon unless I add some option to also read/control those as well.

Now... there are a few things that don't really work for these integration (by design)

I don't like the fact that Firefox session management are directly linked to browser history settings
- maybe for some it's perfectly ok... but for some privacy control settings this can really be a pain
bookmark storage should only be used for on-demand storage and not automatic storage since saving might slow down the computer (didn't test that well but I'm pretty sure that storing in localStorage will be faster than the Bookmark system)
- at the same time as I iterated many times, bookmark storage is there to stay because it provides a lot of useful features, but more than anything else because it's part of the identity of the addon itself. Might be strange... but the whole Session Sync naming comes from this fact: you can sync bookmarks
Adding a third session-management tab might be really confusing for a lot of people
There are plenty of addons that already try to do that, I don't really think competition in the same space makes that much sense

I think that Session Sync has more to do with a Bookmark Manager than an actual Session Manager. It's hard to tell how people see it actually. Personally I only use it as a great organizer and less for any kind of classic session management feature (I don't use the auto-saving feature at all). All my sessions are not really intended for restoring, but mostly great snapshots of various cumulative research sessions or well organized resources on various themes. That's why the bookmarking is a perfect fit for this purpose.

I actually implemented this addon only because I really needed this and because no other addon was close to this experience. Later on I found that there are a few people that might consider it useful for them also. Since I'm all for customization I added various features and options to expand the experience, and I will continue to do so (whenever I have time).

A browsing session can represent a lot of things for many people. Some see every window as a separate session, other see all of them as a unique session. Then there are the containers, pinned tabs, hidden tabs. And on top of that there's the history. There are so many features that affect this concept of session that I really find it hard to fit it in one single addon. For me it's what I described above.

Ultimately, I personally think that the integrated Firefox session management should only remain a good alternative for most users but in the end I don't really see how this can be customized and how you can provide fine control options. Maybe I'm missing something but just getting access to the data won't solve customization and any advanced control features that users might look for.

Session Sync may not fix everything, might still be buggy or with some weird implementations (auto-sync certainly is and I plan an update) but it's intended to fit right where it is at this moment. Not yet an all around powerful session management while being a good session bookmark organizer.

My plan for the future SessionSync is simple (yet quite complex to implement well)

expand the control features for a more detailed session-manager (full featured)
allow complex bookmarking controls (more folders or even full bookmark access)

RobertAb1 commented 6 years ago

@ReDEnergy

(1) Gabriel - will fixing any of the mentioned bugs be useful for you? Can you list them? I am just trying better to understand problem.

(2) You wrote here that you need the ability to add custom annotations to bookmarks. Can you create a bug and let me know about it? Bookmark Management was mentioned together with Session Management in one of Mozilla documents. There is no mention in later ones.

(3) I thought that storage.local API is now converted from JSON into IndexedDB storage (Bug 1406181, article in Mozilla Blog). But you are still using JSON storage.local.

(4) Different session/bookmark managers are build in different way. Tab Session Manager (TSM) is completely different. It is relying now on IndexedDB storage.local. However, MySessions seems to use bookmarking system like yours - what is the difference here? MySessions has also autosave option - is MySession using also storage.local? And do you know how Session Boss is working?

(5) I completely agree with you that it is better to have session managers build in different way. If one approach does not work very well, other could better. Or if Mozilla will change something which is important for one WebExtension, other could still work unaffected. But also distinct approaches used in various managers could make it possible to run them simultaneously in one browser. For example, TSM could be run in full capacity in Firefox (especially with large sessions), while Session Sync could only save sessions and have session recovery set to off. When TSM would fail to recover session after browser restart, Session Sync could be used to recover all windows and tabs (also it could be used to save important pages as bookmarks). Is it possible?

(6) Does you extension offers/will offer history for each tab? (Bug 1378651 and Bug 1381922 are related to this problem.) Does Session Sync remember the order of tabs in windows in session saved both in bookmarks storage and local.storage? Does Session Sync remember scrolling positions and state of browser windows (size, position, minimized/active)? Does Session Sync remember data placed into pages?

(7) I am still using Session Manager (also I am saving some important links in Windows folders, so I can access it later in any browser; it is also easier to back up on other HDD). Question: before and after I started using Session Manager, I have been always experiencing Bug 1235231 (see comment 16 for bug description). Is this bug also present when your WebExtension is working?

Cerberus-tm commented 6 years ago

I wish to express my support for using bookmarks to store sessions. Brilliant! It's a great idea, for exactly the reasons specified by @ReDEnergy . Mainly because it provides the highest possible degree of reliability, which is the most important thing for sessions.

I also wish to express my appreciation for @RobertAb1 and his tireless efforts all over the Internet to promote a stable system to save sessions in Firefox à la Session Manager (which I, too, still use). Good work. (I have been voting for many of those Bugzilla issues over the past year or so.)

RobertAb1 commented 6 years ago

@Cerberus-tm Thank you for your support. I hope that session management API will be prepared well.

Cerberus-tm commented 6 years ago

@RobertAb1 I hope so too. Too bad it keeps being postponed. But Session Sync seems to be a pretty good replacement! The main thing I still miss in it is the ability to still use bookmark search from the adress bar (not really practical any more with Session Sync active).

ReDEnergy commented 6 years ago

@Cerberus-tm thanks as well for support @RobertAb1 sorry for not giving a clear answer regarding all those questions. I intended to do so but sincerely it was a lot of explanation to do and I never found the good time to write everything down. The thing is, I intend to improve the addon as much as I can and thus solving those issues may also help me, but at these very moment the addon is more like a complete replacement for the integrated Session Manager or better said it's somewhere in a gray area.

It's not really a Session Manager (but it can behave like one if you want to and accept the obvious limitations)
It's not really a Bookmarking Manager (but it can be a powerful bookmarking tool if you accept the design choices and you learn to use it's features as so)

That's the thing... It's hard to actually put Session Sync into the generic: Session Management category because it clearly has some flaws or missing features.

I totally intend to evolve it, but definitely it will always remain a hybrid version that will try to meet those two categories in the middle: bookmarking and sessions. That's why, I'm sure it will definitely help me to have those bugs fixed as well. How and when will that help me? It's pretty hard to tell because sadly I was quite packed for the last few months and I don't want to make any new promise or create certain expectations. All I can say is that if it's possible I would like to have some integration with the browser session management as well but not only as an alternative or better said as something complementary.

The one thing I can certainly say is that I still have the full intention to improve it because it's a product I really care about and I know that a few other hundred/thousands people may do as well.

RobertAb1 commented 5 years ago

@ReDEnergy

I have now two lists with bugs: long and short: https://github.com/ReDEnergy/SessionSync/issues/50

You could consider to place this link on AMO page. If necessary, I could create additional issue, so issue #50 would be for long bug list, and new issue for short bug list.

Happy New Year 2019!

ReDEnergy / SessionSync

Use local.storage API instead of bookmarks #44