Library cloud sync - Githubissues

janforster commented 6 years ago

Hi,

It would be nice to have a built-in functionality to save config files and profiles into a cloud platform (google drive/dropbox...) for storage and backup across several machines. Right now I am solving this by using the portable installation which I have (alongside the db file) saved in my google drive, but it would be convenient to have this functionality already out-of-the-box in the app.

Cheers

JosefNemec commented 6 years ago

Are you talking about save files for games or Playnite files?

janforster commented 6 years ago

I mean the Playnite files: For example, I want to make sure that the way I categorised my games in Playnite is the same across all of the PCs I use.

JosefNemec commented 6 years ago

Couldn't you just save Playnite's database file to your Dropbox folder? Other files can stay in default paths (even when using installer and not portable version) and everything should work just fine.

janforster commented 6 years ago

During the resolution of #181 I figured why is the cloud backup relevant besides holding the database: I need to be able to sync also my categorisation of games - Which does not seem to be part of the database as I lost all of it while moving back to 2.4.

JosefNemec commented 6 years ago

Categories are definitely part of the database, that looks like some issue related to #181 crash.

janforster commented 6 years ago

As discussed on Discord: It would be nice to have a native cloud support (i.e. Playnite requests access to rights to the cloud storage and pulls/pushes updates there on its own). This should solve the current issue with locking of the database while Playnite is running from a cloud folder.

tylerszabo commented 5 years ago

Cloud Sync that's meaningful cross-device would likely need some tweaking or restructuring of the database schema. Especially when considering installation-specific information.

I'm not sure a "game" and a "library item" are necessarily the same thing all the time. Different data in a game have different loci:

Title (Global)

Information that doesn't depend on editions or vendor

Critic score, time to beat, description, additional resources (wiki, Forum), etc.

Store-SKU (Global)

A particular SKU in a particular game store. For example: Wasteland 2 (Steam), Wasteland 2: Directors Cut (Steam), Wasteland 2: Director's Cut (GoG) are all distinct store-SKUs despite all potentially being considered the same game, "Wasteland 2".

Specific Title, Platform, Publisher, etc.

Installation (Personal)

There may be more than one on a single machine (multiple copies of a game with different configs), may exist on multiple machines, may not exist at all (uninstalled everywhere)

Install directory, Launch Actions, Installed/Running states, etc.

"Concept" (Personal)

A personal concept of a particular "game". It's possible any particular person would want to think of a game in terms of its particular Store-SKU or Title, or Installation. It may even be a cross-platform idea - one might want to be able to count how much time was spent playing Minecraft regardless of whether they played it on a console or on their PC.

Play time, Last played, Play count, Category, etc

Each library is inherently personal but the "Global" information can be initially populated from shared databases (such is IGDB or the stores) and conceptually lives globally.

It could also make sense to consider this part of a larger idea where things like play time are shown potentially as aggregations (local counter + game store A's counter + game store B's counter) or Store-SKUs can be grouped into a single title, or multiple installs treated independently.

poperigby commented 5 years ago

Could Joplin's code help? It's a note taking application with good syncing capabilities. The syncing feature could be a useful reference.

JosefNemec commented 4 years ago

Is this still needed? We no longer prevent access to library files when application is running (since Playnite 4), so 3rd party sync tools shouldn't have any issues syncing Playnite's files.

I honestly don't want to add this as native feature because supporting all these various services would be a ton of work on something that could be achieved by simply using desktop tools that come with those services.

janforster commented 4 years ago

I think we still need this. Syncing using portable files is not very reliable. I am running it that way since I started using playnite and it happens to me at least once every two weeks that the file gets corrupted and I have to redo everything from a backup.

This is actually the main reason I switched to GOG Galaxy 2, where sync works flawlessly.

JosefNemec commented 4 years ago

If the sync gets corrupted then you should probably switch to a different cloud provider. I'm using OneDrive and I haven't notice any issues.

The way Galaxy 2 does it comes with a lot of limitations and requires bespoke backed service to run. If I was to do it in the similar way I would have to charge for it because I don't have a resources to run something like that for free. And the alternative of doing it via existing cloud services (OneDrive, Google Drive, Dropbox etc.), would be very time consuming and hard to maintain.

tylerszabo commented 4 years ago

I think there's one part of this that is still relevant - the metadata that's related to the specific machine (launch action, installation directory, installed/uninstalled state) causes issues when syncing across machines with different games / install locations.

JosefNemec commented 4 years ago

I'll think about it some more, but I'm not sure how to implement this. I would have to completely rework (again) how we store library data and separate machine specific information somehow.

More that I think about it, I'm not sure I will be able to do this using some existing cloud solutions (even if we directly support them) and will have to implement our own library/account/profile service. But again, that would be so much effort and would require a lot of resources, that could be probably spent on more smaller features. You have to keep in mind that Playnite is still mainly developed and supported by a single person.

I'm really torn on this issue. On one point it would be really cool to have some online profile that would show your Playnite library and allowed you to sync that between machines, but then I realize how complex is that to implement...

tylerszabo commented 4 years ago

Every time I think about this problem it gets complicated with edge-cases and even then it would certainly require non-trivial changes. Perhaps another layer of abstraction; a database normalization, or both and possibly a migration would be needed (and that would create its own pain).

It's possible that eventually it'll be a moot point: for instance a slow migration to metadata provider plugins might make each attribute primarily sourced by a plugin such that the core DB really only handles attributes related to the local system. Or perhaps there's possibility to include and commit some data by reference when reading from and writing to the library.

This seems like something that should sit in backlog to be considered for future decisions while focus is made elsewhere. I'd mostly urge you not to close this but to let it sit as a V-next until it makes sense.

doublah commented 4 years ago

Wonder if there's some kind of library that would make this easier to implement and not put maintainership burden on you.

nathanmerrill commented 4 years ago

I don't care about cloud sync, but I do care about computer-specific entries. Notably, I am syncing my library, and the location of a bunch of games are different on the two computers. The only workaround is to force the paths to be the same.

Perhaps this issue can be split up? The first part would be to separate data by computer, the second is the sync. I think both of those tasks are still quite complex, but perhaps not quite as daunting taken individually.

JosefNemec commented 4 years ago

It's not a bad idea to separate device specific data. But still, it would be a fairly complex thing to implement.