Feature/save users to db

[CapCalamity]

This implements a basic storage for Users and Channels.

UserList is now modified to work with Channels, that contain Users, instead of Dictionary<string, HashSet<string>>.

Channels are implemented so that each addition / removal of Users to them is saved directly to the appropriate table in the SQLite database configured in the App.config file.

Issues i'm aware of right now:

• By default, any lists created by EF from the DB use lazy-loading to retrieve the containing data.
  This means that large Channels with large fanbases will take some time to be retrieved.
  I've had channels (the largest ones at the time) take about one minute to retrieve and construct all object from the DB (After short testing: about 10k users retrieved in ~80s).
  This is obviously not ok, but im note sure how we can effectively get around that...
• i think the database cannot be used with external tools
  • Due to this i could not verify the internal structure of the Database myself.
• The secret password for the database is hard coded in the App.config and easily viewable in this repo...

[iPaat]

Thanks a lot for your contribution. I have a few questions:

• On which location will the database will be saved? Can we have that in our isolated Storage together with settings.json?
• If I understood it right this will produce a SQLite file. If I'm not wrong you can browse this file with SQLite Browser
• You use ObservableCollection<User> to store the users within a channel. Can you give me further information why?

By default, any lists created by EF from the DB use lazy-loading to retrieve the containing data. This means that large Channels with large fanbases will take some time to be retrieved. I've had channels (the largest ones at the time) take about one minute to retrieve and construct all object from the DB. This is obviously not ok, but im note sure how we can effectively get around that...

We should get rid of that as fast as we can.

I think we can let the user wait for a minute on application start (big streamers should know that a big database need some time to load (or we start the application and load the user list in background)) but after that, the user list should only be updated every minute within the user list class or with the join/part/message Events. On database side we don't need a live user list. We only need to store every user who was once in the channel.

I think we've had a communication problem. The user list should be something different from the user database.

[CapCalamity]

• Currently, the database is called store.db3 and saved in the working dir - right next to the executable
  Yes i can change that to the Isolated Storage.
• I tried to open the file with that exact tool, but it simply said the file was corrupt...
• The reason i used an ObservableCollection<Users> is to be able to detect when changes are made to the UserList and save the changes to the DB.
  This way i can simply save everything, without keeping track of change-sets and applying them on my own.

I don't think we should remove the immediate update of the DB, because it doesn't cost us anything. We should create some way to let the user know we are currently importing the Channel / Users.

I think we could create a seperate thread on startup that imports the Channels on its own and then triggers some kind of event like OnChannelsLoaded, maybe even trigger it for each loaded Channel seperatley - OnChannelLoaded.

[iPaat]

If I understand you right. The database will be a copy of the current live list of who is online and who not?

[CapCalamity]

Basically, yes.

[CapCalamity]

Storing the database in IsolatedStorage might be a little more complicataed than i initially thought.
Would it be okay if we shift it to the more easily accessible AppData/Local?
IsolatedStorage would be in AppData/Local/IsolatedStorage/{Hashes}/{MoreHashes}/{*EvenMoreHashes*}/.../

[iPaat]

Okay, this is not what I had in mind.
I thought we could have a list of users in the database. Every user who visited once the stream should be stored in the database. Other Plugins & Modules could get that data and work with that.
What I had in mind to store:

• Id
• [a] UserName -> as identifier
• UserDisplayName
• [a] UserStatus -> Nothing|Moderator|Broadcaster
• UserFirstSeen -> Date

Points marked with [a] are provided by the api.

It would be nice to have the possibility for Plugins to add more data to the user. For example how long they had watch the stream and so on.

To get a list of currently online users can be got via API within seconds. I don't know why we could need a database with a current online user list.

[iPaat]

You could get the Isolated Storage with: IsolatedStorageFile.GetStore(IsolatedStorageScope.User | IsolatedStorageScope.Assembly, null, null);

[CapCalamity]

Okay, i'll see how i can change it to accomplish that instead...

The point of IsolatedStorage is to obscure the Paths to the files you want. You don't need the Path to the file to get its contents, just the name.

But, to configure the Database we need a complete filename, including the Path - which we don't get with IsoStorage.

As this doesn't stop the User from viewing / tampering our Data anyway, we can use the simpler AppData just as well. That way we get full Paths, and the App behaves like any other Application out there.

[iPaat]

Okay I get what is your point with isolated storage. Should me move settings.json to AppData/Local too?

[CapCalamity]

I think, yes. As is said, the App would then work like every other application out there. And if users would need to edit settings they would find it more easily

[iPaat]

That seems better to me as the isolated storage.

[CapCalamity]

I've encountered more obstacles than i imagined, i need to restart this with a clean slate... I will close this PR and open new ones for the things that worked