Use a database system rather than json

[ghost]

As the title says: move from JSON over to a database system such as SQLite or MySQL.

When it comes to adding multiple channels which contain a large amount of content it takes a considerable amount of time and increases exponentially as more are added. This time seems to be spent on regenerating then channels.json and it also doesn't seem to be taking advantage of a system's resources as its CPU usage is minimal during this period.

[vexorian]

The json is useful because it allows you to modify the json with scripts and eventually write other prorgrams to generate channels.

I am not really sure if the slow down is because of the json (it being javascript, I think you would see an increase in CPU usage when building the giant json if the size of t he json was the problem), but if that was the case this can be fixed by representing the channels as a folder of jsons instead of a gigantic json.

[ghost]

Separate json files for each channel would be certainly be better than what is implemented right now.

I’d like to add more channels and content but the UI takes forever to list the channels with a json file that’s just over 250MB.

[jasongdove]

As the title says: move from JSON over to a database system such as SQLite or MySQL.

I definitely agree with this. Even some normalization would have a dramatic impact on the JSON size; the current dev version duplicates server details for every program (9.6MB of my 24MB channels.json).

The json is useful because it allows you to modify the json with scripts and eventually write other prorgrams to generate channels.

I don't think changing the data store would prevent scripting or other external data modification.

[Wolfie713]

Like I said in a different change request (maybe on gitlab), I believe it's better to have a reference to the server with the channel/program information vs the server details. If the IP address needs to change for any reason (changing the computer it's on or just changing the IP for personal reasons), then it breaks everything currently. But referring to the server by a consistent unique ID (regardless of IP or computer name or server name) would let everything continue to work, so long as the information for the connected server is updated.

I also believe that using a database (such as an SQL one) will provide many benefits. One table for channels, then another for programs... and if my idea (in another feature req) is implemented, custom programs, blocks, schedules. Would make a lot of the work a bit easier.

[C-Fu]

I think a good compromise would be something like channelnumber.json, eg. 1.json, 2.json, and so on. While mariadb/mysql would be ideal, that would hugely add complexity I think.