Unified Themes & Repository Cleanup

[baxysquare]

With the recent announcement that Ozone is stepping forward as the default UI for RetroArch, perhaps it's time to develop a unified theme format that would work across Ozone, XMB, Ludo and any other UIs.

It would also go a long way toward sprucing up the repository and give all the UIs an equal foundation to work outward from. It might also be a way to reduce the overall footprint by reducing some of the redundancies in the codebase.

[baxysquare]

@jdgleaver As per our discussion on #353, I have rearranged my theme repository in preparation to move to a universal theme format. I tried to compile some of the the fonts, scripts and sounds from this repository to help flesh things out. I also tried my hand at making rudimentary CFG files, but at this point, they're simply placeholders for now.

From here, I figured we'd take a look at the files in GLUI, Ozone, menu_widgets, Switch and NXRGUI directories to figure out the best way to go about adding their icons as well as renaming the icons that are specific to XMB. My thought is that these themes could be merged with Monochrome into one unified default theme that could serve as the baseline for completing the other themes. What do you think?

[jdgleaver]

You're making excellent progress here!

About the only thing I'm not sure about at the moment is the theme CFG files. I think it might be better to have these in a separate top-level directory - so we'd have automatic.cfg, neoactive.cfg, etc., where the CFG file also includes the relevant icons directory and sounds files. From a code implementation perspective, I think that's cleaner - i.e. we don't mess around with having to find directories based on the theme name any more; we just select a single CFG file and everything we need is inside.

I think you're right that the current GLUI, Ozone and menuwidgets icons can become part of the monochrome icon set - that's certainly most appropriate. Prepending each menu-specific icon with the driver name (or `widget` in the case of menu_widgets) does seem like the way to go.

But one thing that's just occurred to me - while the source files can certainly be merged, I think there should be some division when it comes to generating the output PNG files. Basically, if the resultant icons all get dumped in the same folder, then it means I have to install everything regardless of which driver I want to use. e.g. GLUI is my driver of choice - I shouldn't have to keep a folder full of the various 'system' icons just to make it work, when it only needs a handful of menu icons + widget stuff. I don't think this hinders the unification effort though. This would just happen at the icon generation stage - the build scripts would simply loop through and package appropriate bundles. So you could have a top level output icon theme directory with common, xmb, ozone, and glui subdirectories (each with the different icon size subdirectories). Actually, this could even be a separate post-processing stage - i.e. all the icons are generated as you have them already, but then we have a packaging script that copies them to a structured output directory (and maybe zips each theme by icon size, for downloading via the online updater).

I guess that's probably quite a bit to think about...

[baxysquare]

So I went ahead and moved the CFG files to a themes folder at the root level. I also started spitballing on the code for the CFG file itself. My original intent was to split the icons by resolution so it could be determined by build which icons would end up in a given platform so they had only what they needed but perhaps could have the option to download other sets in the online updater.

I see where you're going with splitting the icons into subdirectories. Whether that split should happen at /png/ or within the resolution directories is up to the larger group to decide what would work best. For example, if GLUI only had 256 pixel graphics, all the CFG values would point to /uni/icons/themename/png/256/glui/.

font_default = /uni/fonts/font-Regular.ttf
font_bold = /uni/fonts/font-Bold.ttf

240p_background = /uni/backgrounds/png/themename_sd.png
240p_common = /uni/icons/themename/png/64/common/
240p_xmb = /uni/icons/themename/png/64/xmb/
240p_ozone = /uni/icons/themename/png/64/ozone/
240p_glui =  /uni/icons/themename/png/64/glui/

480p_background = /uni/backgrounds/png/themename_sd.png
480p_common = /uni/icons/themename/png/128/common/
480p_xmb = /uni/icons/themename/png/128/xmb/
480p_ozone = /uni/icons/themename/png/128/ozone/
480p_glui =  /uni/icons/themename/png/128/glui/"

1080p_background = /uni/backgrounds/png/themename_hd.png
1080p_common = /uni/icons/themename/png/256/common/
1080p_xmb = /uni/icons/themename/png/256/xmb/
1080p_ozone = /uni/icons/themename/png/256/ozone/
1080p_glui =  /uni/icons/themename/png/256/glui/

2160p_background = /uni/backgrounds/png/themename_4k.png
2160p_common = /uni/icons/themename/png/512/common/
2160p_xmb = /uni/icons/themename/png/512/xmb/
2160p_ozone = /uni/icons/themename/png/512/ozone/
2160p_glui =  /uni/icons/themename/png/512/glui/

4320p_background = /uni/backgrounds/png/themename_8k.png
4320p_common = /uni/icons/themename/png/1024/common/
4320p_xmb = /uni/icons/themename/png/1024/xmb/
4320p_ozone = /uni/icons/themename/png/1024/ozone/
4320p_glui =  /uni/icons/themename/png/1024/glui/ 

[jdgleaver]

This is good stuff, and it's helped to clarify my thoughts a little :)

I reckon the source files (including /png/) should be organised as you already have them, and the menu driver/widget split should be at the final packaging stage. We can of course discuss this with the group as things progress, but I think it's a nice approach - it means you can carry on as you like, and we'll just need to write some appropriate packaging scripts once the source files are ready.

Regarding the split by resolution - again, for the source files your approach is perfect. For the packaging (and CFG files), I think there would be value in having the actual 'bundles' zipped by resolution - so I mean you'd have something like:

automatic_1080p.zip
`- automatic/
   |- backgrounds/
   `- icons/
      |- common/
      |- xmb/
      |- ozone/
      `- glui

...and a separate bundle for 720p, 2160p, etc., where the contents always has the same layout and names.

The benefit here is that we can have an assets submenu in the online updater with entries for each resolution, and when the user selects one it overwrites what is currently installed. This means we never have redundant, unwanted images wasting disk space - there is always exactly one set installed locally. So we pick a best guess resolution during the build process, and then the user can override this cleanly.

The CFG files would then lose the resolution info, and we'd just have something like:

# All paths specified relative to the user's local assets directory

font_default = fonts/font-Regular.ttf
font_bold = fonts/font-Bold.ttf

background = themename/backgrounds/themename.png
icons_common = themename/icons/common/
icons_xmb = themename/icons/xmb/
icons_ozone = themename/icons/ozone/
icons_glui =  themename/icons/glui/

(Maybe fonts should be bundled on a per-theme basis as well - I guess it depends how we want to handle the assets download)

What do you think...?

[baxysquare]

Sounds like we're making progress. If you (and the group) feel like the arrangement of source files is great, then we'll use that as the foundation. I'm also flexible with how we arrange source files, so long as it continues to eliminate redundancy while optimizing the available resources. Since the packaging will be handled by the CFG and scripts, it gives us maximum flexibility, plus it really should improve the overall end-user experience.

Just for reference and not just to be contradictory, I decided to try duplicating and reorganizing the uni directory and create a bytheme structure that would be more inline with what you're proposing, but applied to the overall file structure.

For this experiment, theme directories are capitalized and theme-specific items would always be contained within their folder. The commonly shared items such as fonts, scripts, sounds and configuration files are kept in lower case directories. That way we could still minimize duplication for items that are shared by themes. My favorite example of this is the mplus-1p-regular font. Right now, it's duplicated in multiple themes and at 1.6MB an instance. That adds up quickly in the current paradigm.

[jdgleaver]

Yes, I like where you're going with this!

The bytheme layout is very much what I had in mind. You could even use that as the basis for the whole 'unified' repository - something like:

bytheme/
`- theme_name/
   `- icons/
      |- common/
      |  |- 1024/
      |  |- 128/
      |  |- 256/
      |  |- 512/
      |  `- 64/
      |- glui/
      |  |- 1024/
      |  |- 128/
      |  |- 256/
      |  |- 512/
      |  `- 64/
      |- ozone/
      |  |- 1024/
      |  |- 128/
      |  |- 256/
      |  |- 512/
      |  `- 64/
      |- src
      |  |- pdf/
      |  |- png/ <- this could be for glui source files, where we only have png images
      |  `- svg/
      `- xmb/
         |- 1024/
         |- 128/
         |- 256/
         |- 512/
         `- 64/

But of course, since the final layout will be a packaging task, I think you can arrange the source files in the way you think is best. I'll probably be the one who ends up implementing the code to handle all this, and I too am quite flexible :)

I still have a fair chunk of stuff on my immediate TODO list to get through - it looks like the repo will be in good shape by the time I'm ready to start work on theme refactors.

[baxysquare]

@jdgleaver If you're the one handling the code and the bytheme arrangement makes the most sense to you, then this is the way!

As a note, the /src/pdf files are exclusive to baxy-retroarch-themes as I use the Creative Cloud to create and maintain the icons and backgrounds. Since the preferred source format is SVG, I don't include the PDF files to minimize the bloat. Trust me, nobody wants 1.5GB of PDF files hosted here!

There are a few themes and UIs that only have PNG sources, but long-term, I'd personally like to see SVG sources be required in order to be considered an official universal theme. I know that's a hard-line stance, but if all themes had SVG source files, it would make them much easier to create the PNG files in any needed size by batch script. Over half of the XMB themes already have a 1 to 1 SVG source, so the trick would be making sure the remaining XMB and UI themes are on board.

So along that line, I'm willing to create SVG source files for GLUI and any other mainline UI elements that require that work. That work would serve as the basis for making sure Monochrome is the reference theme and give the other themes a base reference framework to work out from.

In terms of the TODO list, I just wanted to have it ready for whenever you're ready. I've been focused on making sure there's a complete common set between Monochrome and the Active/Matic themes and making sure everything is synced with the Libretro DB listing and the DAT-O-Matic pulldown to help future-proof the repository.

If there's anything else I can do to get things ready, please let me know and I will add it to my list!

[jdgleaver]

Wow, this is excellent! Really glad to have you driving this :)

There are a few themes and UIs that only have PNG sources, but long-term, I'd personally like to see SVG sources be required in order to be considered an official universal theme. I know that's a hard-line stance, but if all themes had SVG source files, it would make them much easier to create the PNG files in any needed size by batch script. Over half of the XMB themes already have a 1 to 1 SVG source, so the trick would be making sure the remaining XMB and UI themes are on board.

It might be hard line, but this is an excellent stance. I've already had a taste of trying to reformat existing PNG files (when nothing else was available), and quite frankly it's rubbish. Having SVG everywhere makes everything so much easier to manage.

So along that line, I'm willing to create SVG source files for GLUI and any other mainline UI elements that require that work. That work would serve as the basis for making sure Monochrome is the reference theme and give the other themes a base reference framework to work out from.

Absolutely splendid, and thank you for taking this on!

For GLUI I hope this actually shouldn't be too hard. Virtually all the existing GLUI icons are official images from Google (https://material.io/resources/icons/) which are already available in SVG format - the only difference is that our images are pure white, whereas the official ones are tinted (and a handful are reduced in size, with a little padding). I think the only truly 'custom' ones are:

• cores.png
• key.png
• key-hover.png
• pointer.png (but this is a common icon)
• switch_bg.png
• switch_off.png
• switch_on.png

...and I already have SVG copies of the 'switch_bg/_on/_off' icons.

In terms of the TODO list, I just wanted to have it ready for whenever you're ready. I've been focused on making sure there's a complete common set between Monochrome and the Active/Matic themes and making sure everything is synced with the Libretro DB listing and the DAT-O-Matic pulldown to help future-proof the repository.

Perfect!

If there's anything else I can do to get things ready, please let me know and I will add it to my list!

I think you're already doing more than enough :)

You have this well in order. With good source files we should be ready for anything. I'll let you know when my TODO list has cleared a little, and we can start thinking about final packaging and integration with the code :)

[baxysquare]

I've decided to go all-in on getting files arranged using the bytheme structure by rolling the remnants of the original uni directory into the mix. I hope I haven't jumped the gun, but it seemed like the right thing to do in order to minimize duplicates and move forward.

For now, each theme file has a src directory that potentially needs some work. They each have a backgrounds folder, but they also have a PDF, PNG and SVG directory for the icon sets. In the /Monochrome/src folder I've added a glui-pdf and glui-svg folder to keep the GLUI source files separate. I'm not sure if these are the best locations and philosophies for ordering the source files, but this is a WIP and is subject to change, right?

As you can see in the above reference, I also added the GLUI SVG files to the main repository. A few of them are not perfect 1 to 1 recreations, but they can at least serve as a foundation for corrections.

[jdgleaver]

Wow, this is great :+1:

I love what you're doing here. Now I've seen the layout with those glui source files, though, I'm wondering about the best way to handle these 'special cases' :)

I guess there are two options:

• Put all the source files in e.g. src/svg/ and differentiate between them by a naming prefix (i.e. all glui icons are named glui_<icon_name>.svg)

• Split src into subdirectories matching the output icons:

bytheme/
`- theme_name/
   `- src/
      |- common/
      |- glui/
      |- ozone/
      |- widgets/
      `- xmb/

I guess I'm not sure which I prefer... If I were creating/editing icons, I think separate subdirectories would be easier to manage. From a packaging standpoint, it probably doesn't matter (a packaging script can go by directory or prefix just as easily).

What do you think?

BTW - great job with the glui svg files! Much appreciated. I'll go through and compare them with the originals later, but from a quick glance they look splendid :)

[baxysquare]

Wow that's a tough call. On one hand, I'm a big fan of the prefix method where all the files are one folder. But consistency is important, so I can see the wisdom of arranging them in subdirectories to match the output. I'm guessing the latter would probably make the most sense to veteran and rookie theme artists alike.

[Jaffacakelover]

To clarify, #391 is me adding a bunch of core icons, but finding it impossible to run the optimization shell script that lives in retroarch-assets/xmb (using Windows via MSYS2 or an Ubuntu VM) and a subsequent discussion regarding the script. If this becomes a mandatory step rather than 'nice to have' in future, I feel it should be platform agnostic (maybe even a batch process).

[baxysquare]

@Jaffacakelover I believe the long-term plan is for the PNG creation is to be integrated into the build process. That would mean all an author would need to do is upload the SVGs for the universal theme. From there, the PNG files will be generated and distributed by platform. That will help us pair down the size of the repository and target platforms based on resolution need. For example, the set of icons needed for the Vita may be 64 or 128, while the icons that make it to PS4 will be 256 or 512 and those platforms can be setup so they'd only get the PNG files they need. The preliminary spec is to be able to generate icons between 64 and 1024, covering the smallest handheld screen all the way up to 8K TVs and even beyond.

By moving that process into the build, it will make an author's need to batch convert and/or create PNG files obsolete. The build bot should be able to handle that using Inkscape, ImageMagick or whatever is chosen to handle the process on the server side of things. It will be platform agnostic for the theme author, because there won't be a need to run the script on the desktop side to create and optimize the PNGs. The only requirement to make this happen is SVG sources would have to be required in order to be considered an official universal theme.

Now the tradeoff to this process is the author won't be able to see how the PNGs render until after the builds are released. But I feel that's a small price to pay in order to no longer have to batch convert the files on the desktop.

@jdgleaver This is my understanding of what's being planned to implement the universal themes and cleanup the repository. If it's going to be a different workflow than this, I would invite you to help us better understand how you see the process being implemented.

[jdgleaver]

@baxysquare Yes, you are 100% correct.

The idea is indeed that everything should be in SVG format (excluding the obvious edge case of RGUI wallpapers), and that is the only thing theme authors should worry about - all conversion/optimisation will end up happening via the new build infrastructure. We will still have conversion scripts in this repo, but it will be the packager's job to maintain these - artists shouldn't have to concern themselves with technical details like this, or be burdened with the effort of having to set up their own build environments.

Getting the build process configured on our side is very much on the TODO list - it's just that we're a little overwhelmed at the moment (I've been working full time non-stop on the new infrastructure for well over a month now; we're getting there but it's quite exhausting...)

[baxysquare]

I'd really like to start laying down the foundation for unified themes without impacting the existing directories, files and dependencies. I'm wondering if we should start by creating some of the new common directories. Here's what I propose:

Add the following common directories at the root level:

• sounds
• fonts
• cfg
• scripts
• devtools

From there we can start moving/duplicating the items that will be permanently transitioning to those directories. In the short term, it may increase the size of the repository. But the intent would be to push things forward in preparation for the changes that need to be made to the codebase in other repositories. What do you think?

[hizzlekizzle]

Seems like a good idea to me.

[jdgleaver]

Sorry, I somehow missed this message earlier in the seek...

Yes, I would certainly agree with starting to reorganise the repo like this. I don't think the size of the repo is much of an issue (it won't affect actual download sizes for users), and duplicating files means there will be no disruption to current operations.

[fpscan]

Can we close this issue now?

[baxysquare]

I would prefer to keep it open until @jdgleaver is able to fully implement Unified Themes as outlined here. Adding the directories at #420 is the first major step. Now that it has been merged, we can start moving items into their new locations.

We don't want to break anything as we transition from the old structure to the new one. I believe the next step is to start moving key files from the src directory. In terms of src/xmb, I believe that all of these files can be migrated without breaking anything. But for the other UIs, I'm not quite sure if we can move anything from the src directory. The files in root xmb directory must stay in place until the full transition is complete. If anyone can confirm if there are dependencies before things get shuffled into their new directories, that would be great.

Incidentally, this transition seems like a good time to retire the RetroActive and NeoActive themes. I plan to replace them both with "FlatUX". This will be the first theme that I will be contributing to the new theme structure directories as my next pull release, and I'm hoping it will debut when Unified Themes make their public debut. It is also intended to serve as the foundation for a merge and foundational completion of FlatUI, but I wish for the community to make this decision.

[baxysquare]

@hizzlekizzle I'd like to get your thoughts on all of this, relative to the discussion happening on #461. It looks like I created the sounds directory back in #420 to pave the way for eventual removal from src. However, if there's no desire to implement universal themes, I'm happy to transition the repository back to the the way it was in order to clean things up.