Support non-Solarized themes?

[alphapapa]

@jgkamat @emraher

Hi there,

It's really cool that two people have now used this repo as a base to support other color themes! :)

I was thinking, it would probably be fairly easy to add support for other color themes besides Solarized so they could all be supported from a single repo and build process. That way we could all share the work keeping the site files up-to-date.

Of course, that might mean essentially starting a new repo. Maybe it should be called everything-everything-css? Haha.

Of course, there might occasionally be some minor conflicts about which colors to use. That is, all of the sites/*.styl files would have to be the same, regardless of the color theme, but the color names would be mapped to different color values depending on which color theme file was being used (we'd probably reorganize the files into themes/{solarized,darculized,dracula} directories). Most of the time this would be fine, but occasionally there might be a page element that needed tweaking to look good with a certain theme, and that would be hard or impossible to do.

But it might be worth that small sacrifice if it reduced the maintenance burden.

What do you all think? Would you be interested in this, or would you prefer to keep your variants separate?

Thanks.

[neeasade]

FWIW I also have a personal fork where I template xresources into the styl colors to match my terminal colorscheme -- great project, thanks.

[jgkamat]

Hey @alphapapa!

I think that's a great idea! It would allow people to both add new themes by just adding a couple files, and allow them to switch between themes painlessly (eg: there would be both a solarized-all-sites.css and a darculized-all-sites.css file people can switch between). I would be willing to put in work to make that happen as well! :)

Most of the time this would be fine, but occasionally there might be a page element that needed tweaking to look good with a certain theme, and that would be hard or impossible to do.

That's true. I think that if this was a big enough problem, we could split one color into two, (the color that was the cause of the problem) and use the secondary color on the broken sites. I don't anticipate it being a big problem though.

I'm a little bit worried about the possibility for a TON of css to be generated (since there's a CSS target for every new theme added). Maybe we should set it up in a way that the final CSS is built onto a seperate branch to make PR's easier and reduce clutter? That is possible to do automatically with CI, but I'm still not sure how good of an idea that is.

I probably would still maintain my own fork, but for different reasons (to remove the bigblow stylesheet, to add a few weird quirks not suitable for everyone, eg), but overall, I think this is a great idea!

One more concern though, since this could become something really great in the future (with many contributors) maybe it would make sense to add a proper license (so people can't claim their contributions were proprietary or something like that). I'm fine with anything GPL compatible fwiw (or creative commons).

Let me know what you think! :tada:

[The-Compiler]

I'm a little bit worried about the possibility for a TON of css to be generated (since there's a CSS target for every new theme added). Maybe we should set it up in a way that the final CSS is built onto a seperate branch to make PR's easier and reduce clutter? That is possible to do automatically with CI, but I'm still not sure how good of an idea that is.

Yeah, the generated CSS in the repo has always bothered me a bit as well. FWIW, I'd be fine with the repo being source only, and uploading .css files as GItHub releases or so once in a while.

One more concern though, since this could become something really great in the future (with many contributors) maybe it would make sense to add a proper license (so people can't claim their contributions were proprietary or something like that). I'm fine with anything GPL compatible fwiw (or creative commons).

Yeah, 👍. I'm fine with licensing all my contributions under any free license as per DFSG, but I'd prefer GPL or MIT.

[cal2195]

I've just been working on the gruvbox color scheme based of this repo! It looks great in qutebrowser! :D

I'd love to share it here, whether it be a branch, or a separate repo! :)

[alphapapa]

Hey guys,

I think that's a great idea! It would allow people to both add new themes by just adding a couple files, and allow them to switch between themes painlessly (eg: there would be both a solarized-all-sites.css and a darculized-all-sites.css file people can switch between). I would be willing to put in work to make that happen as well! :)

Great!

That's true. I think that if this was a big enough problem, we could split one color into two, (the color that was the cause of the problem) and use the secondary color on the broken sites. I don't anticipate it being a big problem though.

Sounds good to me. We can cross that bridge when we come to it.

I'm a little bit worried about the possibility for a TON of css to be generated (since there's a CSS target for every new theme added). Maybe we should set it up in a way that the final CSS is built onto a seperate branch to make PR's easier and reduce clutter? That is possible to do automatically with CI, but I'm still not sure how good of an idea that is.

Yeah, the generated CSS in the repo has always bothered me a bit as well. FWIW, I'd be fine with the repo being source only, and uploading .css files as GItHub releases or so once in a while.

Distributing the CSS separately might be an option, but I have two concerns:

1. Keeping the CSS in the repo in sync with the styl files makes it easier to verify the generated CSS by looking at the diffs. For example, if I had paid more attention to the CSS diffs myself, I would have realized that the bigblow sheet added things to the generated CSS that basically ruined the all-sites CSS files. Then when I removed the bigblow sheet and regenerated the CSS, the diffs verified that the problematic parts were removed. If we were to distribute the CSS separately and not keep them in git, it would make this harder.

   This isn't necessarily a deal-breaker, but it is a concern. What do you all think?

2. I don't want to complicate the "release process" (as it were). The current process is to run make, git add ., git commit -a (although I use Magit). If we used GitHub releases for just CSS files, we'd have to manually collect and upload files, etc. That doesn't sound fun to me. :) If we could automate this, I think I'd be okay with it.

   On the other hand, if we consider external web sites part of the "release process," then it's already worse than that. e.g. I haven't updated the Solarized sheets on userstyles.org for a while now. If we could automate all of this, that would be amazing. :)

Yeah, :+1:. I'm fine with licensing all my contributions under any free license as per DFSG, but I'd prefer GPL or MIT.

Yep, big fan of DFSG here. :) GPL is fine with me.

I've just been working on the gruvbox color scheme based of this repo! It looks great in qutebrowser! :D I'd love to share it here, whether it be a branch, or a separate repo! :)

Great! Please keep an eye on this issue, and hopefully we'll have something for you soon. In the meantime, feel free to link your repo. :)

[cal2195]

Another option would be a directory for each theme, allowing for a nice readme for each theme with screenshots like the current one - keeping the sites/* in sync might be annoying, though it could solve the conflict issue?

[alphapapa]

Copying the sites directory for each color theme isn't necessary. All that's necessary is to use a different set of color definitions.

[cal2195]

I meant more for organisational purposes, so each theme would have it's own set of directories. Sharing the sites makes sense, especially if there was some way to override if needed for a theme.

[jgkamat]

Keeping the CSS in the repo in sync with the styl files makes it easier to verify the generated CSS by looking at the diffs. For example, if I had paid more attention to the CSS diffs myself, I would have realized that the bigblow sheet added things to the generated CSS that basically ruined the all-sites CSS files. Then when I removed the bigblow sheet and regenerated the CSS, the diffs verified that the problematic parts were removed. If we were to distribute the CSS separately and not keep them in git, it would make this harder.

I agree with this, but I think that if, say, we had 10 themes, we would have 10 copies of the css diff to comb through for every PR. Maybe we could only generate solarized by default, and update the rest of the themes during a 'release' somehow?

I don't want to complicate the "release process" (as it were). The current process is to run make, git add ., git commit -a (although I use Magit). If we used GitHub releases for just CSS files, we'd have to manually collect and upload files, etc. That doesn't sound fun to me. :) If we could automate this, I think I'd be okay with it.

I have two possible ways to address this.

1. The way I usually do this is by using a CI service (travis or circleci) to build every commit, and push the results to a seperate branch. For example, my website has org files in it's default branch, and generated html files in the gh-pages branch (pushed from CI). This has the downside as to being a little bit difficult to verify the CSS per the previous comment (unless you build it locally every time), but a seperate branch would automatically be updated with the result from building master at all times.
2. Write a script to build a release, and use the github API to push it up (making it easy to author a new release).

I've done a lot of stuff with CI and a little with the github API, so I would be happy to set it up if we decide to go one of these ways.

[alphapapa]

I agree with this, but I think that if, say, we had 10 themes, we would have 10 copies of the css diff to comb through for every PR. Maybe we could only generate solarized by default, and update the rest of the themes during a 'release' somehow?

That's a good point. I guess we can drop that idea. We can dig through the CSS diffs if we really need to.

I've done a lot of stuff with CI and a little with the github API, so I would be happy to set it up if we decide to go one of these ways.

IIUC, having the CI service push to a branch would mean somehow giving permission to the CI container/instance/script to push to a branch. I don't know exactly how that would work, but in general I don't think I like that idea. I'd rather have each developer be able to push a release from their local machine with the API. So if you could set that up, that would be great!

I'm trying to rewrite the Makefile to support multiple themes, but man, Makefiles... :( It's a wonder I got it working in the first place.

[alphapapa]

Well, I've gotten the Makefile to build the CSS files with the modified directory structure, but it only builds them when they don't already exist, which also means that it doesn't track dependencies on the styl files.

I've gone around in circles for several hours trying to make sense of the make manual, googling, etc, but I can't figure it out. The key seems to be using the eval function as a macro to write rules for the CSS files into the Makefile at runtime, with each CSS file depending on the appropriate styl files. But I can't get eval to work at all, and there's no way to see what the Makefile is trying to do with it, even with make --debug=a -rpn. The part I need to see inside is like a black box. The solution is so simple in theory, but seems impossible with make.

We "just" need to make rules like this, one for each CSS file:

$(CSS_DIR)/solarized-dark/github.css: themes/solarized-dark.styl sites/github.styl $(COMMON_FILES)
    $(call make_css,$(call get_part,$@,1),$(call get_part,$@,2))

I would hate to have to generate the Makefile with a script, or use a different build system, but that may be the only option.

If any of you have experience with Makefiles and can figure out how to use eval to write these rules, please feel free. I've had about enough of make. :|

[alphapapa]

I wrote a shell script, argh.sh, to generate a makefile, Makefile2. You can run:

$ argh.sh
$ make -f Makefile2

And it works correctly, with dependency.

But I don't like having to generate the makefile with a script every time we add a theme or site. It's definitely possible to do this completely in the makefile, I guess I'm just not smart enough to figure it out. If any of you can, please do. Thanks.

[alphapapa]

Ok, I wrote a Python script to do it, make.py, and it uses multiprocessing to build in parallel. It checks dependency mtimes to avoid building everything when it's not needed. Seems to work. It's not as concise as the Makefile, and it hard-codes the logic, but it's actually comprehensible, haha. What do you all think?

https://github.com/alphapapa/solarized-everything-css/blob/multi-theme/make.py

[jgkamat]

I think a python makefile is fine, this is a sufficiently complex setup that make would be super painful to get working.

Only comment I have would be it's probably a good idea to make the makefile simply call the python script, so it will work when people run make mindlessly :P.

[alphapapa]

Yep, I will do that, thanks.

Ok, with that done, I think we can easily drop in other theme files and have all the CSS built. Would anyone like to give it a try with theirs?

[alphapapa]

@jgkamat @cal2195

FYI, I moved the per-theme color files into the theme directories. colors.styl is now a special file in each theme's directory. Any other files in the theme's directory are considered a variant of the theme. If there's only one variant, the variant name will be omitted from the CSS files.

I'll need to document this later...

[cal2195]

Looks much neater! :D What are we thinking for screenshots of each theme? A readme for each theme would be nice, maybe linked from the main README.md?

[alphapapa]

Good point, I hadn't thought about screenshots. We could continue putting them in the screenshots directory, or we could put them in the theme subdirs. I think I'd prefer to put them in the screenshots directory, to keep the theme subdirs code-only.

How about this: we make a subdir in the screenshots directory for each theme, and we put the screenshots and theme-specific readme in each one, and link to them from the main readme?

If we want to get really fancy, we could automate the screenshots using a headless browser.

[cal2195]

That sounds good to me! :) It'd be nice to have the screenshots part of the make file, and possibly even automating the whole git commit process - make css files, update screenshots and then commit - keeps the css up to date if we keep it in the repo! :)

[alphapapa]

That would indeed be very cool.

Notes:

1. We would need a way to link each file in sites to a domain, so e.g. we might need to rename github.styl to github.com.styl.
2. If we do automate screenshots, we should also put pngcrush in the pipeline.

[jgkamat]

One of the concerns I have with this is if screenshots are updated frequently it will lead to a bunch of binary data in git, which isn't compressed well at all (and it will be updated every so often). Maybe we could upload it to a external image store to get around this (although automating that would be tricky).

[alphapapa]

Yes, that's a good point. Automatically generating screenshots should be a separate make target that's only run when necessary.

[blu3r4d0n]

Is this still an active idea? I'd love to help in anyway I could.

[jgkamat]

@philipnelson99, yup, this is almost done, see #35! I've been a bit busier than I'd have liked, but I need to do a final scan over everything to make sure it all looks good before I sign off. If you'd like to speed it along, a review (and test) of that branch would be much appreciated! :smile:

[blu3r4d0n]

@jgkamat Alright! I think I may have a go at it! Although I have exams this week so my testing my be a bit sporradic.