paperdigits commented 4 years ago

I think it'd be a good idea to move to a static site generator that we're using more and more in the community, hugo. It's a single binary, battery included package that is fast. You don't have to worry about upgrading python versions or keeping up to date with plugins. It just works.

patdavid commented 4 years ago

I'm inclined to agree here, but we'd need to check dependencies of any py plugins that have been written. Also, we don't control the server or domain name, so we'd need to make sure Hugo can be available on the server and included in the site build+deploy there.

paperdigits commented 4 years ago

Yes, I'd like to get approval from the project at large before going forward with any work, I guess that'd be @turbogit and @houz. We'd also not want to loose any functionality. This is just a conversation to get started and see how everyone feels about it.

TurboGit commented 4 years ago

I cannot really tell as I don't know Hugo system at all! Can you describe a bit the pro & cons of this solution ?

paperdigits commented 4 years ago

Pros:

static binary to build the website & is a batteries included style package, no plugins to update with Hugo
version of version upgrade is very easy, just update your distro package, docker container (maintained by @darix for our other hugo projects), or grab the binary. Right now for Pelican we're waiting on a port to python3. Python2 has been dropped from many distros, so people can't even build the website anymore.
great familiarity in the community - both @patdavid and I both use this package extensively in other projects. Having continuity between projects makes things easier (not really a pro for the darktable project itself, but I'd like to start adding more resources to the website, but I don't know jinja2, the templating engine of Pelican, and I don't really want to learn another template engine).
uses a more standard yaml-based metadata for posts.

Cons:

nobody on the team knows go (the language hugo is written in)
extensibility means writing go code and rebuilding the binary (though I don't think we'd need to do this)

paperdigits commented 4 years ago

I'm sure others will chime in with some cons... @houz, can we run hugo on the server? Or would you consider moving to a hosted solution to build, like say buildbot or something like that? Or even github actions?

allan2 commented 4 years ago

@paperdigits how about another static site generator, NextJS?

Posts and translations written in Markdown. Site automatically built from git. Strong ecosystem with plugins for translations and images (lazy loading images and proper sizing).

patdavid commented 4 years ago

@allan2 isn't NextJS a framework (not a static site generator)?

I'd say that we are going to want Markdown no matter which static site generator is being used. We'd also rather not have to rely on any JS in the final page if we can avoid it (I think the site should work just fine now w/o JS installed).

paperdigits commented 4 years ago

It looks like NextJS recently gained the ability to render static sites, but it's a React JS framework.

@allan2 are you wanting to help maintain the website or are you just making a suggestion?

Hugo will give us posts in markdown and translation also in markdown. Hugo, a single binary builds the site.

I don't see plulgins as a plus for static site generators anymore, just another things that needs to be maintained. To be perfectly honest, we haven't done a great job at managing things with lots of dependencies. I've used jekyll and middleman (ruby), pelican (python), and metalsmith (javscript). Upgrading major versions of all three of these is not fun. With hugo, it is a single binary, there isn't anything to maintain. Both @patdavid and I are comfortable with hugo.

patdavid commented 4 years ago

More importantly, I think minimizing requirements and barriers to contributions will help increase engagement in the long run. (Someone wanting to write something for the website only needs a single binary file to build and view the site - nothing else needed).

I'm not against other solutions, but if we have the manpower to migrate it makes sense to consolidate things into a more streamlined tool (so it gets out of the way and let's people contribute faster/easier).

It may also help to ease our i18n problems a bit? (Not 100% sure about this).

allan2 commented 4 years ago

@paperdigits I'd like to help maintain the website and design as well. I think Gatsby, Next, and Hugo are all capable SSGs, although I haven't used Hugo before.

I've started on a demo that I'd like to share with you in a few days time. Would that be alright? In the meantime, I'll think on your other concerns and get you a detailed response soon.

paperdigits commented 4 years ago

@allan2 please do share whenever you're ready.

allan2 commented 4 years ago

Thanks for the patience.

Check out the proposed site. Code is here: https://github.com/allan2/darktable.org

It's very rough still. What do you guys think? P.S. can I move discussion about the site to the mailing list?

Release Notes for v0.1

This is the first release. Many areas are unfinished.

new skeleton (header, footer, main)
multilingual site support
new sections for home page
download page
team page
migration script to convert .md posts to YAML gray matter
blog posts to HTML with proper paths

TODO:

numerous CSS fixes, viewports
create consistent style and branding
blog snippets
collect feedback!

paperdigits commented 4 years ago

@allan2 that looks like a decent start. I'm still not sold on the static site generator you used, as the javascript stuff comes and goes way too fast for my taste.

paperdigits commented 3 years ago

I've started looking at the current templating code in order to port this over to Hugo. This will allow us to put the user guide and lua docs back on the main site. Does anyone have any objection to this? @houz?

allan2 commented 3 years ago

Hugo does seem like an excellent framework. Looking forward to it.

robbymilo commented 3 years ago

Hey @paperdigits, really like the idea and would love to help. I have experience with hugo and using it in build pipelines for documentation websites (see https://grafana.com/docs/grafana/latest/). Want to chat/sync up on this?

paperdigits commented 3 years ago

@robbymilo that'd be great. I'll get a few things started, then we can sync up. It'll be mostly just translating the jinja2 templates to go templates, then implementing the few plugins that we have currently in pelican to go templates as well.

You can find me in #darktable on freenode, or get me on matrix: @paperdigits:matrix.org or by email.

paperdigits commented 3 years ago

From an email with @robbymilo, I figured we should have the plan here as well:

The path for the port should be this, so far as I can tell:

Transform content metadata to yaml. Every post needs --- before and after the metadata. I've come up with some sed scripts to do that.
Move the content so it fits Hugo's expectations and matched the current website structure when published
Configure Hugo. Figure out the post slugs, etc so it matches the current content.
Make a new Hugo theme. Move current static assets to the correct place: CSS, JavaScript, fonts, etc
Translate current templates from jinja2 to go templates. While doing so, get rid of any content in the current templates (like the home page). There aren't that many templates.

robbymilo commented 3 years ago

@paperdigits I think that's a great path. I've got a lot of those mostly finished here: https://github.com/robbymilo/dtorg/tree/robbymilo/hugo (could use some help with sed scripts)

As I dig deeper I'm finding more and more work. There's a lot of hard-coded HTML in markdown and templates.

paperdigits commented 3 years ago

@robbymilo I wouldn't worry about HTML in the markdown, that is valid markdown, after all.

I'd also avoid shortcodes in the markdown, as it makes us dependent on hugo and is not valid markdown.

paperdigits commented 3 years ago

My branch is here: https://github.com/paperdigits/dtorg/tree/hugo-port

I'm still looking through some of the content, we need to handle links like {filename}, {article.title}, and {attach}.

I'd also like to make a distinct theme in the themes directory rather than have templates in the main repo.

@robbymilo your commits are a bit hard to follow.

robbymilo commented 3 years ago

@robbymilo I wouldn't worry about HTML in the markdown, that is valid markdown, after all.

Sure.

I'd also avoid shortcodes in the markdown, as it makes us dependent on hugo and is not valid markdown.

The shortcode I added was due to markdown syntax within HTML. As far as I know markdown within block-level HTML is not valid, so if a shortcode doesn't work then the markdown will need to change.

My branch is here: https://github.com/paperdigits/dtorg/tree/hugo-port

I'm still looking through some of the content, we need to handle links like {filename}, {article.title}, and {attach}.

It looks like just removing {filename} and {attach} works, {article.title} could be changed to a shortcode.

I'd also like to make a distinct theme in the themes directory rather than have templates in the main repo.

No opinion here. Either works.

@robbymilo your commits are a bit hard to follow.

Yes, it's very much a work in progress state and would need a rebase before a PR.

robbymilo commented 3 years ago

I have made good progress on my branch: https://github.com/robbymilo/dtorg/tree/robbymilo/hugo

I also have a preview (https://5fc102d09ad37b6b31257bba--darktable.netlify.app/) deploying to Netlify via GitHub actions.

I think we are at a point where we should combine branches and start a final todo list @paperdigits?

paperdigits commented 3 years ago

@robbymilo wow, yes. I am taking a look at it now.

paperdigits commented 3 years ago

@robbymilo where is the best place to keep this list?

I've tried to be very careful with my contentdirectory, but it doesn't render anything when I drop it in your repo. I still need t get rid of a few content things, then we can try and sync up.

robbymilo commented 3 years ago

@paperdigits Let me put together a draft PR.

paperdigits commented 3 years ago

Sounds good!

On December 5, 2020 1:20:59 AM PST, Robby Milo notifications@github.com wrote:

@paperdigits Let me put together a draft PR.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/darktable-org/dtorg/issues/113#issuecomment-739151196

paperdigits commented 3 years ago

This was done in the last few days.

darktable-org / dtorg

Migrate website static site generator to Hugo #113

Release Notes for v0.1