[SEO][META] Provide some basic SEO meta tags OOTB

klonos commented 3 years ago

Current OOTB header for a vanilla Backdrop site, when visiting the home page:

<head>
    <meta charset="utf-8">
<link rel="shortcut icon" href="https://example.com/core/misc/favicon.ico" type="image/vnd.microsoft.icon">
<link rel="alternate" type="application/rss+xml" title="Home page feed" href="https://example.com/rss.xml">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="Generator" content="Backdrop CMS 1 (https://backdropcms.org)">
    <title>Home | Backdrop</title>
    [stylesheets ... ]
    [scripts ... ]
  </head>

Proposing to add the following basic meta tags:

Home page

Allow these to be configured from the site information page in /admin/config/system/site-information (same as site name, slogan, and favicon).

<meta property="og:site_name" content="Site Name">
<meta property="og:type" content="website">
<meta property="og:url" content="https://example.com">
<meta name="description" property="og:description" content="Some short description for the site,
  if provided under admin/config/system/site-information.">

<link rel="image_src" href="https://example.com/sites/default/files/site_logo.jpg">
<meta property="og:image" content="https://example.com/sites/default/files/site_logo.jpg">
<meta property="og:image:type" content="image/jpeg">
<meta property="og:image:width" content="200px">
<meta property="og:image:height" content="200px">
<meta property="og:image:alt" content="A description of what is in the image (not a caption).
  If the page specifies an og:image it should specify og:image:alt">

Note: see https://developers.facebook.com/docs/sharing/best-practices

Use og:image:width and og:image:height Open Graph tags to specify the image dimensions to the crawler so that it can render the image immediately without having to asynchronously download and process it.

Personal note: long sentence w/o any punctuation triggers my OCDs 😅 ...you'd expect that Facebook would know better 🤷

Social-media-specific meta tags

OpenGraph (og:*) seems to be supported by Facebook, twitter has specific tags:

<meta name="twitter:card" content="** Required!!! **
  There are currently four card types available:
  - 'summary': refines your web content into a snapshot image with a thumbnail,
    title and description. Users can even see the URL of your website displayed
    at the bottom of the card. The idea is to give your viewers a preview of
    your content before they click through to your site. If you use Twitter
    to accelerate your blog traffic, this could be an excellent option.
  - 'summary_large_image': same as 'summary', but trades a bit of the description space
    for a larger image (at least 280 x 150 pixels / still no more than 1MB)
  - 'app': not suitable for sites
  - 'player': suitable for multimedia content
  ">
<meta name="twitter:site" content="Site name">
<meta name="twitter:title" content="Title of content (max 70 characters) - og:title equivalent">
<meta name="twitter:creator" content="The content creator / author.">
<meta name="twitter:description" content="Description of content (max 200 characters)
  og:description equivalent">
<meta name="twitter:image" content="https://example.com/sites/default/files/site_logo.jpg
  URL of image to use in the card.
  At least 120×120 pixels and no more than 1MB
  JPG, PNG, WEBP and GIF formats are supported.
  Only the first frame of an animated GIF will be used.
  SVG is NOT supported.
  equivalent to og:image">
<meta name="twitter:image:alt" content="A text description of the image (max 420 characters).
  equivalent to og:image:alt">

Additional meta tags to consider for Post nodes

<meta property="og:type" content="article">
<meta property="og:title" content="Article title">
<meta property="og:site_name" content="Site name">
<meta property="og:url" content="https://link.to/article">
<meta property="og:image" content="https://link.to/post/image">
<meta property="og:image" content="https://link.to/post/another-image-if-field-configured-for-multiple">

<meta property="article:published_time" content="When the article was first published.
  Format should be: date (year, month, day) and an optional time component (hours, minutes)
  see: https://en.wikipedia.org/wiki/ISO_8601">
<meta property="article:modified_time" content="When the article was last changed. - ISO_8601">
<meta property="article:expiration_time" content="When the article is out of date after. - ISO_8601
  Can be used when unpublished content is allowed to be accessed by anonymous,
  but is marked as expired/unpublished.">
<meta property="article:publisher" content="Publisher(s) of the article.
  It could be the site name, or URL, or both.">
<meta property="article:author" content="Writer(s) of the article.
  Could be a string with the user name, or a link to the user profile.">
<meta property="article:author" content="...can be declared multiple times,
  to include multiple authors.">
<meta property="article:author" content="...although Drupal/Backdrop
  do not support multiple authors per node AFAIK.">

<meta property="article:section" content="A high-level section name. E.g. Technology">
<meta property="article:tag" content="Tag words associated with this article.">
<meta property="article:tag" content="...multiple tags may be defined.">
<meta property="article:tag" content="...another tag">

Additional meta tags to consider if the node is part of a Book

(if the book module is enabled)

<meta property="og:type" content="book">
<meta property="og:title" content="Node title">
<meta property="og:site_name" content="Site name">
<meta property="og:url" content="https://link.to/book">
<meta name="description" property="og:description" content="If the body field is configured
  to include a separate summary - otherwise omit this meta tag">
<meta property="og:image" content="https://link.to/post/image - if this is a Post node,
  otherwise omit this meta tag">
<meta property="og:image" content="...more images">
<meta property="book:author" content="A string with the user name, or a link to the user profile.">
<meta property="book:release_date" content="When the node was published.
  Format: date (year, month, day) and an optional time component (hours, minutes)
  see: https://en.wikipedia.org/wiki/ISO_8601">
<meta property="book:tag" content="Tag words associated with this book - if the node is of type Post">
<meta property="book:tag" content="...multiple tags may be defined.">
<meta property="book:tag" content="...another tag">

Generic meta tags to consider for any node type:

<meta name="description" property="og:description" content="If body field allows separate summary.
  Omit if no summary.
  If this is the home page, then use the summary specified in admin/config/system/site-information">
<meta name="author" content="user name">

Meta tags to consider for user profile pages:

<meta property="og:type" content="profile">
<meta property="og:url" content="Can be an external link to social media,
  but for Backdrop it could be https://link.to/user/profile">
<meta property="og:image" content="Can support multiple images, but for Backdrop core
  we only support a single one - and only if the 'Enable user pictures' setting is enabled,
  otherwise this tag should be omitted.">
<meta property="profile:username" content="The account username">
<meta property="profile:first_name" content="John - If we ever support first/last name OOTB">
<meta property="profile:last_name" content="Doe - If we ever support first/last name OOTB">

Views-generated pages

Perhaps consider extending views to allow a visitor-facing description (besides the admin description), then use that as <meta name="description" property="og:description" content="..."> if the page being rendered is a view.

Page title

Note that we are currently rendering this:

<title>Home | Backdrop</title>

If we want to support or optimize for OpenGraph and schema.org, the we should be doing this instead:

<title property="schema:name">Home | Backdrop</title>
<meta property="schema:name" content="Home | Backdrop">
<meta name="description" property="schema:description og:description"
  content="Add description, if provided in admin/config/system/site-information">

The above combines w3c (name="description"), schema.org (property="schema:description"), and OpenGraph (property="og:description").

indigoxela commented 3 years ago

Wouldn't that conflict with existing contrib modules?

philsward commented 3 years ago

Curious, are facebook and twitter search engines? If we're talking about SEO, then I would recommend only adding tags for search engines.

The vast majority of sites will want to be indexed for search, but not all websites are going care about their visibility on social media.

After thinking about it, it's not a bad idea to include basic tags for the social media sites... I do think the majority of social media tags should be left to contrib. Let core handle the most important ones though. Wondering if the title needs updated to include Social Media?

It also might not be a bad idea to create a new area in configuration specifically for these types of tags and let contrib either extend that area or alter it. I think it would be nice to have core handle the initial bits and give contrib a designated place to add to it. (In other words, outside of the currently proposed area)

klonos commented 3 years ago

Wouldn't that conflict with existing contrib modules?

It might. That's why we discussed to implement this in a way that would easily allow contrib to either override or completely remove any core-added meta tags.

These are the SEO-related modules I'm aware of, along with their current usage counts:

https://github.com/backdrop-contrib/metatag ~320 / ~1600
https://github.com/backdrop-contrib/base_meta <10 / ~1600
https://github.com/backdrop-contrib/seo_meta ~155 / ~1600
https://github.com/backdrop-contrib/custom_meta no data / no release on b.org

Perhaps that list and data will help us decide re possible impact on existing sites. The way I am reading that data is:

a cumulative ~30% of Backdrop sites are using one of these modules
the rest of ~70% does nothing (assuming that they either don't care, or simply aren't aware)

It seems that if we coordinated any core changes with the maintainers of the most popular of these, we'd minimize risk of any possible breakage. Perhaps the battle plan would be to:

get this issue into an RTBC state, but do NOT commit any changes to core.
allow popular contrib modules to release updated versions that handle these changes.
commit any changes to core, and ship them with the next minor release.
allow 70% of Backdrop sites to profit!

My point is that if we implemented default meta tags, we'd provide an effortless benefit to all Backdrop sites OOTB, without site builders/owners having to lift a finger ...unless they wanna tweak further, in which case they should install and configure one of the contrib modules. But I do understand that we shouldn't break existing sites 👍

indigoxela commented 3 years ago

It seems that if we coordinated any core changes with the maintainers of the most popular of these, we'd minimize risk of any possible breakage.

I'm afraid that could never cover custom implementations. I'm a bit scared of doing such a potentially breaking change, and I don't fully understand the need for it. Things are in contrib already.

Honestly, for me this currently looks like a 2.x issue, but I might be wrong.

klonos commented 3 years ago

@indigoxela what if the solution we come up with is done in way that doesn't break contrib and doesn't break custom code? Wouldn't the improved OOTB SEO benefit the 70% of Backdrop sites? (which is 10% short of being the 80%, which we prioritize)

What we can do is to work towards making sure that core is adding tags only if contrib/custom solutions have not provided any already. Alternatively, make sure that there is proper "weight", so that contrib/custom implementations come after the core ones (so that they can override it).

A quick research shows that adding meta tags is done via backdrop_add_html_head(), so if contrib/custom implementations stick to that, they should be able to override what core adds. I would like us to explore that option before we dismiss any solution, in fear that it might break existing sites.

update: https://www.zyxware.com/articles/4817/drupal-how-to-add-meta-tags-programmatically-to-custom-pages-in-drupal suggests that hook_html_head_alter() can be used to override meta tags:

https://docs.backdropcms.org/api/backdrop/1/search/hook_html_head_alter

Alter XHTML HEAD tags before they are rendered by backdrop_get_html_head().

klonos commented 3 years ago

Curious, are facebook and twitter search engines? If we're talking about SEO, then I would recommend only adding tags for search engines.

No, they aren't search engines @philsward. That's why I mentioned that: a) Facebook respect/support og:* tags, which are an open standard that is used by search engines. b) twitter uses custom twitter:* tags, which is why I said that we should consider adding them.

In fact, I am not proposing that we add all of the tags I list in the issue summary. Every section is a result of my research of what seems to be supported, and what at the same time already matches what we output. For example:

body field summaries would match name="description" and property="og:description"
config_get('system.core', 'site_name') matches property="og:site_name"
node author matches property="article:author"
etc.

...not all websites are going care about their visibility on social media.

Agreed 👍 ...AFAIK, the social media tags are not being used to produce something that social media crawls actively. The are being used when for example someone posts a link of a site in one of those social media platforms. For example, this is what was produced when I posted a link to a site with such tags in our company's Slack channel:

Notice how the link to the AltaGrade site produced an automatic "preview" with a beautiful custom summary in Slack (the way that the site owner specifically configured it for that page - not just the random x first characters of the page content). Also notice that the link to backdropcms.org produced absolutely nothing! That's what we are trying to improve here, OOTB.

...I think it would be nice to have core handle the initial bits and give contrib a designated place to add to it. (In other words, outside of the currently proposed area)

Yup, that's the idea: core provides an automated, "lightweight" set of defaults, which make sense and work for 80% of the cases. At the same time, nothing is hardcoded, so that contrib/custom solutions can build on top of that.

klonos commented 3 years ago

Honestly, for me this currently looks like a 2.x issue, but I might be wrong.

@indigoxela here's another thought: how about some sort of setting (either visible/configurable via the admin UI, or "hidden" as a settings.php option) which makes sure that all these new SEO options are not enabled for existing sites (no update hook to add it), whereas it is on by default for new sites?

indigoxela commented 3 years ago

@klonos Besides the fact that I still don't get the "why"... :wink:

Some of the tags you propose need values set, that can't exist "automagically", like the image field for "og:image" and the image attributes. Some need clear decisions, where to use them (e.g. not everything is supposed to be an "article with an author". This requires at least some base knowledge, what these meta tags are supposed to do.

That's why I'm totally fine with the fact that most of these special meta tags are a job for contrib. In fact, different existing modules provide different complexity, which is another advantage of contrib (lightweight tool or complex beast - depending on demands).

Regarding convenience for the site builder or admin when everything (or at least a lot) is in core: Is it really that exhausting to click around a little bit to install a contrib module via the installer? :confused: My personal experience with those additional meta tags is: the module is installed very quickly - the cumbersome part is to set them up.

But maybe the SEO experts in our community (I'm none) want to share their thoughts here, too.

philsward commented 3 years ago

I can see where custom content types become a bit hairy with defaults, but the fact of the matter is that Backdrop does nothing outside of the bare minimum for SEO and social media.

I like the idea of core handling the foundation of SEO tags by creating a common and familiar place to configure it with a few basic defaults, then let contrib extend it.

I think the biggest reason 70% of sites don't use the contrib is because they aren't aware of them, or the site owners don't know they need them, or in my case I completely space on adding them. Then there's the issue of 4 different modules that claim to do the same thing, just a little different so it becomes overwhelming to decide which one is actually the best and going to do what is needed, and also going to do what is advertised.

I think for content types, views, terms and layouts, just having a place where tags can be configured (minimum of title and description) would be great. Leave them blank with no default but at least they have the ability to allow seo tags. Let a contrib like metatag insert the token. This would also allow custom entity pages to tie into the core seo api by default so contribs don't have to manually add it to the list later.

If core has an API for SEO that allows contrib to tie into it, then make it a sub-module of core that can be disabled to reduce clutter and overhead for say an internal or locally hosted site.

So, I'll agree that a good idea is to have core handle the foundation, and I also think it's a good idea to not enforce too many arbitrary defaults.

Some of the contrib will require major rewrites to tie into the api, but that's the cost of making things better at the core level. It's always been frustrating to remember to add xmlsitemap and metatag to every site after everything else that needs to be done to a site to get it going. If I were to see a blank title and description meta when adding content to a page, it's a reminder to go add the token metatag to help automate it. If it isn't there, I'll completely forget to add it.

I think I would start by incorporating something like the metatag module as the foundation and stripping out some of its complexities, leaving those complexities inside metatag as contrib.

findlabnet commented 3 years ago

So, in case of positive decision. We need new core module that should:

provide basic, or default set (require another round of discussion to decide which ones) of tags;
works with content types, taxonomy and views pages;
multilingual (or translatible) from the start;
provide convenient API for plugable extensions - SEO is mostly gray area and rules of the game may vary quickly and unpredictably;
find/detect already existing (hardcoded/programmed on theme level or added by contribs) tags and inform site owner/admin how to deal with (fix or migrate).

I forgot something?

indigoxela commented 3 years ago

Some of the contrib will require major rewrites to tie into the api, but that's the cost of making things better at the core level.

I think, this needs some feedback by the module maintainers (one already did) - at least the ones we know - as Backdrop "values the needs of the contrib developer over the needs of the core developer". Keep in mind, we can't get feedback regarding custom code (in custom modules or themes) out there.

Migrations from D7 also have to be considered.

If the proposed solution requires a fundamental API change, we have to be very careful, and potentially have to postpone the change until 2.x.

stpaultim commented 3 years ago

@indigoxela - I don't know if this helps with the "why" question, but this was discussed during one session at Backdrop LIVE. The discussion was about all the things that almost every site needs, but are not currently included in Backdrop core. We discussed things like metatags and site maps. The consensus was that many site owners are not aware of the need for these things and may never implement them despite the fact that virtually every site probably should have them.

We then talked about ways that Backdrop Core might help these site owners. I think that there was some consensus that (in that specific discussion) that if Backdrop Core should not try to take on too much of this work, but if there is some basic meta tag information that could could reliably provided, then maybe that should be provided by core and we should leave the advanced/custom stuff to contrib.

I support the goal of making life easier for this type of user, but I'm not sure what is practical and/or if there are other good reasons to NOT do this.

philsward commented 3 years ago

@findlabnet forgot layouts.

Single path layouts will need this as well.

klonos commented 3 years ago

@klonos Besides the fact that I still don't get the "why"... 😉

@indigoxela the why is clearly stated here: https://backdropcms.org/philosophy

Simplicity: Write code for the majority.

Focus: Include features for the majority.

I believe that I clearly demonstrated that it seems that 70% of the Backdrop sites out there do not do anything about SEO, which hurts site owners (and they might not even know about it). This also hurts Backdrop as a product, as having some "automagic" SEO out of the box would increase its marketing value.

Is it really that exhausting to click around a little bit to install a contrib module via the installer? 😕

No, that part is not exhausting at all.

...the module is installed very quickly - the cumbersome part is to set them up.

Exactly! We are making knowledge assumptions here. People managing/building/owning these sites need to:

know what SEO is, and that implementing it for their site would benefit their business...
assuming they know that, they then need to know that for Backdrop they need contrib modules in order to achieve that without custom solutions
assuming they know that, they then need to know that contrib modules exist for SEO
assuming they know that, they then need to know which one of these contrib modules to choose
assuming they know that, they then need to know how to configure the one they chose
assuming they know that, they need to know SEO and how to tie it with each Backdrop component (home page, user profiles, nodes etc.)

That is a lot of work as you can see. And sure, they can hire people to do it for them, but that then goes against the "affordably" in "build highly customized websites affordably".

That's why I'm totally fine with the fact that most of these special meta tags are a job for contrib.

Fully agree with that 👍 ...I am NOT proposing to merge the Metatag contrib module functionality into Backdrop core; I am NOT proposing that we add ALL of the meta tags I have researched and added in the issue summary - just the ones that make sense, and are minimal, non-breaking change.

...the fact of the matter is that Backdrop does nothing outside of the bare minimum for SEO and social media. ...core handling the foundation of SEO tags by creating a common and familiar place to configure it with a few basic defaults, then let contrib extend it.

...if there is some basic meta tag information that could could reliably provided, then maybe that should be provided by core and we should leave the advanced/custom stuff to contrib.

Yup. That ^^ 👍

If core has an API for SEO that allows contrib to tie into it...

SEO is basically adding meta tags to certain items in the <head> of each page. And Backdrop already has the necessary functions to allow for that. I'm pretty sure that all the SEO contrib modules use these functions.

Keep in mind, we can't get feedback regarding custom code (in custom modules or themes) out there.

I'm not sure that people would custom-code SEO when there are modules like Metatag, but I really doubt it. I'll admit that we don't have the metrics to back this up, but we don't have any metrics to claim the opposite either. I'd be willing to have whatever core solution comes out of this have a flip on/off switch (whether that's a separate module, or a global "Enable basic SEO" setting).

One thing I'd want to know is that if people custom-code SEO, then how would they do it? Would it be by creating entirely new templates? in other words, would they bake SEO into their custom themes? Having some knowledge around that would help us avoid breaking things for them.

Migrations from D7 also have to be considered.

Can you please elaborate on that @indigoxela? I agree that this is an important point 👍

jackaponte commented 3 years ago

I believe that I clearly demonstrated that it seems that 70% of the Backdrop sites out there do not do anything about SEO, which hurts site owners (and they might not even know about it).

My shop runs dozens of Backdrop sites as containers/CMS front-ends for CiviCRM; we consciously don't want to do anything about SEO for those sites. Just dropping that perspective in here!

klonos commented 3 years ago

...we consciously don't want to do anything about SEO

Wondering why that @jackaponte. I thought that SEO was a good thing in general. Does it have hidden drawbacks?

edit: I just saw your reply in Zulip @jackaponte

As I posted on the Github issue, my shop has a number of certainly not illegal nor testing Backdrop sites that serve primarily as containers for CiviCRM, with no public content at all. We have no interest in SEO! (For those particular sites.)

Fair enough, but that doesn't mean that any baked-in SEO provided by core would hurt those sites. Right? ...or am I missing something?

philsward commented 3 years ago

I think @jackaponte is pointing out additional bloat their environment sites don't need.

I like the overall idea, but I would push for it to be a sub-module of core that can be disabled for this very reason. If a local site or internal site doesn't need it, shut it off.

Otherwise out of the box, I agree that @klonos assumption that SEO is wanted, is a proper assessment. I've wished for years that Drupal would have done a better job on this front. It's like they made it great for SEO back in 2005 and then never gave it another thought.

backdrop / backdrop-issues