search

ethereum / ethereum-org-website

Ethereum.org is a primary online resource for the Ethereum community.
https://ethereum.org/
MIT License
5.14k stars 4.86k forks source link

Request: do SEO optimization for eips.ethereum.org #343

Closed virgil closed 4 years ago

virgil commented 5 years ago

Is your feature request related to a problem? Please describe. I often google for specific EIPs, for example, say EIP1948. But the pleasant display at eips.ethereum.org rarely (if ever) comes up. So then I have to go eips.ethereum.org and then manually search for it.

Describe the solution you'd like When I search for something like "EIP1948", "EIP-1948", or "EIP 1948", I want eips.ethereum.org to be the first or second hit on Google.

Describe alternatives you've considered I have no access rights to attempt this.

Additional context To reduce competition for this space, one thing we could use is use Google Webmaster Tools to remove sites all URLs matching: https://github.com/ethereum/EIPs/blob/master/EIPS/*.md from the Google index. I don't actually recommend doing this until we have eips.ethereum.org at least in the top 10 of results.

samajammin commented 5 years ago

Thanks for bringing this up @virgil! I think there's lots of opportunities for improvement depending on how we want to approach this.

To start, What is the ultimate result of what you'd like to achieve? Making EIPs more easily discoverable? Once they are discovered, making them more easily digestible (i.e. through a more appealing web UI)?

To your point, spot-checking a handful of Google search queries clearly shows eips.ethereum.org is not ranking well for terms that it could/should be. Even the Github repository's URLs (among other sites) seem to always rank higher than the website.

It's a bit difficult to quantify how much opportunity exists if we were to improve rankings. There's no tracking on the site to know it's current usage (I've submitted a PR to add Matomo) but tools like SimilarWeb suggest it's not getting much traffic. From some basic keyword research in Google Adwords, there doesn't appear to be significant volume (other than "erc20" and "erc721" related terms that get >1K searches/month) but I suspect there's a bunch of long-tail searches for specific EIPs like the ones you mentioned that form significant search volume in aggregate.

I agree it would be valuable if this content was more easily discoverable, though it’s debatable if it’s within the scope of this project (vs. creating this issue in the EIPs repo). In my mind, there are two core options.

  1. Optimize the existing site (eips.ethereum.org)
  2. Migrate content from eips.etherum.org to ethereum.org/eips

For option 1, there's some simple changes we could make to potentially get some wins, like updating the title tag templates of EIP pages to always include the number of the EIP. For instance, EIP 1948 that you mentioned - it's current title tag is Non-fungible Data Token | Ethereum Improvement Proposals. Instead of appending | Ethereum Improvement Proposals to every page's title tag, which is super generic, it should probably be more targeted (like Non-fungible Data Token | EIP 1948).

Optimizations to the current site (e.g. updating title tags) would definitely help but I believe the surest approach to achieve rankings with EIP pages would be to migrate the content from eips.ethereum.org to ethereum.org/eips. While SEO is no perfect science, it's generally agreed that subdirectories perform better than subdomains. Moving / mirroring content on ethereum.org/eips could rank well quickly from all the link authority the core ethereum.org domain has built up over the years (26K referring domains on ethereum.org vs. 1K referring domains on eips.ethereum.org). Migrating to ethereum.org could also bring UX advantages, e.g. we already have a site search feature that would allow people to search by EIP number. It'd also be easier to integrate EIP content into our Translation Program if the content exists within this repository.

To be clear, if we were to move the site content from eips.ethereum.org to ethereum.org/eips, I'm not suggesting we also migrate the entire Github repository. It would probably just be a script to pull in merged EIP pages & 301 redirect rules on eips.ethereum.org. I think it makes sense to keep that dedicated repo to create & discuss EIPs.

Might be worth fleshing out pros/cons of options & gathering some feedback from people. I'm not sure who we'd need to gather rough consensus from to make either initiative happen. @jjmstark @Souptacular feel free to weigh in.

Souptacular commented 5 years ago

Tagging Nick Johnson who I believe runs eips.ethereum.org. @arachnid

Souptacular commented 5 years ago

My opinion: There isn't harm in using Matamoto to gather user data (as long as we put a pop-up on EIPS.ethereum.org) if it will help our SEO. How best to accomplish this I will need to think on. I believe eips.ethereum.org runs on Jekyll.

evertonfraga commented 5 years ago

That's a great idea @virgil,

As an EIP user, i'd love to see a summary of the EIP page, pretty much like this:

Featured-Snippet-as-number-1-result-in-Google-SERP

The most straightforward way to do it, is defining the Structured Data Markup, a JSON included into the source code. 

<!-- JSON-LD markup generated by Google Structured Data Markup Helper. -->
<script type="application/ld+json">
{
  "@context" : "http://schema.org",
  "@type" : "Article",
  "name" : "EIP 663: Unlimited SWAP and DUP instructions",
  "author" : {
    "@type" : "Person",
    "name" : "Alex Beregszaszi"
  },
  "datePublished" : "2017-07-03",
  "articleSection" : "Abstract Currently, SWAP and DUP instructions are limited to a stack depth of 16. Introduce two new instructions, SWAPn and DUPn, which lift this limitation and allow accessing the stack up to its full depth of 1024 items.",
  "articleBody" : "Motivation</H2> […]",
  "url" : "https://eips.ethereum.org/EIPS/eip-663"
}
</script>
Arachnid commented 5 years ago

While I don't have any objection to EIPs being on ethereum.org/eips, doing this would require an automated process (presumably via travis) to build and push the site on each update. I'm not up for maintaining such a system right now, and I'm not sure how ethereum.org is rendered at present in any case.

For other changes, the source of the site is all in the EIPs repo - PRs are welcome!

fulldecent commented 5 years ago

This thread will turn into a discussion. But I propose the following to address SEO concerns.

A brief introduction: SEO is my day job.

Proposed workplan:

samajammin commented 5 years ago

Thanks for your review & comment @Arachnid! With any solution we choose, I'm definitely not looking to add any additional overhead for you or for any other EIP editors/maintainers. I'm working as a full-time developer on ethereum.org full time & I believe I could have the capacity to maintain this system. I just want to first confirm with others that it would actually add value.

Thanks for the thoughtful proposal @fulldecent! I agree with all your suggestions. One question regarding:

Update all links in this repo that link to the GitHub.com/

Could you clarify this one? Update the links to what?

I'm also curious to hear your take on the other option I mentioned above - migrating the content from eips.ethereum.org to ethereum.org/eips. In my opinion (as someone who has also had an SEO day job) & from professionals I've spoken to, that would be an additional way to improve rankings & provide additional exposure/accessibility of EIPs.

fulldecent commented 5 years ago

Links to GitHub.com

Some places (including in eips.ethereum.org) are linking to EIPs at GitHub.org. Last year if you searched the web for "ERC20 specification" at the top it would be https://github.com/ethereum/eips/issues/20 now you will see Wikipedia or https://eips.ethereum.org/EIPS/eip-20.

This is the effort I am working on.

eips.ethereum.org or ethereum.org/eips

It is critical that the GitHub repo is quickly connected to the published website. This is done now using GitHub pages. Hosting at ethereum.org/eips would require an Apache reverse proxy (or similar) to GitHub pages, or a different deploy pipeline.

I do not recommend moving the EIPs website because:

  1. The EIP editors and authors do not speak for the Ethereum project. So the EIPs should not be directly on ethereum.org.
  2. The extra infrastructure / devops to make it work will not provide incremental value.
  3. It took me a year to ask everybody to repoint their links to https://github.com/ethereum/eips/issues/721 into https://eips.ethereum.org/EIPS/eip-721, along with all the other EIPs. It will be painful to ask them to move again. (This includes websites, patents, SEC filings, academic journals, etc.)
fulldecent commented 5 years ago

@virgil Also, you do not have permission to remove GitHub.com/... from the Google index. Only GitHub has that permission.

fulldecent commented 5 years ago

I have some PRs ready to go:

https://github.com/ethereum/EIPs/pull/2369 https://github.com/ethereum/EIPs/pull/2370 https://github.com/ethereum/EIPs/pull/2371

And other quick fixes while you're here

https://github.com/ethereum/EIPs/pull/2372 https://github.com/ethereum/EIPs/pull/2373

^^ These move in the right direction of SEO.

I think nothing more is in scope here and this issue can be closed as solved after the above are merged.

If we will go further, and I think we should, we can make a new issue to adopt this policy:

EIP numbers are assigned by the EIP editors at the request of EIP authors (merging a PR that creates a DRAFT). Until the draft is published, no reference shall be made to "EIP-####" or "ERC-####". Instead you may refer to "Issue ####" or "PR ####".

And then fix all existing text in /EIPS/. This will strongly incentivize people to make a high-quality document and get an ERC number before spamming the EIPS repository and starting a marketing campaign that "OMG I wrote an EIP, let's raise money". These premature marketing campaigns for pre-draft documents, which link directly to GitHub Issues are directly the reason why Google ranks GitHub issues before our intended (and higher quality) EIPS website.

samajammin commented 5 years ago

Thanks for the replies @fulldecent. I have comments & questions on your recommendation against moving content from eips.ethereum.org to ethereum.org/eips.

The EIP editors and authors do not speak for the Ethereum project

I don't see an issue with that. The purpose of ethereum.org is to provide the best portal to Ethereum. In my mind, if we can improve discoverability of EIPs (a significant aspect of Ethereum & it's evolution), then that would be a valuable service to the community.

The extra infrastructure / devops to make it work will not provide incremental value.

Ignoring the "The extra infrastructure / devops to make it work" (which would be my concern 😄), you don't believe this initiative would provide incremental value from an SEO & UX standpoint?

It took me a year to ask everybody to repoint their links to ethereum/EIPs#721 into https://eips.ethereum.org/EIPS/eip-721, along with all the other EIPs. It will be painful to ask them to move again. (This includes websites, patents, SEC filings, academic journals, etc.)

I also don't see an issue with this - we could set up 301 redirects to the corresponding pages. My take from strictly an SEO standpoint is that the link authority of ethereum.org would more than make up for any loss of "link juice" from implementing 301 redirects. Or am I missing something?

fulldecent commented 5 years ago

Well! If there's somebody that volunteers to do the work then things get easier. Do you have a proposal for how to host the code into the new place? I'm not sure how this deploy step would work in https://github.com/ethereum/ethereum-org-website/ Also I'm not sure why ethereum.org isn't just a static GitHub Pages or similar.

You have a great point. A FIRST step is that there should be a link from ethereum.org to the EIPs. I see that already exists. This addresses the link juice issue.

fulldecent commented 5 years ago

Any action left for this issue?

samajammin commented 5 years ago

Thanks for the nudge @fulldecent.

I'm not sure why ethereum.org isn't just a static GitHub Pages or similar.

It actually is a static site (built with Vuepress). It's quite similar to the Jekyll site that powers eips.ethereum.org in the sense that it generates HTML pages from markdown source files.

Do you have a proposal for how to host the code into the new place?

I don't have a fully fleshed out proposal at this point, though my hunch is that I could write a Travis CI script or a build a simple Github bot to copy over the markdown files from the EIP directory of the EIP project into a /docs/eips/ directory in the ethereum.org project). We could set up a Github webhook to trigger this process on each merge of the EIP repo.

From my current perspective, this content migration is still worth doing. We deployed my meta tag optimizations (https://github.com/ethereum/EIPs/pull/2325) 3 weeks ago yet I haven't seen rankings improve. The current website (eips.ethereum.org) is still nowhere to be found when Googling "EIP [number]". As I outlined above, aside from pure SEO benefits, the primary advantages would be:

For next steps, I think I should discuss this with each of the EIP maintainers (primarily @Arachnid) to get their input on if this would be a valuable initiative. Once I've gathered their feedback (& assuming we agree it's worth pursuing), then I'd be happy to put a more detailed proposal together.

fulldecent commented 4 years ago

@samajammin Please update this link to HTTPS.

Screen Shot 2019-12-21 at 22 05 06

Please accept this https://github.com/ethereum/EIPs/pull/2446

It is is a quick fix. It can be extended in the future by adding this information to the bottom of every EIP.

Just showing you what else I'm working on.

Here is a list of people that think erc721 dot org is the authoritative website for ERC-721.

erc721.org-1backlinkperdomain-subdomains-live-22-Dec-2019_03-07-46.csv.zip

(only the first thousand)

My interest is in getting them to update this and link to the correct URL which is https://eips.ethereum.org/EIPS/eip-721

axic commented 4 years ago

The most straightforward way to do it, is defining the Structured Data Markup, a JSON included into the source code.

<!-- JSON-LD markup generated by Google Structured Data Markup Helper. -->
<script type="application/ld+json">
{
  "@context" : "http://schema.org",
  "@type" : "Article",
  "name" : "EIP 663: Unlimited SWAP and DUP instructions",
  "author" : {
    "@type" : "Person",
    "name" : "Alex Beregszaszi"
  },
  "datePublished" : "2017-07-03",
  "articleSection" : "Abstract Currently, SWAP and DUP instructions are limited to a stack depth of 16. Introduce two new instructions, SWAPn and DUPn, which lift this limitation and allow accessing the stack up to its full depth of 1024 items.",
  "articleBody" : "Motivation</H2> […]",
  "url" : "https://eips.ethereum.org/EIPS/eip-663"
}
</script>

There is already some of this included: https://github.com/ethereum/EIPs/blob/master/_includes/head.html#L34-L42

Here's a larger example: https://mincong-h.github.io/2018/08/22/create-json-ld-structured-data-in-jekyll/

It would be easy to extend the EIPs to support these fields, but would appreciate some guidance which fields make the most impact.

Update: I think "TechArticle" might be the best category for us: https://jsonld-examples.com/schema.org/code/article/techarticle-markup.php

axic commented 4 years ago

@samajammin I would oppose moving eips from eips.ethereum.org to ethereum.org/eips. I do not see any upside, but see a lot of downsides. What is the benefit?

samajammin commented 4 years ago

Sorry for the delayed reply on this! Appreciate you taking the time to review this thread & weigh in @axic.

The benefits I see are primarily laid out in my comment above (https://github.com/ethereum/ethereum-org-website/issues/343#issuecomment-546628589) but to reiterate & add a couple:

I totally understand how this plan may be controversial & I can't guarantee this change would improve everything but from my current perspective, there's near zero risk of ethereum.org/eips performing worse in SEO than eips.ethereum.org currently does. If we see that as the primary goal, I think it's worth trying.

What are the downsides in your mind? It's quite possible there's many I'm overlooking, including risks outside of SEO performance.

samajammin commented 4 years ago

@fulldecent thanks for sharing your update. I don't have the necessary permissions to edit that link in the EIP repo to HTTPS. I suggest you open an issue there or ping one of the maintainers.

fulldecent commented 4 years ago

@samajammin you got it https://github.com/ethereum/EIPs/issues/2487

samajammin commented 4 years ago

@axic would you mind listing the downsides you see? Trying to understand the risks here. Thanks.

jpitts commented 4 years ago

@samajammin I can see the rationale for making EIPs more accessible and findable. One thing to consider is the target audience of the main website. The EIPs are highly technical documents for use by protocol developers and dapp developers, probably more experienced with the tech than visitors to ethereum.org.

One approach may be to generate a page for each EIP, but with content appropriate for the audience of the website.

Example content:

jpitts commented 4 years ago

Going with the TL;DR pages approach brings in more community engagement; this is content and links which would need to be developed. It could greatly improve people's initial engagement with the information as well as a full EIP even with a good overview can be intimidating.

One side benefit of such an approach may be that Google picks up on the links, presenting a navigation when an user searches for something like "EIP-1" or "ERC-20".

fulldecent commented 4 years ago

I have read all the EIPs. I don't think there is any way to make them usable for a non-technical audience.

Also I have talked with all the EIP authors in the past year. They are interested to publish something. It will be impossible to require them to rewrite a summary for a new audience. And even if they did, it will be written poorly.

fulldecent commented 4 years ago

On the other hand, if you need one person to that actually can write the content (i.e. an actual editor role) then I can do that. It will be a bit of effort and I can write the pages so that it will explain every draft and final EIP and how it is important to the community.

jpitts commented 4 years ago

One inspirational piece of infrastructure is "Simple Wikipedia" (https://simple.wikipedia.org), this is a simplified mirror of many Wikipedia pages. Looking up various topics in Simple Wikipedia proves the importance of having good editing on that content. TL;DR is challenging to write!

But in this case, TL;DR actually isn't the right term for what ethereum.org does. The content perhaps is more like a "executive summary". This would be really helpful to have for each EIP!

ethereum.org's summary for Ethereum (https://ethereum.org/what-is-ethereum/):

Ethereum is the foundation for a new era of the internet:

  • An internet where money and payments are built in.
  • An internet where users can own their data, and your apps don’t spy and steal from you.
  • An internet where everyone has access to an open financial system.
  • An internet built on neutral, open-access infrastructure, controlled by no company or person.

And Simple Wikipedia's TL;DR for Ethereum (https://simple.wikipedia.org/wiki/Ethereum):

"Ethereum is a system on the Internet where people can make deals." LOL

samajammin commented 4 years ago

Good idea. Thanks for pushing the discussion forward.

Update on the latest around this. A couple weeks ago I created a proposal to migrate EIP content to ethereum.org & presented this at the 3/11 & 3/25 EIPIP public calls. The response & feedback was mostly positive. We created an action item to investigate the implementation further, which I did:

My current thinking & proposal is to take an iterative approach to next steps:

  1. Launch the "Intro to EIPs" page on ethereum.org (https://github.com/ethereum/ethereum-org-website/issues/831), then wait & see if that improves SEO for the EIPs site (which could take week/months to see SEO gains)
  2. Add site search to the EIP site (similar to the search recently added to https://blog.ethereum.org/search/). This could be a quick UX win for that site. Given that they’re both Jekyll sites, it should be pretty easy to mimic that implementation.

I welcome feedback & ideas around this plan. From that initial "Intro to EIPs" page (#831), we could easily build out additional pages (e.g. TL;DR / executive summaries for specific EIPs that @jpitts described). Or perhaps we eventually gain additional momentum behind the proposal to migrate the content entirely.

samajammin commented 4 years ago

@fulldecent regarding your earlier proposals, specifically:

Update all links in this repo that link to the GitHub.com/... (I can do this)

I do agree it would be beneficial to update internal links. Looking at the EIPs website (e.g. EIP 1), there's a ton of links to the GH version of pages (instead of the eips.ethereum.org version of the page). If that's an initiative you have capacity to take on, I agree it would be worthwhile.

fulldecent commented 4 years ago

That update is already done. I have updated the links where possible and semantically equivalent. E.g. I have updated when an EIP was referenced directly ("this is extending ERC-20, see URL") and not when there was a different reference ("there was significant discussion on this issue (see GH issue 20) and we concluded that...").

Timewise, I did that several months ago.

samajammin commented 4 years ago

@fulldecent awesome! Just to confirm my understanding of the difference w/ an example, looking at EIP 1:

Core - improvements requiring a consensus fork (e.g. EIP5, EIP101), as well as changes that are not necessarily consensus critical but may be relevant to “core dev” discussions (for example, EIP90, and the miner/node strategy changes 2, 3, and 4 of EIP86).

All the links in this section (& pretty much all the links I see in EIP 1) link to the respective Github page. Some like to the Github issue but some even link to the Github markdown page (which doesn't provide any additional context). Seems to me they could all link to the eips.ethereum.org page instead? At the least, all the links that reference the Github markdown page could be updated, ya?

github-actions[bot] commented 4 years ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

samajammin commented 4 years ago

Given that https://ethereum.org/eips/ has been shipped, I'm going to leave this closed.

You can find the site search issue here: https://github.com/ethereum/EIPs/issues/2607

fulldecent commented 4 years ago

Looks good as deployed. As always, if you're looking for content and a project manager to lead ethereum.org so that it beats Hyperledger for attracting new people, and EF is supporting the project, then call me.

fulldecent commented 4 years ago

Related --> https://github.com/ethereum/EIPs/pull/2738