gatsbyjs / gatsby

The best React-based framework with performance, scalability and security built in.
https://www.gatsbyjs.com
MIT License
55.21k stars 10.33k forks source link

Q on the forced trailing '/' on URLs & redirects - What's happening here? #15317

Closed pewStocky closed 4 years ago

pewStocky commented 5 years ago

Hey all,

Before Gatsby integration we had our URLs as non trailing slash: /mortgage-calculator

Since remaking certain pages in Gatsby my team have said that Gatsby 'forces' a trailing slash: /mortgage-calculator/

We previously had a server side redirect setup (nginx) to remove the trailing slash.

If you go to the '/' URL version, you get redirected to the non '/':

image

However in the address bar, the URL still has a '/'

image

The fact that this '/' doesn't seem to be getting picked up seems to be confirmed by checking the HTTP status code of the non '/' URLs: /mortgage-calculator

image

Can anyone advise what is happening here? I thought '/' were forced but currently this '/' seems more like a 'cosmetic' '/' and not an actual coded '/'

Apologies in advance for any newbie language and thank you for any help!

freiksenet commented 5 years ago

Hello @pewStocky!

You could try using gatsby-plugin-remove-trailing-slashes for that.

Hopefully this works for you! Thanks for using Gatsby! :purple_heart: :muscle:

pewStocky commented 5 years ago

Hey @freiksenet - thanks for this we'll look into it.

Regardless of whether that does work or not, is there any chance you're able to know why some tools / Google etc see that URL as non '/' even though it shows in the URL?

Would be great to know re learning / knowledge of Gatsby etc.

Thanks for your time / help!

sidharthachatterjee commented 5 years ago

@pewStocky Apologies for missing this!

What you're seeing is intended behaviour as far as I can tell. Let's go through this step by step:

Does that make sense?

Now to solve your issue, I would just use the plugin that @freiksenet linked to and remove the nginx rewrite

pewStocky commented 5 years ago

Hey Sid,

Thank you so much for getting back to me! Super helpful.

Few follow ups:

1: Why does Gatsby add a trailing slash by default? Can this be changed our side to not add it by default?

2: When Gatsby adds the '/' back in (because the URL is different to the canonical) what canonical URL is that looking at? One we've set ourselves?

Also I believe our team tried that plugin but it didn't work...

Thank you for the response though super helpful!

sidharthachatterjee commented 5 years ago

@pewStocky

1: Why does Gatsby add a trailing slash by default? Can this be changed our side to not add it by default?

Gatsby adds the trailing slash by default as a convention because it makes it easy to host sites on hosts that expect a trailing slash for pages in a folder (with an index.html file in the folder). gatsby-plugin-remove-trailing-slashes should remove all trailing slashes for you.

2: When Gatsby adds the '/' back in (because the URL is different to the canonical) what canonical URL is that looking at? One we've set ourselves?

The canonical URL is the one that it generated by default (the one with the trailing slash). You can see these with a query like allPages { nodes { path } } } or by inspecting the page-data.json files in the output directory.

You can set it yourself when creating pages manually using the createPage action (https://www.gatsbyjs.org/docs/actions/#createPage)

In case gatsby-plugin-remove-trailing-slashes isn't working for you, could you please post details on what isn't working and perhaps a minimal reproduction? That'll help us debug the plugin!

kimbaudi commented 5 years ago

@pewStocky - When you say "Gatsby add a trailing slash by default", do you mean during development (i.e., localhost:8000) or on production (yourdomain.com) or both? I'm a bit confused, because GatsbyJS doesn't add trailing slash by default for me during development. However, I am using Netlify to deploy my site and there is a setting to enable "pretty URLs" that will automatically add trailing slash by default for my site on production.

So during development, if I go to localhost:8000/about, it would just take me to localhost:8000/about.

But on production, if I go to mydomain.com/about, it would redirect (301) me to mydomain.com/about/.

In my case, I prefer the pretty URLs and wanted to enable trailing slash by default during development so it matches production. At this point, I'm not able to figure out how to do that.

pewStocky commented 5 years ago

Awesome thank you both, i'll discuss the above with our dev team and see what the situation is, and will try and provide the code to showcase the plugin isn't working etc Will update soon as we can thank you!!!!!

gatsbot[bot] commented 5 years ago

Hiya!

This issue has gone quiet. Spooky quiet. ๐Ÿ‘ป

We get a lot of issues, so we currently close issues after 30 days of inactivity. Itโ€™s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! ๐Ÿ’ช๐Ÿ’œ

dougwithseismic commented 5 years ago

What needs doing right now to move this forward? The unwanted behaviour is that when opening a url without a trailing slash, the browser performs a 301 redirect to the trailing slash url

Whether locally hosted or on netlify, regardless of whether 'gatsby-plugin-remove-trailing-slashes' is used, a 301 redirect is performed.

  1. Visit domain.com/example-url
  2. network 301 referral to domain.com/example-url/ occurs
  3. Browser removes trailing slash artificially Result: domain.com/example-url

image

kuworking commented 5 years ago

For me, without Netlify pretty urls enabled, and without removing trailing slashes in Gatsby, I can see that I can access to both versions of the page, with and without the trailing slash, without any redirection

In development where I am removing the trailing slashes, I can see that the versions with the slashes are not created, but anyway I can access both versions from the browser and get a 200 status (first time it was a 304, after removing the cache I get a 200)

*This created a minnor problem when importing a image with './...' since with a slash or without made a difference

dougwithseismic commented 5 years ago

[Potential Fix] Incredible! After your comment I checked my Netlify settings and indeed, everything is greyed out and looks as though Pretty URLs should be disabled but in fact, isn't.

To do this, you need to go into that option, manually untick Pretty URLs and lock it back up. That removed all traces of 301 for me.

Netlify > Build & Deploy > Post Processing > Asset Optimization > 'Disable Asset Optimization' is ticked. Everything else is greyed out but ticked. If I untick this option, untick Pretty URLs and retick 'Disable Asset Optimization' I no longer have my 301 redirect issue!

So.. can we assume this is a weird UX issue with Netlify?

301 occurs with these settings... image

.. but not with these. image

ximbong commented 5 years ago

@dougwithseismic I think it is most likely a bug from their side, otherwise would be a very questionable UX decision.

Thanks a ton for the help! This works for me!

kuworking commented 4 years ago

@dougwithseismic I somehow have stumped again against this, and I can confirm that making sure that Netlify pretty-urls is truly unclicked as you say effectively removes the redirect problem

Thanks :)

toobulkeh commented 4 years ago

I fixed this issue by removing the plugin remove-trailing-slashes. It doesn't work locally, but it works on netlify.

(localhost on the left, netlify site on the right)

Screen Shot 2019-11-10 at 5 11 11 PM
github-actions[bot] commented 4 years ago

Hiya!

This issue has gone quiet. Spooky quiet. ๐Ÿ‘ป

We get a lot of issues, so we currently close issues after 30 days of inactivity. Itโ€™s been at least 20 days since the last update here. If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open! As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! ๐Ÿ’ช๐Ÿ’œ

github-actions[bot] commented 4 years ago

Hey again!

Itโ€™s been 30 days since anything happened on this issue, so our friendly neighborhood robot (thatโ€™s me!) is going to close it. Please keep in mind that Iโ€™m only a robot, so if Iโ€™ve closed this issue in error, Iโ€™m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else. As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community! ๐Ÿ’ช๐Ÿ’œ

AAverin commented 4 years ago

Was this closed in favour of #9207? Issue is still present and is still causing major SEO issues due to unwanted redirects

lawwantsin commented 4 years ago

This is a problem in AWS as well. Gatsby is a scam.

LoopsGod commented 3 years ago

[Potential Fix] Incredible! After your comment I checked my Netlify settings and indeed, everything is greyed out and looks as though Pretty URLs should be disabled but in fact, isn't. To do this, you need to go into that option, manually untick Pretty URLs and lock it back up. That removed all traces of 301 for me. Netlify > Build & Deploy > Post Processing > Asset Optimization > 'Disable Asset Optimization' is ticked. Everything else is greyed out but ticked. If I untick this option, untick Pretty URLs and retick 'Disable Asset Optimization' I no longer have my 301 redirect issue! So.. can we assume this is a weird UX issue with Netlify? 301 occurs with these settings... image .. but not with these. image

This is unbelievable, this solved my issue. You are a legend. Works for Nuxt.js

jon-sully commented 3 years ago

Greetings all who arrive here from Google ๐Ÿ‘‹๐Ÿป

For starters, please do not disable Pretty URLs on Netlify to solve this issue. That change does not solve this issue and instead serves your site's content on both the trailing slash and non-trailing slash versions of the paths, which tends to cause negative SEO results (duplicate content).

This discussion was continued in the following Discussion thread here in the Gatsby repository and lead to an in-depth analysis on what's going on, why it's happening, and how to work with it:

https://github.com/gatsbyjs/gatsby/discussions/27889#discussioncomment-254854

I strongly encourage newcomers / Google-comers to review that thread before following the above.

toddpadwick commented 1 year ago

Hey @jon-sully i found this via google as having the same issues for our Nuxt site. However, disabling pretty URLs, is in fact the right solution for us, because Nuxt already does this out of the box. but, as you say, for Gatsby that might not be right. i tested a trailing slash url, and even without the pretty url feature, it redirects to without.

jon-sully commented 1 year ago

Greetings! And yeah, good to come back to this one from 2 years ago. I haven't kept up with the trailing slash stuff on Gatsby too much over the last two years but I believe Gatsby's most recent version does actually ship with trailing slash options baked in that render all prior tooling / packages useless? I haven't tested that, but I recall having discussions with the Gatsby team about the matter. YMMV!

toddpadwick commented 1 year ago

thanks @jon-sully . honestly been pulling my hair out (what little I have left) on this for days and finally found that little pretty url feature that was screwing everything up haha. was breaking all our 404s and stripping all query strings out so all the clients PPC ads were breaking.