nuxt-modules / robots

Tame the robots crawling and indexing your Nuxt site.
https://nuxtseo.com/robots
412 stars 30 forks source link

What's the idea of this module? #1

Closed Zip753 closed 6 years ago

Zip753 commented 6 years ago

Hi! I randomly stumbled upon your repo in the Google search and skimmed through your code, so from what I can tell you're implementing a Nuxt.js module that would generate robots.txt file by passing some basic options there. The question is: couldn't you just write that robots.txt by hand and place it in static/robots.txt? Or the idea is that you can provide some runtime-dependent values in nuxt.config.js and have robots.txt dynamically generated for you?

WilliamDASILVA commented 6 years ago

@Zip753 Indeed, it would be better and easier if I wanted a static robots.txt to create one by hand and place it in the static folder. Since it was not the case for me in my project, I created this modue to inject some variables and, as you said, dynamically generate that file like you would do for a /sitemap.xml. I tought I would share it, maybe someone would be happy to stumble on it during his stackoverflow searches. Who knows...

Basic usage:

{
  modules: [
    'nuxt-robots',
 ],
  'nuxt-robots': {
    Disallow: '/admin',
    ...
  },
}

Inject whatever you want, from anywhere:

const someGeneratedPathsHere = [
  '/admin', '/users', ...
];
...
{
  modules: [
    'nuxt-robots-module',
 ],
  'nuxt-robots-module': {
    UserAgent: 'Googlebot',
    Disallow: someGeneratedPathsHere,
    ...
  },
}

I may update the readme to make it clear Hope I'm clear enough on my explanation :)