[Comment] Dynamically Generating Sitemap.xml for ASP.NET MVC

[RehanSaeed]

https://rehansaeed.com/dynamically-generating-sitemap-xml-for-asp-net-mvc/

[RehanSaeed]

judasegg commented on 2015-09-22 15:04:03

Regarding this sitemap generator - using your boilerplate, if I go to localhost/sitemap.xml I see the sitemap, but have no physical sitemap.xml file in the solution itself. Can you clarify - because this is MVC, that is the reason that we don't have a physical sitemap.xml file? Unless I'm missing some generation step to create an actual file at the root. It just seems a bit strange that robots.txt is a physical file, but the sitemap appears to simply be a route?

[RehanSaeed]

Muhammad Rehan Saeed commented on 2015-09-22 15:13:52

Regarding this sitemap generator - using your boilerplate, if I go to localhost/sitemap.xml I see the sitemap, but have no physical sitemap.xml file in the solution itself. Can you clarify - because this is MVC, that is the reason that we don't have a physical sitemap.xml file? Unless I'm missing some generation step to create an actual file at the root. It just seems a bit strange that robots.txt is a physical file, but the sitemap appears to simply be a route?

We are dynamically generating the sitemap.xml file instead of having to have a physical file. The advantage is that the sitemap is always up to date and we don't need to continually generate a new physical file and replace the old one.

Actually, if you are using ASP.NET Core Boilerplate, the robots.txt file is also generated dynamically. There is no physical robots.txt file. There is less of a reason to do this. Mostly it is to create the link to the sitemap, which you would otherwise have to manually add. You can read my other recent blog post about this subject.

[RehanSaeed]

Tobias Oort commented on 2015-09-28 11:58:47

I would suggest adding some caching - even generating the sitemap build time instead of runtime. This greatly reduces impact when your site gets spidered by a few search engine (and Asp.net MVC afaik doesn't really let you change the routing of the app at runtime, so runtime resolution of the sitemap is relatively useless). Also, you might want to use Attributes to annotate ControllerActions to either in/exclude stuff from the sitemap.

[RehanSaeed]

Muhammad Rehan Saeed commented on 2015-09-28 19:18:42

I would suggest adding some caching - even generating the sitemap build time instead of runtime. This greatly reduces impact when your site gets spidered by a few search engine (and Asp.net MVC afaik doesn't really let you change the routing of the app at runtime, so runtime resolution of the sitemap is relatively useless). Also, you might want to use Attributes to annotate ControllerActions to either in/exclude stuff from the sitemap.

Agreed! I didn't discuss it in this blog post but I have added caching to the ASP.NET Core Boilerplate project template. The only problem is that you can't use the OutputCache attribute because if you are using sitemap index files, then you want each sitemap to be cache busted at the same time. Instead, I've gone with using a memory cache internally.

Attributes can't go to the database and get the ID's of all items, which you can then use to generate routes for all of those items, they can only be used for really basic static routes without any database referenced parameters. You could also generate a static sitemap but then you would have to generate this file every time your database changes or have some kind of job running to create the file periodically.

Unless you have a very high traffic site, generating a sitemap once a day and then caching it, is not going to affect performance that much for most people. If it is, you are probably running a site the size of StackOverflow.

[RehanSaeed]

Goldenboy commented on 2015-10-14 12:20:28

Wow. Very Nice tutorial. I implemented this and it works fine with attribute routing. But when i tried to use convention based routing, i was getting the error of "A route named 'Home/Index' could not be found in the route collection.".. Please, is there any work around i need to do to get this to work with the convention based routing?

[RehanSaeed]

Muhammad Rehan Saeed commented on 2015-10-14 15:08:56

Wow. Very Nice tutorial. I implemented this and it works fine with attribute routing. But when i tried to use convention based routing, i was getting the error of "A route named 'Home/Index' could not be found in the route collection.".. Please, is there any work around i need to do to get this to work with the convention based routing?

I'm not sure why you are getting an error without taking a look at your code. Can I suggest posting your question along with some code on StackOverflow, it's the best place to get answers quickly.

By the way, ASP.NET Core has attribute routing turned on by default and it seems to now be the default. I definitely prefer it but everyone has their own preference.

[RehanSaeed]

Jack Chen commented on 2015-11-10 10:13:16

Thanks. I learned a lot.

[RehanSaeed]

luke commented on 2016-08-05 09:38:59

Hi Rehan, I'm trying to convert you example in VB. To do that I need a working example, can you help? Thank you very much.

[RehanSaeed]

Muhammad Rehan Saeed commented on 2016-08-05 09:43:44

Hi Rehan, I'm trying to convert you example in VB. To do that I need a working example, can you help? Thank you very much.

Create a new project using the ASP.NET Core Boilerplate project template. You can get it from the Visual Studio Gallery.

[RehanSaeed]

amir commented on 2016-11-26 15:38:08

hello

I want to generate my all post in site map.

for example :

mysite.com/news/1
mysite.com/news/2
mysite.com/news/newsid

how should i do ?

[RehanSaeed]

Muhammad Rehan Saeed commented on 2016-11-27 07:50:52

hello

I want to generate my all post in site map.

for example :

mysite.com/news/1
mysite.com/news/2
mysite.com/news/newsid

how should i do ?

Does the blog post not help? What is the problem?

[RehanSaeed]

amir commented on 2016-11-29 06:57:13

Does the blog post not help? What is the problem?

My project is Database First.

how should I put my news id in sitemap and punt it in the loop ?

you just create sitemap for Home, About and Contact

[RehanSaeed]

amir commented on 2016-11-30 16:48:17

Hello

please put project files in your post

I want to download it

thanks

[RehanSaeed]

amir commented on 2016-12-03 16:06:46

Hello

please put project files in your post

I want to download it

thanks

[RehanSaeed]

Muhammad Rehan Saeed commented on 2016-12-05 13:41:57

Hello

please put project files in your post

I want to download it

thanks

Create a project using the ASP.NET Core Boilerplate project template and you can look at the code there.

[RehanSaeed]

amir commented on 2016-12-13 07:42:02

Hello I install it and i create GetSitemapNodes function

but when i pass a url to Url , is show this error :

Property or indexer Url cannot be assigned to - it is read only

for example :

new SitemapNode()
    {
        Url = urlHelper.AbsoluteRouteUrl("Home"),
        Priority = 1
    });

[RehanSaeed]

Muhammad Rehan Saeed commented on 2016-12-17 10:44:38

Hello I install it and i create GetSitemapNodes function

but when i pass a url to Url , is show this error :

Property or indexer Url cannot be assigned to - it is read only

for example :

new SitemapNode()
{
    Url = urlHelper.AbsoluteRouteUrl("Home"),
    Priority = 1
});

I think you need to spend some time learning C#. There are really good courses on Pluralsight which are free to view for 30 days.

[RehanSaeed]

Abdulrahman commented on 2016-12-17 20:45:32

Wouldn't it be more dynamic, if you would generate your sitemap through reflection? In your solution you have to update the notes by yourself, if anything changes or will be added.

[RehanSaeed]

Muhammad Rehan Saeed commented on 2016-12-21 09:08:00

Wouldn't it be more dynamic, if you would generate your sitemap through reflection? In your solution you have to update the notes by yourself, if anything changes or will be added.

What if you have products in a database. Each product has an ID. To generate the URL for each product, you need to get all product ID's from the database and iterate over them to get the product URL's. This cannot be done using reflection.

[RehanSaeed]

John commented on 2017-02-16 16:54:47

Even though I have added

new XDeclaration("1.0", "utf-8", "yes"),

when calling the sitemap.xml I do not see the beginning of the xml as in an xml file

Is that a problem for search engines

[RehanSaeed]

Muhammad Rehan Saeed commented on 2017-02-18 11:41:51

Even though I have added

new XDeclaration("1.0", "utf-8", "yes"),

when calling the sitemap.xml I do not see the beginning of the xml as in an xml file

Is that a problem for search engines

You are mistaken, the XML declaration is not shown in Firefox but if you save the document and view the content, you will see it.

[RehanSaeed]

abdelmonem Mohammad commented on 2017-03-23 15:20:58

What about submitting the sitemap file to google? It asks about the location of the sitemap of your site.

[RehanSaeed]

Muhammad Rehan Saeed commented on 2017-03-23 16:27:17

What about submitting the sitemap file to google? It asks about the location of the sitemap of your site.

That is where the sitemap pinger comes in. Here is some code that does just that.

[RehanSaeed]

abdelmonem Mohammad commented on 2017-03-24 09:03:34

That is where the sitemap pinger comes in. Here is some code that does just that.

Allah bless you. How generous you are! Two years ago you wrote this and still respect people asking you. Forgive me for asking again as I just used boilerplate that you wrote about. Should I submit the sitemap myself or just direct google to the root of the site. I read in google site that it is preferable to google to have that dynamic sitemap as well as a file in the root directory. The dynamic one will notify any update apart from the organic one that has all the URLs. Once again Allah bless you and your family for your kindness.

[RehanSaeed]

Muhammad Rehan Saeed commented on 2017-03-27 16:53:13

Allah bless you. How generous you are! Two years ago you wrote this and still respect people asking you. Forgive me for asking again as I just used boilerplate that you wrote about. Should I submit the sitemap myself or just direct google to the root of the site. I read in google site that it is preferable to google to have that dynamic sitemap as well as a file in the root directory. The dynamic one will notify any update apart from the organic one that has all the URLs. Once again Allah bless you and your family for your kindness.

You don't need a static sitemap if you have a dynamic one. I never got around to writing a blog post about submitting (pinging) your sitemap URL to Google and Bing through code. You could do it manually if it doesn't change very often. Hope that helps!

[RehanSaeed]

Sergiu commented on 2017-05-05 12:02:03

Hi Rehan, I get an 404 error not found for https://mysite.com/sitemap.xml in MVC5 even if I added the next lines:

In HomeController.cs I have this:

[Route("sitemap.xml")]
public ActionResult SitemapXml(int? index = null)

Can you tell what is wrong? Thanks, Sergiu

[RehanSaeed]

Sergiu commented on 2017-05-05 12:04:34

I was referring to handler in web.config (add name="SitemapXml" path="sitemap.xml" verb="GET" type="System.Web.Handlers.TransferRequestHandler" preCondition="integratedMode,runtimeVersionv4.0")

[RehanSaeed]

Muhammad Rehan Saeed commented on 2017-05-05 14:42:47

I was referring to handler in web.config (add name="SitemapXml" path="sitemap.xml" verb="GET" type="System.Web.Handlers.TransferRequestHandler" preCondition="integratedMode,runtimeVersionv4.0")

If you've added the section I describe in the web.config correctly, then you should not get a 404. Maybe check for typos?

[RehanSaeed]

Kemal Can ÖZÇELİK commented on 2017-05-20 20:11:33

You saved my day. Thank you.

[RehanSaeed]

Erdem Köşk commented on 2017-06-02 14:02:52

I want to say that thanks for amazing tutorial. You saved my day.

[RehanSaeed]

Anthony commented on 2017-07-22 00:15:04

how about multi-language sitemap

http://www.example.com/deutsch/

[RehanSaeed]

Muhammad Rehan Saeed commented on 2017-07-23 12:42:21

how about multi-language sitemap

http://www.example.com/deutsch/

Is there some special consideration for multi-language sitemaps? The sitemaps.org doesn't list anything.

[RehanSaeed]

Robert Swilley commented on 2017-07-28 20:46:11

This was very helpful to me. Thanks!

[RehanSaeed]

MrNams commented on 2018-12-01 07:18:16

How to write sitemap at location \wwwroot\sitemap.xml? I have developed utility to generate sitemap for my website mrnams.com Problem I have is, when code generate sitemap.xml and try to save in \wwwroot\sitemap.xml,then its fails with access denied. As I am using plesk, I can't access IIS, also I don't want to give write permissions to all other static files. Please help.

[RehanSaeed]

Muhammad Rehan Saeed commented on 2018-12-01 11:01:50

How to write sitemap at location \wwwroot\sitemap.xml? I have developed utility to generate sitemap for my website mrnams.com Problem I have is, when code generate sitemap.xml and try to save in \wwwroot\sitemap.xml,then its fails with access denied. As I am using plesk, I can't access IIS, also I don't want to give write permissions to all other static files. Please help.

Why are you trying to save the sitemap to the wwwroot folder? Just serve it in the response body.

[RehanSaeed]

MrNams commented on 2019-01-22 13:51:29

Why are you trying to save the sitemap to the wwwroot folder? Just serve it in the response body.

Thank you for valuable reply Is it ok for Search Engine to serve dynamically. also how to prevent hackers who programatically call sitemap and keep servers busy. Generally we use captcha to avoid robotics filling forms. But here we can't use captcha.

Thanks in advance.

[RehanSaeed]

Mesut commented on 2019-10-07 18:37:14

Hi Rehan, I get an 404 error not found for https://mysite.com/sitemap.xml in MVC5 even if I added the next lines:

In HomeController.cs I have this:

[Route("sitemap.xml")]
public ActionResult SitemapXml(int? index = null)

Can you tell what is wrong? Thanks, Sergiu

you should check your webconfig file. He had told, you should add ""

[RehanSaeed]

larrybud commented on 2020-03-28 13:58:35

this really isn't creating a "dynamic" sitemap in that the pages aren't automatically detected. You still need to manually add the path to the page into the code.

[RehanSaeed]

Rashmi commented on 2020-04-09 16:09:18

Our site is multilingual website. We have almost 15 languages. For eg: www.xxx.com/en-us www.xxx.com/fr-fr and so on.. How do I generate sitemap files dynamically for different language urls?