search

oqtane / oqtane.framework

CMS & Application Framework for Blazor & .NET MAUI
http://www.oqtane.org
MIT License
1.87k stars 541 forks source link

Sitemap caching for added performance with large sites #2748

Closed thabaum closed 1 year ago

thabaum commented 1 year ago

The sitemap could use caching to help with large sites with search engines crawling.

sbwalker commented 1 year ago

Caching without cache invalidation support will actually cause more problems than benefits as it will not keep the site map results aligned with the actual site configuration. This could even result in security issues if a page was secured but the site map cache was not refreshed. In addition, the general recommendation is to only optimize for performance after application performance has been measured and a performance bottleneck has been identified.

thabaum commented 1 year ago

cache invalidation was something I was trying to discuss. When a page or module is updated this is when sitemap cache should be invalidated and re-cached. Also a throttle for if things are being updated a lot in short period of time. So maybe a flag would be thrown possibly and if it is true that something has updated cache is invalidated and the sitemap is rebuilt on next visit. I was thinking about an option to turn this on/off as smaller sites would not need this feature for just a few pages. Also a button to clear sitemap cache.

thabaum commented 1 year ago

For purpose of security of an HTTP link, shouldn't Oqtane (if set to) redirect all pages to SSL pages in the event anyone forgets to make a link secured even if it was a sitemap page? I don't know if this is something that is set in an "SSL Enabled" or "SSL Enforced" like dnn does to help keep all links SSL. Only time not to is maybe when you have a back end that has load balancing. Even then you are recommended to SSL encrypt now days. But for speed some sacrifice security. I am not yet 100% familiar with this feature in Oqtane yet. Has anyone used it with a load balancer yet?

Not in any disagreement with invalidating cache as this is very important other than a 1 hour time limit which is not a correct implementation for Oqtane. I left that PR that way just to get the ball rolling here while playing with Oqtane caching. I have read that caching sitemaps is important for speed. Also as complicated as Oqtane is; for developers of modules there can be many sitemaps per module such as a shopping cart system for listing products or product categories. For example a sitemap address of www.oqtane.org/blog/sitemap should potentially be considered I believe, correct me if wrong as maybe Sitemap.org is obsolete.

sbwalker commented 1 year ago

The goal in Oqtane is to keep the framework as lightweight and simple as possible - so at this point we will not include caching for the sitemap.xml. Realistically it gets called very infrequently so there is no need to cache it... and caching would add a lot more complexity to account for invalidation requirements.

Oqtane will not support the DNN SSL features. Proper separation of concerns indicates that SSL should be handled by the environment - not the framework.

Oqtane already has the ability to include Urls from Modules in the sitemap.xml - follow this link to see it in action: https://www.oqtane.org/pages/sitemap.xml (all of the blog post Urls are included in the SiteMap). The implementation is here: https://github.com/oqtane/oqtane.blogs/blob/master/Server/Manager/BlogManager.cs

thabaum commented 1 year ago

Since this is not something to be included at this point in Oqtane Framework due to the complexities it would most likely create I will close this as an issue for now. I completely agree with keeping Oqtane as lean and simple as possible for developers to build upon.