-
## Issue Description
The documentation containing instructions to edit permalinks is too technical and many hosting platforms now limit SSH access. The new documentation should show users how to use …
imgnx updated
12 months ago
-
- add "no index" to search pages if a parameter is not affecting the keywords in output such as: vocalized, script, view...
- add "no follow" to clickable links that should not be indexed, like Juz:N,…
-
This group meets during the second half of office hours each week. The TODO list in this top message will be updated regularly to reflect current progress.
**TODOs:**
- [ ] Change/add Drupal ling…
-
From Terry 5/3/2024
@SJPitman brought up an issue about the DCE essays not being indexable and findable on Google. I've been looking into it and think we need to address it. Has to do either with Rea…
-
Define a robots.txt file to help search crawlers better understand our site and prevent them from crawling unnecessary links.
More information: https://moz.com/learn/seo/robotstxt
https://develo…
-
I noticed that CeWL doesn't follow subdomains.
`cewl http://www.domain.com `
does not traverse into http://sub.domain.com
`cewl http://domain.com `
does not work. Neither does
`cewl ht…
-
I want to add twitter card meta tag for my project using react-helmet,
I need help how to add the meta tag to render dynamic data in ReactJS
https://developer.twitter.com/en/docs/tweets/optimize-wit…
-
YAJ listing pages don't last very long (unsure how long eBay ones last), and aucfan/the billion other proxy sites that save YAJ pages aren't guaranteed to keep them.
This could be extended for *every…
-
One nice property of file systems is the timestamp on files, it helps to decide which sites are more important to recrawl. The crawler return additional metadata such as the date of the newest file(s)…
riptl updated
5 years ago
-
The content of a Whizu page is mostly generated on the server side and injected in the client with JavaScript. It should be verified that Google supports this and can index the generated content. If n…