-
# Rudder
On détermine la liste des liens à explorer via le premier lien que l'on donne puis par les autres liens que le parser trouvera.
Utilisation du fichier "robots.txt" ?
# Parser
L'ob…
-
i am parsing robots.txt to retrieve sitemapindex and then url in sitemap, when parsing i have those warn for each url in sitemap but it is normal url are note under http://.../sitemap and moreover sit…
-
Hi
What is the language in the readme file and why? Why not to use a more common language, like english?
Also, I see there is a 2.0 version but is not the same as the one in the master. Why?
…
-
Redmine 404 error
body{
font-family: Trebuchet MS,Georgia,"Times New Roman",serif;
color:#303030;
margin:10px;
}
h1{
font-size:1.5em;
}
p{
font-size:0.8em;
}
Page not found
The page you were…
-
certbot-nginx 0.9.3
OS: Ubuntu 14.04
Command: `~/certbot-auto --nginx -d domain.ru -d www.domain.ru`
Actual output: `Cannot find a VirtualHost matching domain domain.ru.`
Domain config:
```
se…
DeKaN updated
7 years ago
-
When upgrading to 2.13, running `ember init` removes the `bower.json` file without prompting despite it containing dependencies.
Steps to reproduce:
`cat bower.json`:
```
{
"name": "orange-…
-
Hi, with a brand new Phoenix project I am experiencing an error where the app is unable to find the Ecto.Adapters.Postgres.Connection module. I was able to build and compile (and run) a Phoenix app ye…
-
Hi, with a brand new Phoenix project I am experiencing an error where the app is unable to find the Ecto.Adapters.Postgres.Connection module. I was able to build and compile (and run) a Phoenix app ye…
-
Hi,
Thanks for such an amazing library. I have a question and I hope to get some answers or suggestions. Is there any way to set the robots option to obey rules which are specified for a specific b…
-
From Google Robots.txt Specifications [Order of precedence for group-member records](https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt?hl=en#google-supported-non-group-membe…