-
Hi,
I have set a rule in robots.txt that
Disallow: /mailto:%20iasdf%66o%40r%65%69asdfdf%2ede
Disallow: /news-letter/unsub
And started the cron job to index but the job always indexed the abov…
-
Once we can generate 3D shapes #16 we should be able to adapt the CPPNs to create soft robots. I haven't done this before, but others have:
http://jeffclune.com/publications/2013_Softbots_GECCO.pdf
…
-
```
Your Name:
Overview of Lab Comments: i.e. Was this lab good or bad? Too fast? Too Slow? Does it need something new?
--- For each specific Issue (Copy this section as many times as necessary) ---…
lmock updated
9 years ago
-
Необходимо обновить robots.txt
1.For lykke.com
[robots.txt](https://github.com/LykkeCity/LykkePublicWebsite/files/1075069/robots.txt)
2.For streams.lykke.com
[robots.txt](https://github.com/L…
-
My suggestion, or attempt:
1. Add metatag name "robots" to template that includes head, and replace pageProperty as follows:
This works as soon as I manually enter a page_properties …
-
would be nice if gateways handled:
```
> curl 127.0.0.1:8080/robots.txt
User-agent: *
Allow: /
```
notes:
- allow or disallow API?
- this should be optional (config var?)
- this should redirect rob…
-
```
What steps will reproduce the problem?
1. put Disallow: /example/ in robots.txt
2. or put /example/ in Excluded URL patterns
3. what it works if you put /example/* in Excluded URL patterns
Wha…
-
```
What steps will reproduce the problem?
1. put Disallow: /example/ in robots.txt
2. or put /example/ in Excluded URL patterns
3. what it works if you put /example/* in Excluded URL patterns
Wha…
-
```
What steps will reproduce the problem?
1. put Disallow: /example/ in robots.txt
2. or put /example/ in Excluded URL patterns
3. what it works if you put /example/* in Excluded URL patterns
Wha…
-
```
What steps will reproduce the problem?
1. put Disallow: /example/ in robots.txt
2. or put /example/ in Excluded URL patterns
3. what it works if you put /example/* in Excluded URL patterns
Wha…