fix robots.txt generation (#282)

This resolves #282.

We now append a complete block including user-agent to the robots.txt, so that we always generate valid results, independent of the previously added content:

User-agent: *
Disallow: */cache/cachify/

Without any other plugin, this results in:

User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

Sitemap: http://example.com/wp-sitemap.xml

User-agent: *
Disallow: */cache/cachify/

We also add line breaks before and after this block, because we should not rely on the existence of such be previous/following logic out of our control.

Additional change: In #190 we apparently switched from a "robots_txt" action (which is not a valid action) to the do_robots action. Actually the first is a filter and I think we should prefer the filter logic, so we switch (back) to robots_txt. Also added a test case for this.

pluginkollektiv / cachify

fix robots.txt generation (#282) #283