fix: don't fetch robots.txt from code-bus

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (9796e35) to head (541f163). Report is 1 commits behind head on main.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #595 +/- ## ========================================= Coverage 100.00% 100.00% ========================================= Files 50 50 Lines 4248 4234 -14 ========================================= - Hits 4248 4234 -14 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[github-actions[bot]]

:tada: This PR is included in version 6.10.1 :tada:

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot :package::rocket: