fix: use system environment variables for robots

[chanceaclark]

What/Why?

Uses the deployment URL for robots.txt.

Testing

https://catalyst-latest-git-fix-robots-bigcommerce-platform.vercel.app/robots.txt

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
catalyst	🔄 Building (Inspect)	Visit Preview	💬 Add feedback	Jul 9, 2024 9:30pm
catalyst-latest	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 9, 2024 9:30pm

5 Skipped Deployments

| Name | Status | Preview | Comments | Updated (UTC) | | :--- | :----- | :------ | :------- | :------ | | **catalyst-1millionproducts-store** | ⬜️ Ignored ([Inspect](https://vercel.com/bigcommerce-platform/catalyst-1millionproducts-store/5wZgvbuQBiBBxiDSQtAscXpNhXwb)) | [Visit Preview](https://catalyst-1millionproducts-store-git-eb6ba2-bigcommerce-platform.vercel.app) | | Jul 9, 2024 9:30pm | | **catalyst-au** | ⬜️ Ignored ([Inspect](https://vercel.com/bigcommerce-platform/catalyst-au/5Gs8W8WzLAqta4qV53PMUAMohdat)) | [Visit Preview](https://catalyst-au-git-fix-robots-bigcommerce-platform.vercel.app) | | Jul 9, 2024 9:30pm | | **catalyst-test-store** | ⬜️ Ignored ([Inspect](https://vercel.com/bigcommerce-platform/catalyst-test-store/CMb6hwsrYP5GtLJZGPdLBhsKYk8S)) | [Visit Preview](https://catalyst-test-store-git-fix-robots-bigcommerce-platform.vercel.app) | | Jul 9, 2024 9:30pm | | **catalyst-uk** | ⬜️ Ignored ([Inspect](https://vercel.com/bigcommerce-platform/catalyst-uk/GBw6iBSfLevr6SCa1i2sxUKcmtQc)) | [Visit Preview](https://catalyst-uk-git-fix-robots-bigcommerce-platform.vercel.app) | | Jul 9, 2024 9:30pm | | **catalyst-unstable** | ⬜️ Ignored ([Inspect](https://vercel.com/bigcommerce-platform/catalyst-unstable/5h8YZg3fWkpZji2PJKqimCKNUgaR)) | [Visit Preview](https://catalyst-unstable-git-fix-robots-bigcommerce-platform.vercel.app) | | Jul 9, 2024 9:30pm |

[changeset-bot[bot]]

🦋 Changeset detected

Latest commit: 377d8ef3e6bffaeed300e774c8b3ff16ad46e88f

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

| Name | Type | | -------------------------- | ----- | | @bigcommerce/catalyst-core | Patch |

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[github-actions[bot]]

⚡️🏠 Lighthouse report

Lighthouse ran against https://catalyst-latest-326v6rgkk-bigcommerce-platform.vercel.app

🖥️ Desktop

We ran Lighthouse against the changes on a desktop and produced this report. Here's the summary:

Category	Score
🟠 Performance	87
🟢 Accessibility	100
🟢 Best practices	96
🟠 SEO	82

📱 Mobile

We ran Lighthouse against the changes on a mobile and produced this report. Here's the summary:

Category	Score
🟢 Performance	91
🟢 Accessibility	100
🟢 Best practices	96
🟠 SEO	85