The company I work for has a lot of user generated content, and we generate sitemaps for these and upload them to S3. When users search for our content on google, they often stumble across our sitemaps. We would like for this to not happen.
I found this SO thread that explains how this can be done, which is by specifying the X-Robots-Tag as a header of the S3 object. We tried supplying this data, but we soon realized only these metadata options are allowed.
The Request
We would like to have the X-Robots-Tag extra param available for the upload. This way google won't serve our sitemaps on search.
We use boto3==1.7.48, and it looks like it uses the latest of this library. See here.
The Problem
The company I work for has a lot of user generated content, and we generate sitemaps for these and upload them to S3. When users search for our content on google, they often stumble across our sitemaps. We would like for this to not happen.
I found this SO thread that explains how this can be done, which is by specifying the
X-Robots-Tagas a header of the S3 object. We tried supplying this data, but we soon realized only these metadata options are allowed.The Request
We would like to have the
X-Robots-Tagextra param available for the upload. This way google won't serve our sitemaps on search.We use
boto3==1.7.48, and it looks like it uses the latest of this library. See here.