Docs: mention user-agent default value

[benjimin]

Feature description

The docs for GDAL_HTTP_USERAGENT should say what the default value is.

Additional context

6376 added a default value for the HTTP user agent header used by GDAL. According to comments in the merged code, the default HTTP user agent is GDAL/x.y.z (denoting the version). The configuration docs should be updated to reflect this.

This is important because knowing the user agent header string is necessary to correctly configure firewalls, to allow legitimate access to data collections while blocking spurious scraping by generic web crawlers and AI trainers. (Note some major hosting providers such as AWS charge for egress, not ingress, so significant costs can be inadvertently incurred by permitting general machine access which isn't expected to be making productive use of the structured content of the geospatial datasets.)

[jratike80]

Doxygen documentation about the thing https://gdal.org/doxygen/cpl__http_8h.html#ae83c06d51089bb340ed0b98a6ed8d01d.

[benjimin]

Here is an extract from the current user docs. Note that defaults are expressed for other configuration options but not for this one:

https://gdal.org/user/configoptions.html

• _GDAL_HTTP_USERAGENT=value: When set this string will be used to set the User-Agent header in the http request sent to the remote server._
• _GDAL_HTTP_UNSAFESSL=[YES/NO]: Defaults to NO. Set to "YES" to get the curl library to skip SSL host / certificate verification._

I propose appending the sentence: "Defaults to GDAL/x.y.z where x.y.z is the current version.", and removing the "When set" clause.

[jratike80]

Would you like to try the "Edit on GitHub" button in the top-right corner of the page and create a pull request?

[jratike80]

I made a quick code search and found some potentially different ways to define the user agent:

WMTS: GDAL WMS Driver, can be set in the XML file ogr_geocoding: what is this? Defaults to GDAL/OGR version string ECW: what is this? NCSecwSetConfig(NCSCFG_ECWP_CLIENT_HTTP_USER_AGENT, "ECW GDAL Driver/" NCS_ECWJP2_FULL_VERSION_STRING_DOT_DEL); ogramigocloud: what is this? userAgent << "USERAGENT=gdal/AmigoCloud build:"