search

mongodb-labs / drivers-evergreen-tools

Scripts for MongoDB drivers to bootstrap their Evergreen configuration file - This Repository is NOT a supported MongoDB product
10 stars 64 forks source link

DRIVERS-3002 Use mongodl.py in download-mongodb.sh with starts-with version comparison #529

Closed eramongodb closed 1 day ago

eramongodb commented 1 day ago

Reattempt of https://github.com/mongodb-labs/drivers-evergreen-tools/pull/525.

Extends the version comparison operator in the DB query of matching download entries from an exact string comparison (version=:version) to include a "starts-with" pattern match expression using the LIKE operator. This permits version="5.1" to match "5.1.Z", but it will not match a hypothetical "X.5.0" or "5.10.Z". Similarly, version="5" will match version="5.Y.Z", but will not match a hypoethetical "50.Y.Z". A full version number version=5.1.0 will still match "5.1.0" exactly using the existing exact comparison operator.

The extra version_pattern=f'{version}.% argument is necessary to workaround syntax errors when the % is embedded in the query string itself, e.g. writing version=:version OR version LIKE :version.% in the query string itself raises an exception (various attempts to escape the % character failed):

  File ".evergreen/mongodl.py", line 324, in __call__
    return self._cursor.execute(query, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sqlite3.OperationalError: near ")": syntax error
eramongodb commented 1 day ago

The "starts-with" pattern also addresses the handling of version numbers without a patch version number (e.g. 5.0). The ORDER BY version query operator selects the newest patch version number available given the query parameters. The SUPPORTED_VERSIONS injection is therefore no longer necessary.

eramongodb commented 1 day ago

Some error messages added in https://github.com/mongodb-labs/drivers-evergreen-tools/commit/9142b7055ea5940e59ad41c4b069376f867031da#diff-10255c591aa6273801d14cd261ba71498474b9b21ba02aee1df67e6e9cd60a2fR470 (relating to the DISTRO_ID_TO_TARGET map) were not being printed to stderr, thus leading to the resulting error message being treated as part of the download URL. Failure to account for word splitting in expansions of the resulting Bash variable(s) in subsequent curl commands resulted in some awful behavior. Both issues have been fixed. The current list of missing targets is now output properly (+ nicer formatting) without affecting the contents of the download URL.

The missing DISTRO_ID_TO_TARGET entries (and correponding failure of test-mongodl-full tasks) may be addressed by a followup PR.

blink1073 commented 1 day ago

New error:

[2024/10/24 15:41:55.151] Missing targets in DISTRO_ID_TO_TARGET:
[2024/10/24 15:41:55.151]  - linux_i686
[2024/10/24 15:41:55.151]  - linux_x86_64
[2024/10/24 15:41:55.151]  - osx
blink1073 commented 1 day ago

I think they need to be added to this list: https://github.com/mongodb-labs/drivers-evergreen-tools/blob/1f2e86e253b8e834616d55430c37f3b616680f1f/.evergreen/mongodl.py#L426

blink1073 commented 1 day ago

Ah, I see your comment. I agree, let's merge and iterate. Thank you!