search

voxpupuli / puppet-mongodb

mongodb installation
Apache License 2.0
92 stars 451 forks source link

Roadmap to get this module in better shape #696

Closed h-haaks closed 5 months ago

h-haaks commented 7 months ago

This module needs a few major changes to support later OS versions and mongodb versions.

This issue is created to see if we could gather around a set of smaller tasks to move the module in the right direction.

Feel free to add comments here, and I'll try to make an ordered list of tasks out of it.

Tasks

  1. mongos: init.d scripts to systemd service unit mongob default versions:
    • redhat++ - bump repo version from 3.6 to 4.4
    • debian-10 - bump default version from 4.4.8 to 4.4.29

      698

  2. default to using mongodb repo. Allow only 4.4+ repo versions
  3. bump default repo version to 5.0
  4. add rhel8 and debian11 support
  5. remove debian10 support
  6. cleanup?
  7. release 5.0.0
  8. support installing mongosh
  9. add mongosh provider to custom types
  10. use apt::keyring
  11. add mongodb 6.0 and 7.0 support
  12. add rhel9, debian12 and ubuntu22.04 support
  13. release 5.x.0
  14. remove mongodb 4.x support?
h-haaks commented 7 months ago

Task: Switch to install from mongodb repos as default for all OS distros.

As of now I think only Ubuntu 20.04 have mongodb in their repos, and even Ubuntu 20.04 only have mongodb 3.8 Mongodb is removed in Ubuntu 22.04 Does anyone know SLES?

We have to decide what mongodb major version should be the new default. Look like 5.x is the latest version supporting all platforms without mongosh.

DONE

h-haaks commented 7 months ago

Task: Move repo install from globals to install.

I think of the globals class is more like params, and I don't like it to decalare any resources.

EDIT: decided to keep the globals class and remove all params classes

h-haaks commented 7 months ago

Task: Add support for new OS releases

h-haaks commented 7 months ago

Task: Add support for mongosh

See #677, but I think we should solve this by making side by side providers that use the new command. By doing this we could keep the mongo provider default in the beginning and let users opt in to using the mongosh provider.

The old mongo command is removed in MongoDB 6.x

h-haaks commented 7 months ago

Task: mongodb.conf to yaml

docs https://www.mongodb.com/docs/v4.4/reference/configuration-options/#file-format

EDIT: The existing config is actually yaml. It's just a bit strange looking with dot notation in keys. May be we should take all config as an Hash and use toYAML() in the template? That way the config class params don't limit what could be added to the file.

h-haaks commented 7 months ago

Task: Switch to Apt::Keyring for apt sources

The use of Apt::Key is deprecated in the latest Debian and Ubuntu releases.

DONE

h-haaks commented 7 months ago

Task: Use systemd service file for mongos on all OSes

https://github.com/voxpupuli/puppet-mongodb/blob/master/manifests/mongos/config.pp#L65

DONE

stevenpost commented 7 months ago

Suggested task: Remove params class in favour of module data.

There is currently a mix of module data and the params class, the module should have either one, not both. This just does some cleanup, but keeps things backwards compatible. If you agree that this is a good idea, I get a PR ready.

@h-haaks I'm tasked with upgrading our existing MongoDB setup to version 6.x, it is currently using version 4.4. So I'm currently invested to get this module into shape and get in proper support for 6.x.

DONE

h-haaks commented 7 months ago

@stevenpost I'm kind of in the same spot.

I was thinking about creating a mongodb ( init.pp ) class and move away from globals. I also think the Vox Pupuli practice is to only use hirera data for params that vary based on OS etc.

If you like to help feel free to have a look at it. I'm currently working on bumping minor repo version to 4.4 and fixing outdated configs ...

stevenpost commented 7 months ago

Do you want to keep the module backwards compatible between 4.x and 5.x? I guess not, since it's a major release.

How far back do we need to support old versions of MongoDB?

witjoh commented 7 months ago

Do you want to keep the module backwards compatible between 4.x and 5.x? I guess not, since it's a major release.

How far back do we need to support old versions of MongoDB?

See also https://github.com/voxpupuli/puppet-mongodb/issues/695

As from version 6.0, mongo cli is removed and replaced by mongosh. Mongosh does work with version 4.4.
I haven't encountered any breakage working on this module to try to add support for 5.x and 6.x. But I can't guarantee it.

Moving to mongosh only will be the first big step to get the newer versions working with this module.

Other changes like support for newer OS versions, move away from params.pp will probably not impact mongodb version compatibility ?

Also, mongodb 4.4 reached end of life 2 weeks ago (https://www.mongodb.com/legal/support-policy/lifecycles)

And one can always put in the docs, to use this module current version (v4.2.0) for mongo prior to 5.x.

stevenpost commented 7 months ago

Considering 4.4 has reached EOL, I would drop support for it, and only support 5.0 and up.

h-haaks commented 7 months ago

Oh right. My thought was to drop really old versions at first in a release with updated module dependencies and updated OS support. We could drop 4.4 in the first release I just wanted not to force product upgrade in the first module release in a long time.

witjoh commented 7 months ago

Last release is a bit old, Released Dec 7th 2022.
There are loads of PR merged into master since , we should have indeed a new 4.x release of the module.

Then we can concentrate on 5.x+ only support ..
The 4.x branch can then be only bug fixes maintenance mode ?

h-haaks commented 7 months ago

@witjoh could you do a review of #698?

h-haaks commented 7 months ago

If any of you know apt, feel free to have a look at switching apt::key to apt::keyring. It's ok to include that into the 5.0.0 release if it's ready in time.

stevenpost commented 7 months ago

Oh right. My thought was to drop really old versions at first in a release with updated module dependencies and updated OS support. We could drop 4.4 in the first release I just wanted not to force product upgrade in the first module release in a long time.

I'm looking to update our use the module to the latest commit in master.

I like @witjoh's suggestion of releasing a maintenance 4.x version.

I started looking into all the OS support and versions, yesterday. From what I can tell, no distribution ships a supported release, they either don't ship MongoDB at all (most) or outdated versions (e.g. Debian 10). Dropping support for old versions would significantly simplify things. For example, we could easily change the manage_package_repo parameter from an Optional[Boolean] to a regular Boolean, defaulting to true. Thus getting rid of the special treatment of Debian 10 in the code and tests (that code would break when RHEL 10 gets released next year, btw). I'll start creating PRs next week, probably in draft, so they are available for early review.

I'll see if I have some spare time to tackle the apt keyring issue and test this on a Debian system. My current focus is RHEL.

h-haaks commented 7 months ago

@stevenpost We can't do any more 4.x releases because 'backwards-incompatible' changes has already been merged into master. Have a look at #699 When this is merged I'll add redhat8,debian11 support and remove debian10 which is eol. Then I think we are ready for the 5.0.0 release.

stevenpost commented 7 months ago

@h-haaks I would be very careful with releasing that as a 5.0.0. If we break anything else with the move to mongosh, we need to bump the major version again.

Also Debian 10 is supported by the Debian LTS team till June 30th, see https://wiki.debian.org/LTS. If going with Extended LTS, Jessie is still supported till 2025-06-30, see https://wiki.debian.org/LTS/Extended. I'd keep Debian 10 support as long as MongoDB supports it, and we don't need to do some special cases.

699 looks good to me.

stevenpost commented 7 months ago

Task: mongodb.conf to yaml

docs https://www.mongodb.com/docs/v4.4/reference/configuration-options/#file-format

It already is yaml, see my comment at https://github.com/voxpupuli/puppet-mongodb/pull/706#issuecomment-2009602305

h-haaks commented 7 months ago

You are absolutly right @stevenpost I'll add Debian 11 again in #711 and it should be ok to remove 2.6 from the template name.

h-haaks commented 7 months ago

Seems like @treydock just stepped in and released 5.0.0 two days ago ... #708

h-haaks commented 7 months ago

Task: Add support for Suse enterprise repo

719

h-haaks commented 7 months ago

Task: There seems to be undocumented Amazon support in the module. https://github.com/voxpupuli/puppet-mongodb/blob/b4f0d174243b89b7a67979d3ab973683f7990c38/manifests/params.pp#L44 But in the repo class os family Linux uses Redhat repos.

Switch to amazon repo

720

stevenpost commented 7 months ago

@h-haaks Can you take a look at the acceptance tests for Debian 11 with MongoDB 4.4? They always pass because no test is executed.

h-haaks commented 6 months ago

@h-haaks Can you take a look at the acceptance tests for Debian 11 with MongoDB 4.4? They always pass because no test is executed.

Thats done on purpose because 4.4 is not supported on debian 11 https://www.mongodb.com/docs/manual/administration/production-notes/#platform-support-matrix

I see that some of the 4.4 client tools are available but mongodb-org-server is not https://repo.mongodb.org/apt/debian/dists/bullseye/mongodb-org/4.4/main/binary-amd64/

h-haaks commented 5 months ago

Finally version 6.0.0 is released to forge. As the module now support the latest mongodb versions and the code is in much better shape I see no reason to keep this issue open any more.