The c-deps dependencies are outdated and do not get periodic refreshes

[knz]

Found by @alan-mas in https://github.com/cockroachdb/protobuf/pull/1#issuecomment-831549472

While our Go package dependencies get some amount of scrutiny (we have a healthy stream of updates and dep bumps to go.mod), our c-deps dependencies, handled as Git submodules, do not get this treatment.

For example, our protobuf dep (submodule from github.com/cockroachdb/protobuf) is fairly old, and we haven't brought krb5, jemalloc, snappy and cryptopp forward in time either.

@petermattis @bdarnell can you think of reasons why we need to anchor these dependencies to the particular versions we chose back then?

@kenliu-crl this is linked to the convo we had about the need to formalize our periodic dependency checks. In fact, I'm pretty sure that the c-deps do not get any amount of security checks today via our dependency checkers.

Jira issue: CRDB-7185

[bdarnell]

can you think of reasons why we need to anchor these dependencies to the particular versions we chose back then?

We are pinned to an old version of jemalloc because of #17013.

Other than that, we should generally be moving these forward. Similar to our other deps, I'd aim for one mass upgrade per release cycle and keep an eye on CVEs. (Is there any way to make these deps visible to snyk or other tools we use?)

For the record, snappy and cryptopp are no longer present in the master branch. The current list is geos, jemalloc, krb5, libedit, proj, and protobuf.

[rickystewart]

This doesn't appear to be a dev-inf issue -- updating arbitrary dependencies regularly isn't clearly under the purview of dev-inf, nor is it related to any other work the dev-inf team does. Seems like something the product team should be aware of? I don't know who would be appropriate to spearhead that effort outside of the people already in this thread.

[knz]

Thanks for your answer, but I do find it mildly worrying that:

• this is distinctly an engineering-centric issue (i.e., not something that a PM has much to say about)
• we're observing (through this convo) that no eng team is owning this process.

This is a process problem (specifically, lack thereof) and probably needs to be managed accordingly. Who's the manager in line here? Is that @kenliu-crl? Or maybe something @petermattis has a say about?

[knz]

I could (hypothetically) imagine a world where each of the individual c-deps is managed by the team that is most closely dependent on it:

• geos, proj: SQL experience
• jemalloc: storage (or server?)
• krb5: server

However:

• libedit: no owner currently (we still have no team for CLI stuff generally) (However this particular case is a no-op because the upstream hasn't changed)
• protobuf: no clear owner, although we do want an upgrade

[lunevalex]

Discussed with @kenliu-crl offline the current plan of record is that dev-inf will own identifying and documenting ownership for each dependency in c-deps. Once that happens it will be the responsibility of each owning team to maintain and update their dependencies going forward.

[github-actions[bot]]

We have marked this issue as stale because it has been inactive for 18 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to CockroachDB!