search

prestodb / presto

The official home of the Presto distributed SQL query engine for big data
http://prestodb.io
Apache License 2.0
16.08k stars 5.39k forks source link

Proposal: Upgrade Presto to support Java 17+ #23785

Open imjalpreet opened 1 month ago

imjalpreet commented 1 month ago

Background

As of today, Presto primarily supports Java 8/11 in most production environments. However, with Java 17/Java 21 being a Long-Term Support (LTS) release, it’s becoming increasingly important to evaluate and potentially upgrade Presto’s Java version to take advantage of the modern JVM features, performance improvements, and security enhancements that Java 17 offers.

Many organizations are planning their migrations to Java 17+ or have already made the switch, which raises the need for Presto to remain compatible with these newer Java versions.

Motivation

The upgrade from Java 8 to Java 17/Java 21 would bring several benefits to Presto, including but not limited to:

  1. Long-Term Support: Java 17 is an LTS release, which means it will receive long-term security updates and support until at least 2029. Presto can align with a stable, secure JVM for years to come.
  2. Performance Improvements: Java 17 introduces numerous JVM performance enhancements, including improved garbage collection, JIT compiler optimizations, and enhanced memory management.
  3. Security Enhancements: Java 17 enforces stricter security measures, including updated cryptographic algorithms, tighter access controls, and improvements to the TLS/SSL implementation (such as default TLS 1.3 support). This makes Presto deployments more secure out of the box.
  4. New Language Features: Java 9 to 17 introduce several modern language features
  5. Maintenance and Upgrade: Many third-party libraries are dropping support for Java 8, and continuing to maintain compatibility with such an old JVM version could slow down future Presto development. Moving to Java 17 helps us stay in sync with newer libraries and frameworks, reducing maintenance burdens.

Challenges

  1. Backward Compatibility: Presto currently runs on Java 8/11 in many production environments. The community will need to ensure that any upgrade path preserves backward compatibility or provides a clear transition path for those still on older Java versions.
  2. Custom Extensions and Connectors: Presto’s ecosystem includes many custom extensions and connectors that might rely on older Java APIs or JVM behavior. Thorough testing and potential refactoring may be required to ensure compatibility.
  3. Deprecated APIs: Java 17 has removed many older APIs and introduces stricter encapsulation via the module system. Any usage of internal APIs (like sun.misc.Unsafe) will need to be refactored or use the --add-exports or --add-opens JVM options to bypass module restrictions (though not recommended for the long term).
  4. Performance Tuning: Presto has been fine-tuned to run on older JVMs, particularly with respect to garbage collection, memory management, and threading. While Java 17 brings many improvements, we will need to carefully test and optimize Presto for this new runtime.

Proposal

This issue aims to start a discussion on upgrading Presto’s JVM support from Java 8 to Java 17+, with the following proposed steps:

  1. Community Feedback: Gather feedback from the Presto community about existing deployments on Java 8 and interest in upgrading to Java 17+.
  2. Compatibility Testing: Begin testing Presto with Java 17 in CI pipelines and staging environments to identify any potential issues or regressions. Test across a range of workloads to ensure stability and performance.
  3. Incremental Transition Plan: Develop a clear transition path, where Presto can continue to support Java 8/11 for a limited time while allowing new versions to run on Java 17. This could involve maintaining dual compatibility or providing support for both in the short term.
  4. Deprecation of Java 8 (if required): Once the community is comfortable with Java 17, we could plan for a timeline to phase out support for Java 8, similar to what many other open-source projects have done.

Next Steps

We would like to hear the community’s thoughts and suggestions on:

Looking forward to feedback and thoughts from the community!

imjalpreet commented 1 month ago

Some things we need to consider

Presto on Spark: Spark versions 3.3+ support Java 17.

AWS SDK 1.x: If we want to upgrade till Java 17 we can keep using AWS SDK 1.x. But for later Java versions, we need to transition to AWS SDK 2.x which has been a long pending task. (https://github.com/aws/aws-sdk-java?tab=readme-ov-file#maintenance-and-support-for-java-versions). AWS SDK 1.x is already in maintenance mode and is reaching end-of-support in Dec'25.

tempto (https://github.com/prestodb/tempto): We use tempto to run presto product tests in the CI pipelines. This hasn't been maintained for the last few years, so we would need to upgrade it to Java 17+ and also look at enhancing it to be able to run the product tests on multiple java versions during the transition period.