[Bug]: JdbcDriver causes metaspace leak

[gabrywu]

What happened?

As of now ,org.apache.beam.sdk.extensions.sql.impl.JdbcDriver doesn't de-register itself from java.sql.DriverManager. And this can causes metaspace leak, because it holds a ChildFirstClassLoader on a flink runner. If we submit jobs many times to a same flink cluster, corresponding cluster OOM eventually.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

• [ ] Component: Python SDK
• [X] Component: Java SDK
• [ ] Component: Go SDK
• [ ] Component: Typescript SDK
• [ ] Component: IO connector
• [ ] Component: Beam YAML
• [ ] Component: Beam examples
• [ ] Component: Beam playground
• [ ] Component: Beam katas
• [ ] Component: Website
• [ ] Component: Spark Runner
• [ ] Component: Flink Runner
• [ ] Component: Samza Runner
• [ ] Component: Twister2 Runner
• [ ] Component: Hazelcast Jet Runner
• [ ] Component: Google Cloud Dataflow Runner

[gabrywu]

@Abacn can you help with this one?

[Abacn]

JdbcDriver extends from calcite Driver, and both JdbcDriver and its base class calls register at static statement. Are they both contribute to metaspace leak? I do not find an obvious solution either here.

Are you able to deregister the stall Drivers using DriverManager.Deregister in a DoFn after Impulse() (so it will execute once) or its lacking permission (raising SQL SecurityException) ?

[gabrywu]

@Abacn yes, I remove this Driver by DriverManager.deregister in a tearDown, and it's OK to run DriverManager.deregister multiple times