[Bug] [seatunnel-plugin-discovery] Use error reporting in fatjar mode

antlers-lv commented 3 months ago

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened

After I changed the package of seatunnel-web from shade to fatjar, I started seatunnel-web as jar package, and the following problems occurred

Caused by: java.lang.NoClassDefFoundError: org/apache/seatunnel/api/table/factory/TableSinkFactory
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:370)
        at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
        at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
        at java.util.Iterator.forEachRemaining(Iterator.java:116)
        at org.apache.seatunnel.api.table.factory.FactoryUtil.discoverFactories(FactoryUtil.java:242)
        at org.apache.seatunnel.app.thirdparty.framework.PluginDiscoveryUtil.getConnectorFeatures(PluginDiscoveryUtil.java:70)
        at org.apache.seatunnel.app.bean.connector.ConnectorCache.syncSourceFeature(ConnectorCache.java:136)
        at org.apache.seatunnel.app.bean.connector.ConnectorCache.refresh(ConnectorCache.java:132)
        at org.apache.seatunnel.app.bean.connector.ConnectorCache.<init>(ConnectorCache.java:73)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:211)
        ... 27 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.apache.seatunnel.api.table.factory.TableSinkFactory
        at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        ... 54 common frames omitted

The problem I found is that seatunnel-plugin-discovery uses the URLClassLoader to find all connector extensions, but does not specify the parent for the URLClassLoader. Meanwhile, the current context uses Fatjar's LaunchedURLClassLOader, and the parent in URLClassLoader is set to the default application classloader Launcher@AppClassLoader. As a result, the Seatunnel-APi-related Class introduced into the jar package was not found when the SPI Load was performed.

Finally, I think the URLClassloader should be built with the ClassLoader used by the main line as the parent. Of course I tried and solved the problem.

such as getAllPlugin method of AbstractPluginDiscovery

public Map<PluginType, LinkedHashMap<PluginIdentifier, OptionRule>> getAllPlugin()
            throws IOException {
        List<Factory> factories;
        if (this.pluginDir.toFile().exists()) {
            log.info("load plugin from plugin dir: {}", this.pluginDir);
            List<URL> files = FileUtils.searchJarFiles(this.pluginDir);
            factories =
                    FactoryUtil.discoverFactories(new URLClassLoader(files.toArray(new URL[0]), Thread.currentThread().getContextClassLoader()));
        } else {
            log.warn("plugin dir: {} not exists, load plugin from classpath", this.pluginDir);
            factories =
                    FactoryUtil.discoverFactories(Thread.currentThread().getContextClassLoader());
        }
       ...

SeaTunnel Version

2.3.3

SeaTunnel Config

Not applicable

Running Command

Not applicable

Error Exception

Not applicable

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

github-actions[bot] commented 2 months ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] commented 2 months ago

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.

apache / seatunnel