Change Drop Table Method(dropTableIfExists)

dongyeon94 commented 3 months ago

fix if exists command not support error

supportsTableIfExistsClause() method always true

so my system print out this error

2024-08-29 10:13:48.061 +0900 [INFO] (0001:transaction): SQL: DROP TABLE IF EXISTS "CLAIM_INFO" 2024-08-29 10:13:48.065 +0900 [ERROR] (0001:transaction): Operation failed (first exception:{SQLState=933, ErrorCode=42000}) java.sql.SQLSyntaxErrorException: ORA-00933: SQL command not properly ended at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:450) at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:399) at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1059) at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:522)

dmikurube commented 3 months ago

Are we/you sure that the catch (Exception ex) {} case is always IF EXISTS is not supported? Doesn't it have any other possible error case? Is that error message appropriate for any case?
What was your actual error case? What RDBMS? Version?
supportsTableIfExistsClause() is expected to be implemented by inheritance. For example, embulk-output-oracle had overridded this to return false;
- https://github.com/embulk/embulk-output-jdbc/blob/v0.10.1/embulk-output-oracle/src/main/java/org/embulk/output/oracle/OracleOutputConnection.java#L65-L69
- We've terminated to maintain embulk-output-oracle as we do not have a testing environment for Oracle DB, though.

dongyeon94 commented 3 months ago

our RDMS version is

oracle 11.2

our embulk and jruby version

embulk version 0.11.1
jruby version 9.4.6.0

we try use "embulk-output-oracle" and we get error 'OutputPlugin 'oracle' is not found'

I thought this error was because not supporting oracle, so I changed it to JDBC plugin and test it and try embulk-output-jdbc result

jdbc plugin not support oracle RDMS?

we are already install

embulk-input-oracle (0.9.0)
embulk-output-oracle (0.8.7)
oracle (0.0.1)
embulk-input-jdbc (0.13.2 java)
embulk-output-jdbc (0.10.6 java)

dmikurube commented 3 months ago

Hmm, then the oracle plugin should work, but I understand it may fail as it's not updated recently.

Can you share the "full" exception output? The text format would be more preferred than a screenshot. Also nice to be in the "code format" (``` 〜 ```).

Cc: @hito4t

dongyeon94 commented 3 months ago

full error text is


2024-08-29 13:09:20.849 +0900 [INFO] (main): Loaded JRuby runtime 9.4.6.0
2024-08-29 13:09:22.640 +0900 [INFO] (main): Started Embulk v0.11.1
2024-08-29 13:09:22.825 +0900 [INFO] (0001:transaction): Embulk system property "plugins.input.jdbc" is not set.
2024-08-29 13:09:22.825 +0900 [INFO] (0001:transaction): Embulk system property "plugins.default.input.jdbc" is not set.
2024-08-29 13:09:25.974 +0900 [INFO] (0001:transaction): Gem's home and path are set by system configs "gem_home": "/var/digdag_home/.embulk/lib/gems", "gem_path": ""
2024-08-29 13:09:27.283 +0900 [INFO] (0001:transaction): Loaded JRuby runtime 9.4.6.0
2024-08-29 13:09:27.379 +0900 [INFO] (0001:transaction): Loaded plugin embulk-input-jdbc (0.13.2)
2024-08-29 13:09:27.497 +0900 [INFO] (0001:transaction): Embulk system property "plugins.output.oracle" is not set.
2024-08-29 13:09:27.497 +0900 [INFO] (0001:transaction): Embulk system property "plugins.default.output.oracle" is not set.
2024-08-29 13:09:27.584 +0900 [INFO] (0001:transaction): Loaded plugin embulk-output-oracle (0.8.7)
org.embulk.config.ConfigException: OutputPlugin 'oracle' is not found.
com/google/common/base/Supplier
    at org.embulk.plugin.PluginManager.buildPluginNotFoundException(PluginManager.java:155)
    at org.embulk.plugin.PluginManager.newPlugin(PluginManager.java:99)
    at org.embulk.spi.ExecSessionInternal.newPlugin(ExecSessionInternal.java:358)
    at org.embulk.spi.ExecInternal.newPlugin(ExecInternal.java:50)
    at org.embulk.exec.BulkLoader$ProcessPluginSet.<init>(BulkLoader.java:442)
    at org.embulk.exec.BulkLoader.doRun(BulkLoader.java:516)
    at org.embulk.exec.BulkLoader.access$000(BulkLoader.java:36)
    at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:353)
    at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:350)
    at org.embulk.spi.ExecInternal.doWith(ExecInternal.java:26)
    at org.embulk.exec.BulkLoader.run(BulkLoader.java:350)
    at org.embulk.EmbulkEmbed.run(EmbulkEmbed.java:278)
    at org.embulk.EmbulkRunner.runInternal(EmbulkRunner.java:288)
    at org.embulk.EmbulkRunner.run(EmbulkRunner.java:153)
    at org.embulk.cli.EmbulkRun.runInternal(EmbulkRun.java:108)
    at org.embulk.cli.EmbulkRun.run(EmbulkRun.java:24)
    at org.embulk.cli.Main.main(Main.java:53)
    Suppressed: org.embulk.plugin.PluginSourceNotMatchException
        at org.embulk.plugin.BuiltinPluginSource.newPlugin(BuiltinPluginSource.java:244)
        at org.embulk.plugin.PluginManager.newPlugin(PluginManager.java:76)
        ... 15 more
    Suppressed: org.embulk.plugin.PluginSourceNotMatchException
        at org.embulk.plugin.maven.MavenPluginRegistry.findPluginPaths(MavenPluginRegistry.java:143)
        at org.embulk.plugin.maven.MavenPluginRegistry.lookup(MavenPluginRegistry.java:63)
        at org.embulk.plugin.maven.MavenPluginSource.newPlugin(MavenPluginSource.java:33)
        at org.embulk.plugin.PluginManager.newPlugin(PluginManager.java:82)
        ... 15 more
    Suppressed: org.embulk.plugin.PluginSourceNotMatchException
        at org.embulk.plugin.SelfContainedPluginRegistry.search(SelfContainedPluginRegistry.java:77)
        at org.embulk.plugin.SelfContainedPluginRegistry.lookup(SelfContainedPluginRegistry.java:64)
        at org.embulk.plugin.SelfContainedPluginSource.newPlugin(SelfContainedPluginSource.java:44)
        at org.embulk.plugin.PluginManager.newPlugin(PluginManager.java:88)
        ... 15 more
    Suppressed: org.embulk.plugin.PluginSourceNotMatchException: Failed to initialize embulk-output-oracle.
        at org.embulk.jruby.JRubyPluginSource.newPlugin(JRubyPluginSource.java:89)
        at org.embulk.plugin.PluginManager.newPlugin(PluginManager.java:94)
        ... 15 more
    Caused by: java.lang.NoClassDefFoundError: com/google/common/base/Supplier
        at java.lang.Class.getDeclaredConstructors0(Native Method)
        at java.lang.Class.privateGetDeclaredConstructors(java/lang/Class.java:2671)
        at java.lang.Class.getConstructor0(java/lang/Class.java:3075)
        at java.lang.Class.getConstructor(java/lang/Class.java:1825)
        at org.embulk.plugin.PluginManager.getConstructorWithProperties(org/embulk/plugin/PluginManager.java:130)
        at org.embulk.plugin.PluginManager.newPluginInstance(org/embulk/plugin/PluginManager.java:106)
        at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)
        at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:330)
        at org.jruby.javasupport.JavaMethod.invokeStaticDirect(org/jruby/javasupport/JavaMethod.java:232)
        at RUBY.new_java(/var/digdag_home/.embulk/lib/gems/gems/embulk-0.11.1-java/lib/embulk/java_plugin.rb:92)
        at RUBY.new_java_output(/var/digdag_home/.embulk/lib/gems/gems/embulk-0.11.1-java/lib/embulk/plugin.rb:159)
        at RUBY.new_java_output(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/forwardable.rb:238)
        at org.jruby.RubyClass.finvokeWithRefinements(org/jruby/RubyClass.java:540)
        at org.jruby.RubyClass.finvoke(org/jruby/RubyClass.java:528)
        at org.jruby.embed.internal.EmbedRubyObjectAdapterImpl.doInvokeMethod(org/jruby/embed/internal/EmbedRubyObjectAdapterImpl.java:240)
        at org.jruby.embed.internal.EmbedRubyObjectAdapterImpl.callMethod(org/jruby/embed/internal/EmbedRubyObjectAdapterImpl.java:156)
        at org.jruby.embed.ScriptingContainer.callMethod(org/jruby/embed/ScriptingContainer.java:1448)
        at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)
        at org.embulk.jruby.ScriptingContainerDelegateImpl.callMethod(org/embulk/jruby/ScriptingContainerDelegateImpl.java:644)
        at org.embulk.jruby.LazyScriptingContainerDelegate.callMethod(org/embulk/jruby/LazyScriptingContainerDelegate.java:230)
        at org.embulk.jruby.JRubyPluginSource.newPlugin(org/embulk/jruby/JRubyPluginSource.java:87)
        at org.embulk.plugin.PluginManager.newPlugin(org/embulk/plugin/PluginManager.java:94)
        at org.embulk.spi.ExecSessionInternal.newPlugin(org/embulk/spi/ExecSessionInternal.java:358)
        at org.embulk.spi.ExecInternal.newPlugin(org/embulk/spi/ExecInternal.java:50)
        at org.embulk.exec.BulkLoader$ProcessPluginSet.<init>(org/embulk/exec/BulkLoader.java:442)
        at org.embulk.exec.BulkLoader.doRun(org/embulk/exec/BulkLoader.java:516)
        at org.embulk.exec.BulkLoader.access$000(org/embulk/exec/BulkLoader.java:36)
        at org.embulk.exec.BulkLoader$1.run(org/embulk/exec/BulkLoader.java:353)
        at org.embulk.exec.BulkLoader$1.run(org/embulk/exec/BulkLoader.java:350)
        at org.embulk.spi.ExecInternal.doWith(org/embulk/spi/ExecInternal.java:26)
        at org.embulk.exec.BulkLoader.run(org/embulk/exec/BulkLoader.java:350)
        at org.embulk.EmbulkEmbed.run(org/embulk/EmbulkEmbed.java:278)
        at org.embulk.EmbulkRunner.runInternal(org/embulk/EmbulkRunner.java:288)
        at org.embulk.EmbulkRunner.run(org/embulk/EmbulkRunner.java:153)
        at org.embulk.cli.EmbulkRun.runInternal(org/embulk/cli/EmbulkRun.java:108)
        at org.embulk.cli.EmbulkRun.run(org/embulk/cli/EmbulkRun.java:24)
        at org.embulk.cli.Main.main(org/embulk/cli/Main.java:53)
    Caused by: java.lang.ClassNotFoundException: com.google.common.base.Supplier
        at org.embulk.plugin.PluginClassLoader.loadClass(PluginClassLoader.java:160)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        at java.lang.Class.getDeclaredConstructors0(Native Method)
        at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
        at java.lang.Class.getConstructor0(Class.java:3075)
        at java.lang.Class.getConstructor(Class.java:1825)
        at org.embulk.plugin.PluginManager.getConstructorWithProperties(PluginManager.java:130)
        at org.embulk.plugin.PluginManager.newPluginInstance(PluginManager.java:106)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:330)
        at org.jruby.javasupport.JavaMethod.invokeStaticDirect(JavaMethod.java:232)
        at org.jruby.java.invokers.StaticMethodInvoker.call(StaticMethodInvoker.java:53)
        at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:291)
        at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:324)
        at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)
        at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:76)
        at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:164)
        at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:151)
        at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:212)
        at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:456)
        at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:195)
        at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:346)
        at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)
        at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:128)
        at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:115)
        at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:446)
        at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:92)
        at org.jruby.ir.instructions.CallBase.interpret(CallBase.java:548)
        at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:363)
        at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)
        at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:128)
        at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:115)
        at org.jruby.RubyClass.finvokeWithRefinements(RubyClass.java:540)
        at org.jruby.RubyClass.finvoke(RubyClass.java:528)
        at org.jruby.runtime.Helpers.invoke(Helpers.java:703)
        at org.jruby.embed.internal.EmbedRubyObjectAdapterImpl.doInvokeMethod(EmbedRubyObjectAdapterImpl.java:240)
        at org.jruby.embed.internal.EmbedRubyObjectAdapterImpl.callMethod(EmbedRubyObjectAdapterImpl.java:156)
        at org.jruby.embed.ScriptingContainer.callMethod(ScriptingContainer.java:1448)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.embulk.jruby.ScriptingContainerDelegateImpl.callMethod(ScriptingContainerDelegateImpl.java:644)
        at org.embulk.jruby.LazyScriptingContainerDelegate.callMethod(LazyScriptingContainerDelegate.java:230)
        at org.embulk.jruby.JRubyPluginSource.newPlugin(JRubyPluginSource.java:87)
        at org.embulk.plugin.PluginManager.newPlugin(PluginManager.java:94)
        at org.embulk.spi.ExecSessionInternal.newPlugin(ExecSessionInternal.java:358)
        at org.embulk.spi.ExecInternal.newPlugin(ExecInternal.java:50)
        at org.embulk.exec.BulkLoader$ProcessPluginSet.<init>(BulkLoader.java:442)
        at org.embulk.exec.BulkLoader.doRun(BulkLoader.java:516)
        at org.embulk.exec.BulkLoader.access$000(BulkLoader.java:36)
        at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:353)
        at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:350)
        at org.embulk.spi.ExecInternal.doWith(ExecInternal.java:26)
        at org.embulk.exec.BulkLoader.run(BulkLoader.java:350)
        at org.embulk.EmbulkEmbed.run(EmbulkEmbed.java:278)
        at org.embulk.EmbulkRunner.runInternal(EmbulkRunner.java:288)
        at org.embulk.EmbulkRunner.run(EmbulkRunner.java:153)
        at org.embulk.cli.EmbulkRun.runInternal(EmbulkRun.java:108)
        at org.embulk.cli.EmbulkRun.run(EmbulkRun.java:24)
        at org.embulk.cli.Main.main(Main.java:53)

Error: OutputPlugin 'oracle' is not found.

In our opinion, the plugin can not find the Oracle package

hiroyuki-sato commented 3 months ago

As far as I know, embulk-output-oracle does not support Embulk v0.10 ~ yet. Maybe, You can run embulk-output-oracle using embulk 0.9.24 as a workaround.

dmikurube commented 3 months ago

Ah, yeah, not the Oracle JDBC driver, but com.google.common.base.Supplier was required, which is no longer included since Embulk v0.10.3X.

dongyeon94 commented 3 months ago

we are upgrading embulk version 0.9.25 -> 0.11.1

We know that version 11.1 is a stable version, so we are upgrading the version. Do you have any plan to upgrade v0.10 "embulk-output-oracle" or later??

dmikurube commented 3 months ago

As of https://github.com/embulk/embulk-output-jdbc/pull/302, we (the Embulk project) no longer maintain embulk-output-oracle as we have no testing environment for Oracle.

If you're interested in maintaining it by yourself, or joining other efforts, you may want to talk to @hito4t

dongyeon94 commented 2 months ago

@hito4t How can I joining embulk-output-oracle project?

hito4t commented 2 months ago

@dongyeon94 Thank you for the proposal.

I created the repository for embulk-input-oracle (https://github.com/hito4t/embulk-input-jdbc-external), but not yet for embulk-output-oracle. I'll create the repository from the latest (although not maintained...) source code for embulk-output-oracle.

Please wait for a while.

dongyeon94 commented 2 months ago

I See, thank you. I'll personally customize it and use it. Please contact me when we proceed later.

hito4t commented 2 months ago

@dongyeon94 @hiroyuki-sato I've created the repository for embulk-output-oracle, although it doesn't support the latest Embulk yet. https://github.com/hito4t/embulk-output-jdbc-external

embulk / embulk-output-jdbc

Change Drop Table Method(dropTableIfExists) #346