search

apache / linkis

Apache Linkis builds a computation middleware layer to facilitate connection, governance and orchestration between the upper applications and the underlying data engines.
https://linkis.apache.org/
Apache License 2.0
3.31k stars 1.17k forks source link

[Bug] The client did not judge the failure of uploading to bml, resulting in unclear error messages. #4354

Closed wForget closed 3 months ago

wForget commented 1 year ago

Search before asking

Linkis Component

linkis-computation-governance

Steps to reproduce

error msg in linkis-cg-linkismanager.log:

2023-03-10 09:45:10.300 [ERROR] [Linkis-Default-Scheduler-Thread-1       ] o.a.l.b.r.BmlUploadResult (85) [apply] - errCode: 10905 ,desc: URL /api/rest_j/v1/bml/upload request failed! ResponseBody is {"method":null,"status":1,"message":"error code(错误码): 60050, error message(错误信息): The first upload of the resource failed(首次上传资源失败).","data":{"errorMsg":{"serviceKind":"linkis-ps-publicservice","level":2,"port":9105,"errCode":50073,"ip":"XXX","desc":"The commit upload resource task failed(提交上传资源任务失败):errCode: 60050 ,desc: The first upload of the resource failed(首次上传资源失败) ,ip: XXX ,port: 9105 ,serviceKind: linkis-ps-publicservice"}}}. ,ip: XXX ,port: 9101 ,serviceKind: linkis-cg-linkismanager
2023-03-10 09:45:10.339 [ERROR] [Linkis-Default-Scheduler-Thread-1       ] o.a.l.b.c.i.HttpBmlClient (430) [uploadResource] - user hadoop upload resource failed, status code is 0
2023-03-10 09:45:10.521 [ERROR] [Linkis-Default-Scheduler-Thread-1       ] o.a.l.e.s.s.DefaultEngineConnResourceService (119) [apply] - Failed to upload engine conn to bml, now exit! org.springframework.dao.DataIntegrityViolationException: 
### Error updating database.  Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Column 'bml_resource_id' cannot be null
### The error may exist in URL [jar:file:/home/hadoop/linkis/apache-linkis-1.3.2-SNAPSHOT-bin/LinkisInstall/lib/linkis-computation-governance/linkis-cg-linkismanager/linkis-application-manager-1.3.2-SNAPSHOT.jar!/mapper/common/EngineConnBmlResourceMapper.xml]
### The error may involve org.apache.linkis.engineplugin.server.dao.EngineConnBmlResourceDao.save-Inline
### The error occurred while setting parameters
### SQL: insert into linkis_cg_engine_conn_plugin_bml_resources(          `engine_conn_type`,`version`,`file_name`,`file_size`,`last_modified`,`bml_resource_id`,`bml_resource_version`,`create_time`,`last_update_time`      )         values(?, ?,         ?,?,         ?,?,         ?,         ?, ?)
### Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Column 'bml_resource_id' cannot be null
; Column 'bml_resource_id' cannot be null; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Column 'bml_resource_id' cannot be null
        at org.springframework.jdbc.support.SQLExceptionSubclassTranslator.doTranslate(SQLExceptionSubclassTranslator.java:87) ~[spring-jdbc-5.2.22.RELEASE.jar:5.2.22.RELEASE]
        at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) ~[spring-jdbc-5.2.22.RELEASE.jar:5.2.22.RELEASE]
        at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81) ~[spring-jdbc-5.2.22.RELEASE.jar:5.2.22.RELEASE]
        at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:88) ~[mybatis-spring-2.0.5.jar:2.0.5]
        at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:440) ~[mybatis-spring-2.0.5.jar:2.0.5]
        at com.sun.proxy.$Proxy138.insert(Unknown Source) ~[?:?]
        at org.mybatis.spring.SqlSessionTemplate.insert(SqlSessionTemplate.java:271) ~[mybatis-spring-2.0.5.jar:2.0.5]
        at com.baomidou.mybatisplus.core.override.MybatisMapperMethod.execute(MybatisMapperMethod.java:60) ~[mybatis-plus-core-3.4.1.jar:3.4.1]
        at com.baomidou.mybatisplus.core.override.MybatisMapperProxy$PlainMethodInvoker.invoke(MybatisMapperProxy.java:148) ~[mybatis-plus-core-3.4.1.jar:3.4.1]
        at com.baomidou.mybatisplus.core.override.MybatisMapperProxy.invoke(MybatisMapperProxy.java:89) ~[mybatis-plus-core-3.4.1.jar:3.4.1]
        at com.sun.proxy.$Proxy139.save(Unknown Source) ~[?:?]
        at org.apache.linkis.engineplugin.server.service.DefaultEngineConnResourceService$$anonfun$org$apache$linkis$engineplugin$server$service$DefaultEngineConnResourceService$$refresh$2.apply(DefaultEngineConnResourceService.scala:204) ~[linkis-application-manager-1.3.2-SNAPSHOT.jar:1.3.2-SNAPSHOT]
        at org.apache.linkis.engineplugin.server.service.DefaultEngineConnResourceService$$anonfun$org$apache$linkis$engineplugin$server$service$DefaultEngineConnResourceService$$refresh$2.apply(DefaultEngineConnResourceService.scala:187) ~[linkis-application-manager-1.3.2-SNAPSHOT.jar:1.3.2-SNAPSHOT]
        at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) ~[scala-library-2.11.12.jar:?]
        at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186) ~[scala-library-2.11.12.jar:?]
        at org.apache.linkis.engineplugin.server.service.DefaultEngineConnResourceService.org$apache$linkis$engineplugin$server$service$DefaultEngineConnResourceService$$refresh(DefaultEngineConnResourceService.scala:187) ~[linkis-application-manager-1.3.2-SNAPSHOT.jar:1.3.2-SNAPSHOT]
        at org.apache.linkis.engineplugin.server.service.DefaultEngineConnResourceService$$anon$1$$anonfun$run$1$$anonfun$apply$1$$anonfun$apply$mcV$sp$1.apply(DefaultEngineConnResourceService.scala:113) ~[linkis-application-manager-1.3.2-SNAPSHOT.jar:1.3.2-SNAPSHOT]
        at org.apache.linkis.engineplugin.server.service.DefaultEngineConnResourceService$$anon$1$$anonfun$run$1$$anonfun$apply$1$$anonfun$apply$mcV$sp$1.apply(DefaultEngineConnResourceService.scala:110) ~[linkis-application-manager-1.3.2-SNAPSHOT.jar:1.3.2-SNAPSHOT]
        at scala.collection.immutable.Map$Map1.foreach(Map.scala:116) ~[scala-library-2.11.12.jar:?]
        at org.apache.linkis.engineplugin.server.service.DefaultEngineConnResourceService$$anon$1$$anonfun$run$1$$anonfun$apply$1.apply$mcV$sp(DefaultEngineConnResourceService.scala:110) ~[linkis-application-manager-1.3.2-SNAPSHOT.jar:1.3.2-SNAPSHOT]
        at org.apache.linkis.engineplugin.server.service.DefaultEngineConnResourceService$$anon$1$$anonfun$run$1$$anonfun$apply$1.apply(DefaultEngineConnResourceService.scala:108) ~[linkis-application-manager-1.3.2-SNAPSHOT.jar:1.3.2-SNAPSHOT]
        at org.apache.linkis.engineplugin.server.service.DefaultEngineConnResourceService$$anon$1$$anonfun$run$1$$anonfun$apply$1.apply(DefaultEngineConnResourceService.scala:108) ~[linkis-application-manager-1.3.2-SNAPSHOT.jar:1.3.2-SNAPSHOT]
        at org.apache.linkis.common.utils.Utils$.tryCatch(Utils.scala:48) ~[linkis-common-1.3.2-SNAPSHOT.jar:1.3.2-SNAPSHOT]
        at org.apache.linkis.engineplugin.server.service.DefaultEngineConnResourceService$$anon$1$$anonfun$run$1.apply(DefaultEngineConnResourceService.scala:115) ~[linkis-application-manager-1.3.2-SNAPSHOT.jar:1.3.2-SNAPSHOT]
        at org.apache.linkis.engineplugin.server.service.DefaultEngineConnResourceService$$anon$1$$anonfun$run$1.apply(DefaultEngineConnResourceService.scala:107) ~[linkis-application-manager-1.3.2-SNAPSHOT.jar:1.3.2-SNAPSHOT]
        at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) ~[scala-library-2.11.12.jar:?]
        at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186) ~[scala-library-2.11.12.jar:?]
        at org.apache.linkis.engineplugin.server.service.DefaultEngineConnResourceService$$anon$1.run(DefaultEngineConnResourceService.scala:106) ~[linkis-application-manager-1.3.2-SNAPSHOT.jar:1.3.2-SNAPSHOT]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_271]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_271]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_271]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.0_271]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_271]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_271]
        at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_271]
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Column 'bml_resource_id' cannot be null
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_271]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_271]
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_271]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_271]
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:403) ~[mysql-connector-java-5.1.49.jar:5.1.49]
        at com.mysql.jdbc.Util.getInstance(Util.java:386) ~[mysql-connector-java-5.1.49.jar:5.1.49]
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:936) ~[mysql-connector-java-5.1.49.jar:5.1.49]
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3933) ~[mysql-connector-java-5.1.49.jar:5.1.49]
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3869) ~[mysql-connector-java-5.1.49.jar:5.1.49]
        at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2524) ~[mysql-connector-java-5.1.49.jar:5.1.49]
        at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2675) ~[mysql-connector-java-5.1.49.jar:5.1.49]
        at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2465) ~[mysql-connector-java-5.1.49.jar:5.1.49]
        at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1915) ~[mysql-connector-java-5.1.49.jar:5.1.49]
        at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1254) ~[mysql-connector-java-5.1.49.jar:5.1.49]
        at com.alibaba.druid.pool.DruidPooledPreparedStatement.execute(DruidPooledPreparedStatement.java:497) ~[druid-1.1.22.jar:1.1.22]
        at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:47) ~[mybatis-3.5.6.jar:3.5.6]
        at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:74) ~[mybatis-3.5.6.jar:3.5.6]
        at com.baomidou.mybatisplus.core.executor.MybatisSimpleExecutor.doUpdate(MybatisSimpleExecutor.java:56) ~[mybatis-plus-core-3.4.1.jar:3.4.1]
        at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:117) ~[mybatis-3.5.6.jar:3.5.6]
        at com.baomidou.mybatisplus.core.executor.MybatisCachingExecutor.update(MybatisCachingExecutor.java:85) ~[mybatis-plus-core-3.4.1.jar:3.4.1]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_271]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_271]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_271]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_271]
        at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:63) ~[mybatis-3.5.6.jar:3.5.6]
        at com.sun.proxy.$Proxy146.update(Unknown Source) ~[?:?]
        at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:197) ~[mybatis-3.5.6.jar:3.5.6]
        at org.apache.ibatis.session.defaults.DefaultSqlSession.insert(DefaultSqlSession.java:184) ~[mybatis-3.5.6.jar:3.5.6]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_271]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_271]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_271]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_271]
        at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:426) ~[mybatis-spring-2.0.5.jar:2.0.5]
        ... 30 more

Expected behavior

After uploading to bml we should judge whether it is successful.

Your environment

Anything else

No response

Are you willing to submit a PR?

github-actions[bot] commented 1 year ago

:blush: Welcome to the Apache Linkis community!!

We are glad that you are contributing by opening this issue.

Please make sure to include all the relevant context. We will be here shortly.

If you are interested in contributing to our website project, please let us know! You can check out our contributing guide on :point_right: How to Participate in Project Contribution.

Community

WeChat Assistant WeChat Public Account

Mailing Lists

Name Description Subscribe Unsubscribe Archive
dev@linkis.apache.org community activity information subscribe unsubscribe archive
tclwh commented 1 year ago

没人回复嘛

wForget commented 1 year ago

没人回复嘛

This problem is not serious. It is the failure to upload bml that leads to NPE so that the error message is not clear. More clear errors can be seen in the bml service.

这个问题并不严重，是上传 bml 失败导致了 NPE 以至于错误信息不明确，在 bml 服务中可以看到更明确的错误。