WeBankFinTech / DataSphereStudio

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.
https://github.com/WeBankFinTech/DataSphereStudio-Doc
Apache License 2.0
3.08k stars 1k forks source link

Failed to create workflow in a project #64

Closed RainFlying closed 4 years ago

RainFlying commented 4 years ago

The error message was:

" operation failed(操作失败)s!the reason(原因):HttpClientResultException: errCode: 10905 ,desc: URL http://127.0.0.1:9001/api/rest_j/v1/bml/upload request failed! ResponseBody is {"method":null,"status":1,"message":"error code(错误码): 50073, error message(错误信息): 提交上传资源任务失败:\n### Error updating database. Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'linkis.linkis_resources_task' doesn't exist\n### The error may involve com.webank.wedatasphere.linkis.bml.dao.TaskDao.insert-Inline\n### The error occurred while setting parameters\n### SQL: INSERT INTO linkis_resources_task( resource_id,version,operation,state, submit_user,system,instance, client_ip,err_msg,start_time,end_time,last_update_time, extra_params ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)\n### Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'linkis.linkis_resources_task' doesn't exist\n; bad SQL grammar []; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'linkis.linkis_resources_task' doesn't exist.","data":{"errorMsg":{"serviceKind":"bml-server","level":2,"port":9999,"errCode":50073,"ip":"DataSphere-web1","desc":"提交上传资源任务失败:\n### Error updating database. Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'linkis.linkis_resources_task' doesn't exist\n### The error may involve com.webank.wedatasphere.linkis.bml.dao.TaskDao.insert-Inline\n### The error occurred while setting parameters\n### SQL: INSERT INTO linkis_resources_task( resource_id,version,operation,state, submit_user,system,instance, client_ip,err_msg,start_time,end_time,last_update_time, extra_params ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)\n### Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'linkis.linkis_resources_task' doesn't exist\n; bad SQL grammar []; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'linkis.linkis_resources_task' doesn't exist"}}}. ,ip: DataSphere-web1 ,port: 9004 ,serviceKind: dss-server "

According to the page https://github.com/WeBankFinTech/DataSphereStudio/blob/master/docs/zh_CN/ch1/DSS%E5%AE%89%E8%A3%85%E5%B8%B8%E8%A7%81%E9%97%AE%E9%A2%98%E5%88%97%E8%A1%A8.md#6-dss%E5%88%9B%E5%BB%BA%E5%B7%A5%E7%A8%8B%E6%8A%A5%E8%A1%A8linkislinkis_resources_task%E4%B8%8D%E5%AD%98%E5%9C%A8, I needed to run db/moudle/linkis-bml.sql manually.

Tried this SQL file, found only linkis_resources_task table was missing, ran the query manually on the mysql session. The above error was gone, found a new one.

" operation failed(操作失败)s!the reason(原因):HttpClientResultException: errCode: 10905 ,desc: URL http://127.0.0.1:9001/api/rest_j/v1/bml/upload request failed! ResponseBody is {"method":null,"status":1,"message":"error code(错误码): 50073, error message(错误信息): 提交上传资源任务失败:\n### Error updating database. Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Column 'system' cannot be null\n### The error may involve com.webank.wedatasphere.linkis.bml.dao.TaskDao.insert-Inline\n### The error occurred while setting parameters\n### SQL: INSERT INTO linkis_resources_task( resource_id,version,operation,state, submit_user,system,instance, client_ip,err_msg,start_time,end_time,last_update_time, extra_params ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)\n### Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Column 'system' cannot be null\n; SQL []; Column 'system' cannot be null; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Column 'system' cannot be null.","data":{"errorMsg":{"serviceKind":"bml-server","level":2,"port":9999,"errCode":50073,"ip":"DataSphere-web1","desc":"提交上传资源任务失败:\n### Error updating database. Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Column 'system' cannot be null\n### The error may involve com.webank.wedatasphere.linkis.bml.dao.TaskDao.insert-Inline\n### The error occurred while setting parameters\n### SQL: INSERT INTO linkis_resources_task( resource_id,version,operation,state, submit_user,system,instance, client_ip,err_msg,start_time,end_time,last_update_time, extra_params ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)\n### Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Column 'system' cannot be null\n; SQL []; Column 'system' cannot be null; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Column 'system' cannot be null"}}}. ,ip: DataSphere-web1 ,port: 9004 ,serviceKind: dss-server "

mysql> show create table linkis_resources_task \G
*************************** 1. row ***************************
       Table: linkis_resources_task
Create Table: CREATE TABLE `linkis_resources_task` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `resource_id` varchar(50) DEFAULT NULL COMMENT '资源id,资源的uuid',
  `version` varchar(20) DEFAULT NULL COMMENT '当前操作的资源版本号',
  `operation` varchar(20) NOT NULL COMMENT '操作类型.upload = 0, update = 1',
  `state` varchar(20) NOT NULL DEFAULT 'Schduled' COMMENT '任务当前状态:Schduled, Running, Succeed, Failed,Cancelled',
  `submit_user` varchar(20) NOT NULL DEFAULT '' COMMENT '任务提交用户名',
  `system` varchar(20) NOT NULL DEFAULT '' COMMENT '子系统名 wtss',
  `instance` varchar(50) NOT NULL COMMENT '物料库实例',
  `client_ip` varchar(50) DEFAULT NULL COMMENT '请求IP',
  `extra_params` text COMMENT '额外关键信息.如批量删除的资源IDs及versions,删除资源下的所有versions',
  `err_msg` varchar(2000) DEFAULT NULL COMMENT '任务失败信息.e.getMessage',
  `start_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '开始时间',
  `end_time` datetime DEFAULT NULL COMMENT '结束时间',
  `last_update_time` datetime NOT NULL COMMENT '最后更新时间',
  PRIMARY KEY (`id`),
  UNIQUE KEY `resource_id_version` (`resource_id`,`version`,`operation`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
1 row in set (0.00 sec)
peacewong commented 4 years ago

Hello, this is caused by the initialization of the bml database, you can see the installation of linkis

RainFlying commented 4 years ago

Hello, this is caused by the initialization of the bml database, you can see the installation of linkis

I followed the installation instruction on the linkis page, no errors were reported.

RainFlying commented 4 years ago

Hello, this is caused by the initialization of the bml database, you can see the installation of linkis

Reinstalled linkis version 0.9.2, the original issue was gone. Another one appeared.

" operation failed(操作失败)s!the reason(原因):HttpClientResultException: errCode: 10905 ,desc: URL http://127.0.0.1:9001/api/rest_j/v1/bml/upload request failed! ResponseBody is {"method":null,"status":1,"message":"error code(错误码): 50073, error message(错误信息): 提交上传资源任务失败:Call From DataSphere-web1/10.1.128.108 to DataSphere-web1:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused.","data":{"errorMsg":{"serviceKind":"bml-server","level":2,"port":9113,"errCode":50073,"ip":"DataSphere-web1","desc":"提交上传资源任务失败:Call From DataSphere-web1/10.1.128.108 to DataSphere-web1:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused"}}}. ,ip: DataSphere-web1 ,port: 9004 ,serviceKind: dss-server "

What service should be listening on port 9000 ? How to delete the half-created workflow ? Should I delete some rows in some tables ?

RainFlying commented 4 years ago

Figured out there were some hadoop services not running.