search

opentelekomcloud / gophertelekomcloud

GopherTelekomCloud is an OpenTelekomCloud clouds Go SDK
Apache License 2.0
20 stars 10 forks source link

Implementation of DataArts Factory job API #673

Closed sgmv closed 4 months ago

sgmv commented 4 months ago

What this PR does / why we need it

The last changes for DataArt API. This PR brings the ability to manage DataArts Factory jobs.

Which issue this PR fixes

Special notes for your reviewer

Test is ok:


=== RUN   TestDataArtsJobExecution
    connections_test.go:146: trying to get ec instance
    connections_test.go:129: trying to get a ssh key pair
    connections_test.go:119: check if test cluster is created
    connections_test.go:119: test cluster is not found, create it
    connections_test.go:119: starting to create a DataArts cluster
    connections_test.go:129: key pair not found, create one
    connections_test.go:133: upload ssh key dataarts-test.pem to obs bucket: dataart-test-bucket
    connections_test.go:146: ec instance not found, create one
    connections_test.go:146: Attempting to create ECSv1
    tools.go:72: {
          "name": "testAllCasesHjuA3",
          "id": "e0516ddf-fb6c-42f3-8f30-c16a78ad4d7e",
          "task": {
            "id": "2c9080b98f1a2efa018fce571f5b0719",
            "name": "rdsCreateBackupJob"
          },
          "datastore": {
            "type": "cdm",
            "version": "2.10.0.100"
          },
          "instances": [
            {
              "id": "6a57a054-08bb-435e-8b4e-112ab7439304",
              "name": "testAllCasesHjuA3-cdm-dn-1-1",
              "type": "cdm",
              "shard_id": "dn-1"
            }
          ]
        }
    connections_test.go:119: DataArts cluster was created
    connections_test.go:119: check cluster status, should be normal
    connections_test.go:146: Created ECSv1 instance: 5db66f22-bc11-485a-b0d7-0f2737944663
    connections_test.go:119: get cluster details
    tools.go:72: {
          "publicEndpoint": "",
          "instances": [
            {
              "flavor": {
                "id": "5ddb1071-c5d7-40e0-a874-8a032e81a697",
                "links": [
                  {
                    "rel": "self",
                    "href": "http://172.16.0.56:8979/7ed5f793b8354ea9b27a849f17af4733/instances/6a57a054-08bb-435e-8b4e-112ab7439304"
                  }
                ]
              },
              "volume": {
                "type": "",
                "size": 0
              },
              "status": "200",
              "actions": [],
              "type": "cdm",
              "id": "6a57a054-08bb-435e-8b4e-112ab7439304",
              "name": "testAllCasesHjuA3-cdm-dn-1-1",
              "isFrozen": "0",
              "components": "",
              "config_status": "In-Sync",
              "role": "Standalone",
              "group": "dn-1",
              "links": [
                {
                  "rel": "self",
                  "href": "https://127.0.0.1/rds/v1.0/instances/6a57a054-08bb-435e-8b4e-112ab7439304"
                },
                {
                  "rel": "bookmark",
                  "href": "https://127.0.0.1/instances/6a57a054-08bb-435e-8b4e-112ab7439304"
                }
              ],
              "paramsGroupId": "26084bb9-e74b-47d5-8be6-c0fbee9449d5",
              "publicIp": "",
              "manageIp": "100.94.2.126",
              "trafficIp": "10.10.0.149",
              "shard_id": "",
              "manage_fix_ip": "192.168.17.92",
              "private_ip": "10.10.0.149",
              "internal_ip": "",
              "resource": [
                {
                  "resource_id": "32650217-168a-4ff3-a2fa-5f7a71d54675",
                  "resource_type": "SERVER"
                }
              ]
            }
          ],
          "security_group_id": "12fcdc62-9bff-4df0-ac14-5258050d004b",
          "subnet_id": "03c5f385-17a0-4312-9e10-9b084edc18a1",
          "vpc_id": "bf200468-ce32-4b03-94d9-609f90983dfb",
          "customerConfig": {
            "failureRemind": "false",
            "clusterName": "mppdbCluster",
            "serviceProvider": "DT",
            "localDisk": "false",
            "ssl": "true",
            "createFrom": "4",
            "resourceId": "",
            "flavorType": "cdm.large",
            "workSpaceId": "",
            "trial": ""
          },
          "datastore": {
            "type": "cdm",
            "version": "2.10.0.100"
          },
          "isAutoOff": false,
          "publicEndpointDomainName": "",
          "bakExpectedStartTime": "",
          "bakKeepDay": "",
          "maintainWindow": {
            "day": "Thu",
            "startTime": "22:00",
            "endTime": "02:00"
          },
          "recentEvent": 0,
          "flavorName": "cdm.large",
          "azName": "eu-de-01",
          "endpointDomainName": "",
          "publicEndpointStatus": {
            "status": "",
            "errorMessage": ""
          },
          "isScheduleBootOff": false,
          "namespace": "cdm",
          "eipId": "",
          "failedReasons": {
            "CREATE_FAILED": {
              "errorCode": "",
              "errorMsg": ""
            }
          },
          "dbuser": "cdm",
          "links": [
            {
              "rel": "self",
              "href": "http://172.16.0.56:8979/7ed5f793b8354ea9b27a849f17af4733"
            }
          ],
          "clusterMode": "sharding",
          "task": {
            "description": "",
            "id": "",
            "name": ""
          },
          "created": "2024-05-31T11:20:38Z",
          "statusDetail": "Normal",
          "config_status": "In-Sync",
          "actionProgress": {
            "creating": "",
            "growing": "",
            "restoring": "",
            "snapshotting": "",
            "repairing": ""
          },
          "name": "testAllCasesHjuA3",
          "id": "e0516ddf-fb6c-42f3-8f30-c16a78ad4d7e",
          "isFrozen": "0",
          "actions": [],
          "updated": "2024-05-31T11:20:38Z",
          "status": "200"
        }
    jobs_execution_test.go:33: create a connection
    jobs_execution_test.go:35: try to get connection
    jobs_execution_test.go:35: create a connection
    jobs_execution_test.go:43: create a script
    jobs_execution_test.go:43: waiting for the script to be created, 3 seconds
    jobs_execution_test.go:52: create a job
    jobs_execution_test.go:102: schedule job cleanup
    jobs_execution_test.go:107: execute a job
    jobs_execution_test.go:112: get job's running status
    jobs_execution_test.go:121: trying to get job results in 60 seconds
    jobs_execution_test.go:104: attempting to delete job: testJob
    jobs_execution_test.go:104: job is deleted: testJob
    jobs_execution_test.go:45: attempting to delete script: testScript
    jobs_execution_test.go:49: script is deleted: testScript
    jobs_execution_test.go:37: attempting to delete connection: testConnection
    jobs_execution_test.go:40: connection is deleted: testConnection
    connections_test.go:121: clean up test cluster
    connections_test.go:122: Attempting to delete DataArts instance: e0516ddf-fb6c-42f3-8f30-c16a78ad4d7e
    connections_test.go:122: DataArts instance deleted: e0516ddf-fb6c-42f3-8f30-c16a78ad4d7e, jobId: 2c9080cf8f1a34f8018fce6206f8038e
    common.go:204: Attempting to delete ECSv1: 5db66f22-bc11-485a-b0d7-0f2737944663
    common.go:221: ECSv1 instance deleted: 5db66f22-bc11-485a-b0d7-0f2737944663
    connections_test.go:139: clean up test bucket
    connections_test.go:135: clean up test key pair
--- PASS: TestDataArtsJobExecution (768.83s)
PASS
otc-zuul[bot] commented 4 months ago

Build failed. https://zuul.otc-service.com/t/eco/buildset/80689080047d46a1b69950aed8843dfc

:heavy_check_mark: otc-golangci-lint SUCCESS in 2m 17s :heavy_check_mark: golang-make-vet SUCCESS in 2m 06s :heavy_check_mark: golang-make-test SUCCESS in 2m 29s :x: golang-make-functional FAILURE in 8m 07s

anton-sidelnikov commented 4 months ago

regate

otc-zuul[bot] commented 4 months ago

Build succeeded. https://zuul.otc-service.com/t/eco/buildset/8e4cbccf142942c0abf2fb739a5f4c6b

:heavy_check_mark: otc-golangci-lint SUCCESS in 2m 14s :heavy_check_mark: golang-make-vet SUCCESS in 1m 50s :heavy_check_mark: golang-make-test SUCCESS in 2m 19s :heavy_check_mark: golang-make-functional SUCCESS in 7m 34s