Support Versions for V2 Export

[sgokaram-saagie]

projectExport should support include_all_versions option for Jobs and Pipelines similar to ExportV1

[ZouhairBear]

How to test :

Version to test : 2.1.16

The Export phase:

Create a build file: Example of build.projectsExport.gradle:

plugins {
  id "io.saagie.gradle-saagie-dataops-plugin" version "2.1.12"
}
saagie {
    server {
        url = "REPLACE_SAAGIEURL"
        login = "REPLACE_SAAGIELOGIN"
    password = "REPLACE_SAAGIEPASSWORD"
        environment = "REPLACE_SAAGIE"
    jwt = true
    }
    project {
        id = "REPLACE_SAAGIE_PROJECT_ID"

    }
     job {
        ids = ['jobId1' ....]
        include_all_versions=true
    }
    pipeline {
        ids = ['pipelineId1'....]
        include_job=true
        include_all_versions=true
    }
    exportArtifacts {
          export_file = "./job/bash.zip" // <=== url "./job/" should exist  
          overwrite=true  // <=== temporary_directory ./tmp'  should exist        
    }
}

Export job using this commande : The command line : gradle -b build.projectsExport.gradle projectsExport

The import phase:

Create a new build file: Example : build.projectsImport.gradle:

plugins {
    id 'groovy'
    id 'io.saagie.gradle-saagie-dataops-plugin' version '2.1.16'
}

saagie {
    server {
        url = "REPLACE_SAAGIEURL"
        login = "REPLACE_SAAGIELOGIN"
    password = "REPLACE_SAAGIEPASSWORD"
        environment = "REPLACE_SAAGIE"
    jwt = true
        acceptSelfSigned = true
    }
    project {
        id = "REPLACE_SAAGIE_PROJECT_ID"
    }
    importArtifacts {
        import_file = "./job/export.zip"
        temporary_directory='./tmp'
    }
}

Then use this command line: The command line : gradle -b build.projectsImport.gradle projectsImport

Expected behvior

> Task :projectsImport
{status=success, job=[{id=job_id_v2, name=bash job name}], pipeline=[]}

BUILD SUCCESSFUL in 5s
1 actionable task: 1 executed

[ZouhairBear]

• I have tested export Job,pipeline from V2 with include_all_versions 2.1.16 : it works as expected

  

• I have tested import Job,pipeline from V2 : it works as expected

  

[sgokaram-saagie]

There is an error in certain cases for import. all.zip Try import the attached all.zip into a new project

task projectImportAll ( // <======== Import artifacts, see that we use the gradle dependOn type: ProjectsImportJobTask) { dependsOn(['projectExportAll']) configuration = saagie { server { url = saagieurl login = saagieuserid password = saagiepassword environment = saagieplatformid jwt = true timeout = saagietimeout } } doFirst { projectImportAll.configuration = saagie { newTaskContext {} project { id = 'bdbd16b2-9010-4fe4-b355-65af1c586c8f' } importArtifacts { import_file = './all/all.zip' temporary_directory= tmpLocation } } } description= 'Import Variables from V1 into V2 Project Variables' taskName = 'projectImportAllV1Var' }

[medamineziraoui]

I tested the zip all.zip

With this script

plugins {
    id 'groovy'
    id 'io.saagie.gradle-saagie-dataops-plugin' version '2.1.16'
}

saagie {
    server {
        url = saagieurl
        login = saagieuserid
        password = saagiepassword
        environment = saagieplatformid
        jwt = true
        acceptSelfSigned = true
    }

    project {
        id = "43aa5302-c5de-4a1e-a54e-6bd5742cc1cf"
    }

    importArtifacts {
        import_file = "/home/amine/Desktop/test_gradle/all.zip"
        temporary_directory = '/home/amine/Desktop/test_gradle/tmp'
    }
}

I got 400 error first time, but then I run it again and I got BUILD SUCCESSFUL

❯ gradle -b build.import.gradle projectsImport   
Starting a Gradle Daemon, 1 busy and 1 stopped Daemons could not be reused, use --status for details

> Task :projectsImport
{status=success, job=[{id=50e7ccd7-7ed4-46c6-87fa-9dc535560c4a, name=ERP Data Cleansing, versions=29}, {id=012e1b69-3267-4155-a9ee-7ea5a68d87af, name=Predictive Maintenance - Prepare data}, {id=d50f6a72-f30f-4522-afd8-1615377f7a44, name=Import ERP Data, versions=17}, {id=c0c10e39-0269-4975-97bc-a223ee9a9212, name=Demo Projects 2, versions=4}, {id=9364fc99-32a5-4dfb-9739-2d8e73d4eea0, name=GradlePluginMigraiton-Manual, versions=5}, {id=c9875e58-8c52-4fb6-80b6-bdf4a9084e3a, name=Build new Model, versions=15}, {id=6d5785a5-d870-4fc7-a509-4647e7322b8f, name=Rscript 03_load_to_pcx.R}, {id=6b525aef-4a5c-4034-aeb2-94e19abf6463, name=pm Extraction batch PLCs, versions=31}, {id=c0a0191b-816f-4424-9d21-dd329aca3a66, name=test_koalas, versions=2}, {id=b28eecf7-f5c4-4aeb-a12a-a4542aa90bf0, name=Build model (sales / ERP), versions=15}, {id=9aca2fc4-fb57-48ed-9d3d-515af07c972d, name=talend job, versions=2}, {id=6716d858-2b9f-4bcf-be6f-21ba191c3c93, name=Creating Job2  2.1.16}, {id=a6a5e83c-05f7-456b-ad17-14fe140914aa, name=Creating Job1  2.1.16}, {id=e26a3ec2-c8b3-4f5b-91bb-b7c5c9a32ff4, name=jupyter/scipy-notebook, versions=14}], pipeline=[{id=f9ad708c-9de5-489e-a3cb-1023d0c57ecf, name=Pipeline 2.1.16 }, {id=d4e94e2c-fdeb-403d-8ffe-a89d316c4279, name=Churn Update model, versions=120}], variable=[]}

Do you mean data inconsistency when exported imported the zip file?

[ZouhairBear]

• This issue was rejected but we could not reproduce the same bug.