Create a workflow to sign Tableau Connector

MaxKsyunz commented 2 years ago

Is your feature request related to a problem? Please describe

There is a connector for Tableau to connect to OpenSearch but it is complicated to install.

It'd be great if the connector was published on Tableau Marketplace as described in opensearch-project/sql-jdbc#26. To do so, the connector needs to be signed.

Describe the solution you'd like

A workflow that takes the TACO file generated by this workflow and signs it.

Describe alternatives you've considered

No response

Additional context

No response

prudhvigodithi commented 2 years ago

[Triage] Hey @MaxKsyunz can you please add some more details, what type of signing is expected to be part of the workflow? Following is the link that shows the supported signing formats. https://github.com/opensearch-project/opensearch-build#signing-artifacts

MaxKsyunz commented 2 years ago

@prudhvigodithi according to documentation, it to be signed by jarsigner.

Can this be done by the build-signer?

anirudha commented 1 year ago

@prudhvigodithi any updates on this one ?

anirudha commented 1 year ago

can we test signing it with a jar signer and we can test with Tableau

prudhvigodithi commented 1 year ago

Hey @MaxKsyunz and @anirudha the jarsigner is not in supported list of the signing formats.

Following are the action items to move forward:

Configure the jarsigner setup.
Incorporate the jarsigner to existing signing workflow.
Create/manage account with tableau marketplace.
Automation to incorporate with universal release mechanism.

@bbarani @dblock @peterzhuamazon @gaiksaya

prudhvigodithi commented 1 year ago

Hey @MaxKsyunz and @anirudha can you please share the priority level for this Tableau Connector release? Like I see the steps on how to install, but agree that it makes easy if its part of Marketplace, based on the priority we can create an issue in this build repo and explore jarsigner setup along with having an account with Tableau Marketplace. @bbarani

anirudha commented 1 year ago

Tableau gallery is waiting on this for a while, whats the blocker or estimate here ?

anirudha commented 1 year ago

@bbarani for adding a priority on this

bbarani commented 1 year ago

@prudhvigodithi Can you list the next steps along with ballpark estimate on this issue?

prudhvigodithi commented 1 year ago

Updated link for TableauConnector with installation steps.

prudhvigodithi commented 1 year ago

Hey to begin the ask is to directly publish the tableau connector to the OpenSearch Website Download page, this would have a new entry under download section which says as Tableau Connector, the .taco file would have signature validation with .asc extension (similar to existing JDBC Driver under OpenSearch Website Download page).

Once its published to the website then the process to publish to Tableau Exchange marketplace is the next step, can you confirm this @anirudha @brijos ?

Following are steps to publish to the OpenSearch Website Download page 1) On board the tableau connector to universal release mechanism (Similar to existing sql-jdbc) 2) Based on tag, create a release and push to https://artifacts.opensearch.org/opensearch-clients/tableau/connector/VERSION/TACO_FILE. 3) Update the documentation website for a user to choose the right version () and download the right connector (). @krisfreedain is this possible form documentation end ? meaning a user should be able to choose a drop of version for individual components on the website and download the right version file, existing setup for individual components on the website is not supported. @bbarani @gaiksaya @dblock

brijos commented 1 year ago

From my side, I need a link to the signed .taco file so that Tableau can begin their testing.

rishabh6788 commented 1 year ago

Echoing @prudhvigodithi comments, i believe we can provide the connector under Downloads section of opensearch.org which would be signed similar to how we are signing maven jars as of now, provide the .asc signature file to provide certain level of integrity to the file. Meanwhile we can work on integrating jarsigner with our opensearch-signer client or see if other tools can provide the feature. @prudhvigodithi @bbarani

bbarani commented 1 year ago

@anirudha @brijos Integrating jarsigner with our signer client is going to take some time since we are exploring multiple options. In the meantime, can we move ahead with publishing the connector under Downloads along with .asc signature file ?

acarbonetto commented 1 year ago

@bbarani @rishabh6788 sorry to tag you, but is there an update on the jarsigner? I'm wondering if you have resolved the blockers yet.

bbarani commented 1 year ago

@acarbonetto We are able to sign the Tableau connector successfully and we are currently validating it. @brijos @anirudha can provide additional details.

gaiksaya commented 1 year ago

Closing this issue as jar signer used to sign .jar and .taco files is integrated with CI system. See above linked PRs

Yury-Fridlyand commented 1 year ago

Jenkins job failed @gaiksaya: https://build.ci.opensearch.org/blue/organizations/jenkins/sql-jdbc-release/detail/sql-jdbc-release/8/pipeline

2023-08-22 18:30:21 INFO     Executing "./opensearch-signer-client -i /var/jenkins/workspace/sql-jdbc-release/shadowJar/opensearch-sql-jdbc-shadow-1.4.0.1.jar -o /var/jenkins/workspace/sql-jdbc-release/shadowJar/opensearch-sql-jdbc-shadow-1.4.0.1.jar -p jar_signer -r True" in /tmp/tmp6autz76q/src
Using environment variable configuration.
Traceback (most recent call last):
  File "/tmp/tmp6autz76q/src/opensearch-signer-client.py", line 405, in <module>
    sign(source, target, platform, config_file, allow_output_overwrite)
  File "/tmp/tmp6autz76q/src/opensearch-signer-client.py", line 259, in sign
    sign_jar(source, target, config.signer_info_vars, allow_output_overwrite)
  File "/tmp/tmp6autz76q/src/opensearch-signer-client.py", line 222, in sign_jar
    signer_s3_client = get_signer_s3_session(role_arn, external_id)
  File "/tmp/tmp6autz76q/src/opensearch-signer-client.py", line 145, in get_signer_s3_session
    response = sts.assume_role(
  File "/tmp/tmp6autz76q/src/.venv/lib/python3.9/site-packages/botocore/client.py", line 535, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/tmp/tmp6autz76q/src/.venv/lib/python3.9/site-packages/botocore/client.py", line 980, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the AssumeRole operation: User: arn:aws:sts::023816108377:assumed-role/OpenSearch-CI-AgentNodeRole/i-05b943cce40c7edbd is not authorized to perform: sts:AssumeRole on resource: ****
Traceback (most recent call last):
  File "/var/jenkins/workspace/sql-jdbc-release/opensearch-build/src/run_sign.py", line 35, in <module>
    sys.exit(main())
  File "/var/jenkins/workspace/sql-jdbc-release/opensearch-build/src/run_sign.py", line 30, in main
    sign.sign()
  File "/var/jenkins/workspace/sql-jdbc-release/opensearch-build/src/sign_workflow/sign_artifacts.py", line 41, in sign
    self.__sign__()
  File "/var/jenkins/workspace/sql-jdbc-release/opensearch-build/src/sign_workflow/sign_artifacts.py", line 83, in __sign__
    super().__sign_artifact__(artifacts, basename)
  File "/var/jenkins/workspace/sql-jdbc-release/opensearch-build/src/sign_workflow/sign_artifacts.py", line 48, in __sign_artifact__
    self.signer.sign_artifact(artifact, basepath, self.signature_type)
  File "/var/jenkins/workspace/sql-jdbc-release/opensearch-build/src/sign_workflow/signer.py", line 32, in sign_artifact
    self.generate_signature_and_verify(artifact, basepath, signature_type)
  File "/var/jenkins/workspace/sql-jdbc-release/opensearch-build/src/sign_workflow/signer_jar.py", line 25, in generate_signature_and_verify
    self.sign(artifact, basepath, signature_type)
  File "/var/jenkins/workspace/sql-jdbc-release/opensearch-build/src/sign_workflow/signer_jar.py", line 47, in sign
    self.git_repo.execute(" ".join(signing_cmd))
  File "/var/jenkins/workspace/sql-jdbc-release/opensearch-build/src/git/git_repository.py", line 85, in execute
    subprocess.check_call(command, cwd=cwd, shell=True)
  File "/usr/local/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command './opensearch-signer-client -i /var/jenkins/workspace/sql-jdbc-release/shadowJar/opensearch-sql-jdbc-shadow-1.4.0.1.jar -o /var/jenkins/workspace/sql-jdbc-release/shadowJar/opensearch-sql-jdbc-shadow-1.4.0.1.jar -p jar_signer -r True' returned non-zero exit status 1.

script returned exit code 1

gaiksaya commented 1 year ago

Looking into it! @Yury-Fridlyand

gaiksaya commented 1 year ago

The run was succesful and shadow jar has been released using the workflow: https://build.ci.opensearch.org/view/Release/job/sql-jdbc-release/11/console https://artifacts.opensearch.org/opensearch-clients/jdbc/opensearch-sql-jdbc-shadow-1.4.0.1.jar

Thanks! Closing this issue!

opensearch-project / opensearch-build