terascope / teraslice

Scalable data processing pipelines in JavaScript
https://terascope.github.io/teraslice/
Apache License 2.0
50 stars 13 forks source link

teraslice-cli asset deploy should find bundled asset by default #3472

Closed godber closed 5 months ago

godber commented 11 months ago

When trying to deploy a teraslice asset to a cluster without specifying the --bundle argument it fails as shown below:

teraslice-cli assets deploy local terascope/elasticsearch-assets
Unable to download terascope/elasticsearch-assets, you can download a prerelease asset by specifying its version.

Example:
    terascope/elasticsearch-assets@v1.2.3

We intend for the command shown above to automatically download the bundle version by default, I think this line was the attempt:

https://github.com/terascope/teraslice/blob/7188767f6b12d0782258043c92d43d14231c0e9c/packages/teraslice-cli/src/helpers/yargs-options.ts#L63

But for some reason its not working.

Downloading with the --bundle option still works:

teraslice-cli assets deploy local --bundle terascope/elasticsearch-assets@v3.5.4
Downloading terascope/elasticsearch-assets@v3.5.4...
elasticsearch-v3.5.4-... ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 0.0s
Asset posted to local: bf127b4c744c9d34284b026962c2b7b81f5f8e9d
busma13 commented 8 months ago

I ran the latest teraslice-cli from my local repo so I could log some output. Without a bundle flag it will default to bundle:true.

teraslice git:(update-node-versions) ✗ packages/teraslice-cli/bin/teraslice-cli.js assets deploy local terascope/elasticsearch-assets@3.5.4
@@@@ cliConfig:  Config {
  args: {
    '0': '/Users/peterluitjens/WORKSPACE/teraslice/packages/teraslice-cli/bin/teraslice-cli.js',
    '': [ 'assets', 'deploy' ],
    bundle: true,
    configDir: '/Users/peterluitjens/.teraslice',
    d: '/Users/peterluitjens/.teraslice',
    srcDir: '/Users/peterluitjens/WORKSPACE/teraslice',
    clusterAlias: 'local',
    asset: 'terascope/elasticsearch-assets@3.5.4'
  },
  configDir: '/Users/peterluitjens/.teraslice',
  aliases: Aliases {
    aliasesFile: '/Users/peterluitjens/.teraslice/aliases.yaml',
    config: { clusters: [Object] }
  }
}
@@@@ clusterInfo:  {
  arch: 'arm64',
  clustering_type: 'kubernetes',
  name: 'ts-dev1',
  node_version: 'v18.18.2',
  platform: 'linux',
  teraslice_version: 'v0.93.0',
  nodeVersion: 'v18.18.2',
  assetString: 'terascope/elasticsearch-assets@3.5.4',
  bundle: true
}
Downloading terascope/elasticsearch-assets@v3.5.4...

Asset posted to local: bf127b4c744c9d34284b026962c2b7b81f5f8e9d

In fact, when setting --bundle='false' I am not able to download any asset as there are none with arm architecture.

teraslice git:(update-node-versions) ✗ packages/teraslice-cli/bin/teraslice-cli.js assets deploy local --bundle='false' terascope/elasticsearch-assets@3.5.1
@@@@ cliConfig:  Config {
  args: {
    '0': '/Users/peterluitjens/WORKSPACE/teraslice/packages/teraslice-cli/bin/teraslice-cli.js',
    '': [ 'assets', 'deploy' ],
    bundle: false,
    configDir: '/Users/peterluitjens/.teraslice',
    d: '/Users/peterluitjens/.teraslice',
    srcDir: '/Users/peterluitjens/WORKSPACE/teraslice',
    clusterAlias: 'local',
    asset: 'terascope/elasticsearch-assets@3.5.1'
  },
  configDir: '/Users/peterluitjens/.teraslice',
  aliases: Aliases {
    aliasesFile: '/Users/peterluitjens/.teraslice/aliases.yaml',
    config: { clusters: [Object] }
  }
}
@@@@ clusterInfo:  {
  arch: 'arm64',
  clustering_type: 'kubernetes',
  name: 'ts-dev1',
  node_version: 'v18.18.2',
  platform: 'linux',
  teraslice_version: 'v0.93.0',
  nodeVersion: 'v18.18.2',
  assetString: 'terascope/elasticsearch-assets@3.5.1',
  bundle: false
}
Unable to download terascope/elasticsearch-assets@3.5.1 asset: Error: Error downloading terascope/elasticsearch-assets@3.5.1: Error: Could not find a release for terascope/elasticsearch-assets (darwin arm64)
    at GithubAsset.download (/Users/peterluitjens/WORKSPACE/teraslice/packages/teraslice-cli/dist/src/helpers/github-asset.js:58:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Object.handler (/Users/peterluitjens/WORKSPACE/teraslice/packages/teraslice-cli/dist/src/cmds/assets/deploy.js:100:29)
godber commented 5 months ago

I'm going to close this as non-reproducible. Ignoring the obvious architecture problem. We should just remove non-bundled assets.