CPDaaS write files as data-asset fails

[victorterpstra]

Detection is using outdated environment variable setting: os.environ['PWD'] == '/home/dsxuser/work'

Replace detection and data directory to '/home/wsuser/work'

To be fixed in v0.5.1

[victorterpstra]

Writing some data asset has been a challenge since the start (DSX, ICP, WS Local/Cloud, etc). Instead of just writing a file to a directory, in general we need to register it as a data asset. The challenge for dse-do-utils has been that the mechanism(s) to achieve this: a. Are different between various platforms (i.e. CPD-on-prem vs CPDaaS) b. Have changed from version to version. Sometimes without any transition time c. The platforms do not offer a consistent way (e.g. an API) to identify what they are and what version

The code (as of v0.5.1.0b) uses an unreliable method to try and detect the platform/version based on existence and values of environment variables and presence of packages.

The v0.5.1.0b has some fixes that make it work both for CPDaaS (Nov '21) and CPDv4.0.2 with the new/default git. This highly likely to break in future versions.

The 2 main implementation issues:

1. Where to write the file?
   • CPDaaS: os.environ['PWD'] == '/home/wsuser/work'
   • CPD v4.0.2: wslib.mount.get_base_dir()
   • CPDv2.5-3.5: '/project_data/data_asset'
2. What mechanism to use to register the file as a data asset?
   • CPDaaS: project_lib
   • CPD v4.0.2: ibm_watson_studio_lib
   • CPDv2.5-3.5: project_lib

[victorterpstra]

As of v0.5.1.1b, it is possible to force the platform choice in the ScenarioManager constructor. Using the enum class Platform with values:

• CPDaaS: As of Nov 2021, CPDaaS uses project_lib
• CPD40: CPD 4.0 uses ibm_watson_studio_lib
• CPD25: for version CPD 2.5-3.5 uses project_lib (but differently from CPDaaS)
• Local: running locally on a workstation If leaving None (default), the ScenarioManager will auto-detect the platform.