Open-EO / FuseTS

Time series Fusion toolbox integrated with openEO
https://open-eo.github.io/FuseTS/
Other
22 stars 4 forks source link

Automatically upload dependencies to artifactory after build #94

Open JanssenBrm opened 1 year ago

JanssenBrm commented 1 year ago

The FuseTS services are reliant on a specific set of dependencies (such as vam.whittaker, GPy, etc.) and its own code. Consequently, it is essential for these dependencies to be installed within the execution environment. While accomplishing this for local execution is relatively straightforward, it presents greater challenges within the context of openEO.

In the current version of the library, the dependencies are provided through openEO's job_options, as shown below:

mogpr_job = mogpr.execute_batch('./mogpr.json', out_format="json", title=f'FuseTS - MOGPR', job_options={
    'udf-dependency-archives': [
        'https://artifactory.vgt.vito.be:443/auxdata-public/ai4food/fusets_venv.zip#tmp/venv',
        'https://artifactory.vgt.vito.be:443/auxdata-public/ai4food/fusets.zip#tmp/venv_static'
    ],
    'executor-memory': '7g'
})

Even though this works as is, it does require an update of the archives each time a change is done in the FuseTS library. A possible improvement would be to either: