search

lithops-cloud / lithops

A multi-cloud framework for big data analytics and embarrassingly parallel jobs, that provides an universal API for building parallel applications in the cloud ☁️🚀
http://lithops.cloud
Apache License 2.0
317 stars 105 forks source link

Breaking changes in the new version #1113

Closed sergii-mamedov closed 1 year ago

sergii-mamedov commented 1 year ago

I tried to upgrade Lithops from version 2.7.0 to 2.9.0. I had the following difficulties:

  1. COS

    • storage_bucket Was not mandatory before 2.8.0, mandatory 2.8.0-2.9.0. And now again not mandatory

  2. VPC

    • ip_address -> floating_ip. Changed on 2.9.0 However, the source code (lithops/lithops/standalone/backends/ibm_vpc/ibm_vpc.py:406) still expects ip_address
    • instance_name There is no description of this option in the documentation for 2.9.0. However, the source code (lithops/lithops/standalone/backends/ibm_vpc/ibm_vpc.py:405) still expects this parameter. This parameter has been removed from the code, it is not in master.

I think it would be very useful for users to have a list of all breaking changes in one place. This would significantly save time during the upgrade.

abourramouss commented 1 year ago

Something to add here, support for aws temporary credentials was not added until 2.9. Before that working with temporary credentials wasn't supported.

I had lost some time thinking my aws configuration was wrong, while the support of temporary credentials wasn't added until 2.9, if that helps to anyone.

JosepSampe commented 1 year ago

Hi @sergii-mamedov, thanks for reporting. Just a couple of comments:

ip_address -> floating_ip. Changed on 2.9.0 However, the source code (lithops/lithops/standalone/backends/ibm_vpc/ibm_vpc.py:406) still expects ip_address

This is a tracked issue that has been solved but unfortunately included in the last release

storage_bucket Was not mandatory before 2.8.0, mandatory 2.8.0-2.9.0. And now again not mandatory

With the exception of the localhost and Infinisapn backends (that don't use buckets) the storage bucket has always been mandatory. Now in master branch it is optional. Can you provide more details on this? What issue have you experienced?

instance_name There is no description of this option in the documentation for 2.9.0. However, the source code (lithops/lithops/standalone/backends/ibm_vpc/ibm_vpc.py:405) still expects this parameter. This parameter has been removed from the code, it is not in master.

This parameter has never been mandatory, it is something internal, so it has never been documented and never necessary in the config. The intance_name is automatically retrieved in https://github.com/lithops-cloud/lithops/blob/2.9.0/lithops/standalone/backends/ibm_vpc/ibm_vpc.py#L397 Can you provide more details on this? If you have to provide intance_name in the config maybe there is an untracked issue. In master branch it has been renamed to master_name, but as I said, it doesn't affect because this parameter is automatically retrieved.

In any case, don't forget to run lithops clean -b <compute_backend> -s <storage_backend> when updating one version to another to remove the cached and temporary data of the older version.

JosepSampe commented 1 year ago

Hi @abourramouss Maybe you were looking at 2.9 docs while using an older version?

The lithops docs in the website shows the docs for the latest release (You can see the version of the docs under the logo in the website). The master branch always shows the docs for the latests changes, so it is not convenient to look at the master branch docs if using an specific release.

If you are using an specific Lithops version, you have to navigate to the specific tag (for example 2.7.0) and then look the docs there

JosepSampe commented 1 year ago

From now on, on each release, you can check the release notes