search

CloudVE / cloudbridge

A consistent interface to multiple IaaS clouds; in Python.
https://cloudbridge.cloudve.org
MIT License
113 stars 51 forks source link

Separate the Cloud Provider Package Installation Completely #291

Closed RyanSiu1995 closed 2 years ago

RyanSiu1995 commented 2 years ago

I have spotted that the SIMPLE or default installation will install all three cloud providers, AWS, GCP and OpenStack. However, in my case, I only use AWS and GCP so it makes my environment having one more unused dependency. This is not an ideal situation. If possible, could we have the package separated installation just like Azure? This make everything aligned and easy to manage.

More specifically, the dependency in OpenStack introduces pyinotify and it is not compatible with MacOS. So, we need to cut it off because we are not using it actually.

nuwang commented 2 years ago

@RyanSiu1995 Thanks for making PR (https://github.com/CloudVE/cloudbridge/pull/292) and initiating this discussion. The original reason for making the simple install include some providers was to make cloudbridge usable immediately by new users as otherwise, it would install a non-functional cloudbridge. We cut Azure out because it was simply too large to include by default.

However, maybe it doesn't matter so much, and allowing full control over which providers are installed is better. We would have to make a major release to include this change, but that's not a real problem. @afgane @almahmoud @FabioRosado Any thoughts?

nuwang commented 2 years ago

@RyanSiu1995 But on another note, I'm not running into any issues on mac with pip install cloudbridge. And pyinotify does not appear to be an openstack dependency anymore, are you installing the latest version (2.2.0) with updated openstack libs?

appdirs==1.4.4
attrs==21.2.0
autopage==0.4.0
boto3==1.20.7
botocore==1.23.7
cachetools==4.2.4
certifi==2021.10.8
cffi==1.15.0
charset-normalizer==2.0.7
cliff==3.9.0
cloudbridge==2.2.0
cmd2==2.3.0
cryptography==35.0.0
debtcollector==2.3.0
decorator==5.1.0
deprecation==2.1.0
dogpile.cache==1.1.4
google-api-core==2.2.2
google-api-python-client==2.31.0
google-auth==2.3.3
google-auth-httplib2==0.1.0
googleapis-common-protos==1.53.0
httplib2==0.20.2
idna==3.3
iso8601==1.0.0
jmespath==0.10.0
jsonpatch==1.32
jsonpointer==2.2
keystoneauth1==4.4.0
msgpack==1.0.2
munch==2.5.0
netaddr==0.8.0
netifaces==0.11.0
openstacksdk==0.59.0
os-client-config==2.1.0
os-service-types==1.7.0
osc-lib==2.4.2
oslo.config==8.7.1
oslo.context==3.4.0
oslo.i18n==5.1.0
oslo.log==4.6.1
oslo.serialization==4.2.0
oslo.utils==4.11.0
packaging==21.2
pbr==5.7.0
prettytable==2.4.0
protobuf==3.19.1
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.21
pyeventsystem==0.1.0
pyparsing==2.4.7
pyperclip==1.8.2
python-dateutil==2.8.2
python-keystoneclient==4.3.0
python-neutronclient==7.6.0
python-novaclient==17.6.0
python-swiftclient==3.12.0
pytz==2021.3
PyYAML==6.0
requests==2.26.0
requestsexceptions==1.4.0
rfc3986==1.5.0
rsa==4.7.2
s3transfer==0.5.0
simplejson==3.17.6
six==1.16.0
stevedore==3.5.0
tenacity==8.0.1
uritemplate==4.1.1
urllib3==1.26.7
wcwidth==0.2.5
wrapt==1.13.3
RyanSiu1995 commented 2 years ago

Regarding to the pyinotify, we are using pip-compile to compile the nested dependencies. Unfortunately, the pip-compile does not respect the sys_platform for some reasons. So, we encounter the issue of pyinotify in this case.

For your information, the pyinotify is specified under this dependency tree. https://github.com/openstack/python-neutronclient/blob/master/requirements.txt#L11 https://github.com/openstack/oslo.log/blob/master/requirements.txt#L12

But no matter how, it will be great if we can have a bigger control in what kind of providers are installed. When this package grows in the future, for example, including alicloud or other clouds, the controlling in the provider selection is a must. You don't want to include more and more unexpected packages installed in your runtime. It makes the deliverable to be bigger and also raises some security concerns

nuwang commented 2 years ago

@RyanSiu1995 Just to let you know that we are taking this up for discussion on tuesday and can hopefully merge this then. I tend to agree with your argument.

FabioRosado commented 2 years ago

Hello folks, apologies for the delay in getting back to you. Yeah it should be fine, we are installing the full cloudbridge so the changes shouldn't affect us in a bad way :smile:

RyanSiu1995 commented 2 years ago

Hi folks, any update on this?

nuwang commented 2 years ago

@RyanSiu1995 Thanks for these changes, the PR has been merged.

RyanSiu1995 commented 2 years ago

Thank you for helping this out! I wonder when the next release will be here.

nuwang commented 2 years ago

@FabioRosado Can you comment on ETA for this: https://github.com/CloudVE/cloudbridge/issues/293? We could probably make a release with that change included?

FabioRosado commented 2 years ago

@nuwang I need to keep working on it, perhaps a week or so? I'm working on this in-between baby duties atm :smile:

nuwang commented 2 years ago

No rush at all on that, baby duties come first :-) Let's release this now, and we can always do a 3.1 later.

nuwang commented 2 years ago

The 3.0.0 release is out: https://github.com/CloudVE/cloudbridge/releases/tag/v3.0.0

Thanks to both for these contributions @RyanSiu1995 @FabioRosado