f5devcentral / f5-bigip-image-generator

Generate custom images for F5 BIG-IP.
Apache License 2.0
34 stars 20 forks source link

Alibaba: option to use the internal OSS endpoint #33

Closed amolari closed 2 years ago

amolari commented 3 years ago

Is your feature request related to a problem? Please describe. Using the internal OSS endpoint, the upload is quite faster and as it doesn't goes through the Internet, we're not required to upload that large files through a webproxy.

Describe the solution you'd like Add an option "use OSS internal endpoint) in the config file, which when value=true, toggles the OSS Endpoint URl to be the internal endpoint. Practically, that would change the file src/lib/python/image/alibaba_disk.py : https://github.com/f5devcentral/f5-bigip-image-generator/blob/8617feca7f3c8f8d0af62e91eb712802f86fe906/src/lib/python/image/alibaba_disk.py#L121 to self.bucket = oss2.Bucket(auth, 'https://oss-' + region + '-internal' + '.aliyuncs.com', bucket_name)

and https://github.com/f5devcentral/f5-bigip-image-generator/blob/8617feca7f3c8f8d0af62e91eb712802f86fe906/src/lib/python/image/alibaba_disk.py#L124 to alibaba_location_json = {"alibaba_location": 'https://oss-{}-internal.aliyuncs.com'.format(region)}

Additional context Note: To use the OSS internal endpoint URL when using a webproxy by default, the following must be added no_proxy env variables: .oss-<region>-internal.aliyuncs.com

kmaloney commented 3 years ago

I have confirmed the performance improvement using the internal OSS endpoint.

The solution will essentially be what you proposed except that the two endpoint declarations (line 121/124) will just be one declaration that is used in both places. This should have been done this way initially as you would have only had to make one change rather than two to modify the behavior.

This issue will be tracked internally using VECLOUD-1275.

kmaloney commented 2 years ago

Resolved in v1.16. Use of internal endpoint can be enabled using the ALIBABA_UPLOAD_INTERNAL_OSS_ENDPOINT configuration parameter. See Alibaba docs.