search

GoogleCloudPlatform / PerfKitBenchmarker

PerfKit Benchmarker (PKB) contains a set of benchmarks to measure and compare cloud offerings. The benchmarks use default settings to reflect what most users will see. PerfKit Benchmarker is licensed under the Apache 2 license terms. Please make sure to read, understand and agree to the terms of the LICENSE and CONTRIBUTING files before proceeding.
https://googlecloudplatform.github.io/PerfKitBenchmarker/
Apache License 2.0
1.89k stars 480 forks source link

VMware vCloud Director provider #2718

Open kgawda opened 3 years ago

kgawda commented 3 years ago

I would like to implement a provider module for VMware vCloud Director based clouds. I already have some PoC code.

Where should I document taken design decisions? Any other hints beside ones form wiki and CONTRIBUTING.md?

I am aware of Issue #1260, but it quite old and does not specify kind of VMware product. So I decided to create a new one.

bvliu commented 3 years ago

Adding @cwilkes for more insight on vmware.

cwilkes commented 3 years ago

Hi @kgawda I'll shepherd your VMware provider through the process of getting into PKB. Since you already have a PoC can you share a link to that and we can talk about a design doc?

kgawda commented 2 years ago

Hi! I know it's been a long time time, but finally I had the space to move this forward. This is a branch with some usable prototype implementation of VMware provider (for Cloud Director): https://github.com/kgawda/PerfKitBenchmarker/tree/vmware-provider Most importantly, there is a brief description of decisions (to be discussed), questions and problems: https://github.com/kgawda/PerfKitBenchmarker/blob/vmware-provider/tutorials/vmware/README.md

Looking forward for any comments and hints!

kgawda commented 2 years ago

@s-deitz, I heard from @cwilkes that you could step in and lead me. I would be grateful for any comments or suggestions.

s-deitz commented 2 years ago

Thanks Konrad. I'm trying to find someone on the team who can help review.

jellyfishcake commented 2 years ago

Hi, Konrad, thank you for this contribution! We really appreciate the addition of coverage to VMWare. This is a very big change, and before I review the actual code, I'd like to discuss some of the design decisions you took. Disclaimer: I personally have not used VMware before, and do not have access to it right now. Therefore I am relying heavily on your expertise/insight here. The more details/explanations you can offer, the better. Thank you! 1) Single provider for different VMware products I want to make sure that if we were to add other VMware products in future, we have a way to do so. Does VMware have VMs/Disks for different products? i.e. I see you added vmware_vcd_virtual_machine.py, which I am assuming is Cloud Director virtual machine. So in future, we perhaps can add vmware_vcenter_virtual_machine.py? 2) Regarding pyvcloud, it does concern me a little that we are deviating from the common pattern in pkb, I need to sync up with my team on why CLI was picked originally, and get back to you. 3) credentials in ENV seems reasonable too. But we have to keep in mind that when there are more users, others may prefer a different way. So I think offering lots of different options encourage usability. If users want to update a bashrc file or supply a flag, they should be able to as well. 4) If we go ahead with pyvcloud, I would say we should pull from the default package in the long run. Unfortunately, pyvcloud does not look very active. It might be fine now but I worry that your branch and the main branch may diverge in future. What is your thoughts on that? 5) Public IP shared between multiple VMs - How does VMware knows which VM a public ip is referring to? If you ssh to one such public ip address, does VMware act as a loadbalancer and pick on VM at random or does it deterministically know which VM? Could we disable the public ips in this case and allow usage of private ips? 6&7) Unfortunately, since everything else in pkb thus far has been cli clients and using credential files, there is likely no mechanism to persist token / test different env for the same provider. It is definitely a challenge to adopting a python client. 8&9) you will need to add a BENCHMARK_CONFIG and corresponding parser, and we currently have a data_disk_type flag (defined in pkb.py) that let's one choose the disk type (for example on GCP it is --data_disk_type=pd-ssd to use a pd-ssd disk).