search

GoogleCloudPlatform / PerfKitBenchmarker

PerfKit Benchmarker (PKB) contains a set of benchmarks to measure and compare cloud offerings. The benchmarks use default settings to reflect what most users will see. PerfKit Benchmarker is licensed under the Apache 2 license terms. Please make sure to read, understand and agree to the terms of the LICENSE and CONTRIBUTING files before proceeding.
https://googlecloudplatform.github.io/PerfKitBenchmarker/
Apache License 2.0
1.88k stars 478 forks source link

No BaseVirtualMachine subclass defined with the attributes for OpenStack #1866

Open gokhan27 opened 5 years ago

gokhan27 commented 5 years ago

I am trying to run benchmarks with perfkit on My OpenStack Environment. When I run './pkb.py --cloud=OpenStack --machine_type=m1.medium --openstack_network=private --benchmarks=ping --image=xenial', I am getting "SubclassNotFoundError: No BaseVirtualMachine subclass defined with the attributes: {'OS_TYPE': 'ubuntu1604', 'CLOUD': 'OpenStack'}" error. Full logs are below: ubuntu@xenialperfkit:~/PerfKitBenchmarker$ ./pkb.py --cloud=OpenStack --machine_type=m1.medium --openstack_network=private --benchmarks=ping --image=xenial 2019-03-05 07:15:33,185 6e735b77 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/6e735b77/pkb.log 2019-03-05 07:15:33,186 6e735b77 MainThread INFO PerfKitBenchmarker version: v1.12.0-1027-g00463d2 2019-03-05 07:15:33,186 6e735b77 MainThread INFO Flag values: --machine_type=m1.medium --cloud=OpenStack --openstack_network=private --benchmarks=ping --image=cirros-0.3.4-x86_64-uec 2019-03-05 07:15:33,357 6e735b77 MainThread INFO Setting --max_concurrent_threads=200. 2019-03-05 07:15:33,382 6e735b77 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo. 2019-03-05 07:15:33,495 6e735b77 MainThread ping(1/1) INFO Provisioning resources for benchmark ping 2019-03-05 07:15:33,495 6e735b77 MainThread ping(1/1) ERROR Error during benchmark ping Traceback (most recent call last): File "/home/ubuntu/PerfKitBenchmarker/perfkitbenchmarker/pkb.py", line 726, in RunBenchmark DoProvisionPhase(spec, detailed_timer) File "/home/ubuntu/PerfKitBenchmarker/perfkitbenchmarker/pkb.py", line 520, in DoProvisionPhase spec.ConstructVirtualMachines() File "/home/ubuntu/PerfKitBenchmarker/perfkitbenchmarker/benchmark_spec.py", line 404, in ConstructVirtualMachines vms = self.ConstructVirtualMachineGroup(group_name, group_spec) File "/home/ubuntu/PerfKitBenchmarker/perfkitbenchmarker/benchmark_spec.py", line 343, in ConstructVirtualMachineGroup vm = self._CreateVirtualMachine(group_spec.vm_spec, os_type, cloud) File "/home/ubuntu/PerfKitBenchmarker/perfkitbenchmarker/benchmark_spec.py", line 659, in _CreateVirtualMachine vm_class = virtual_machine.GetVmClass(cloud, os_type) File "/home/ubuntu/PerfKitBenchmarker/perfkitbenchmarker/virtual_machine.py", line 75, in GetVmClass CLOUD=cloud, OS_TYPE=os_type) File "/home/ubuntu/PerfKitBenchmarker/perfkitbenchmarker/resource.py", line 47, in GetResourceClass (base_class.name, kwargs)) SubclassNotFoundError: No BaseVirtualMachine subclass defined with the attributes: {'OS_TYPE': 'ubuntu1604', 'CLOUD': 'OpenStack'} 2019-03-05 07:15:33,496 6e735b77 MainThread ping(1/1) ERROR Exception running benchmark Traceback (most recent call last): File "/home/ubuntu/PerfKitBenchmarker/perfkitbenchmarker/pkb.py", line 874, in RunBenchmarkTask RunBenchmark(spec, collector) File "/home/ubuntu/PerfKitBenchmarker/perfkitbenchmarker/pkb.py", line 726, in RunBenchmark DoProvisionPhase(spec, detailed_timer) File "/home/ubuntu/PerfKitBenchmarker/perfkitbenchmarker/pkb.py", line 520, in DoProvisionPhase spec.ConstructVirtualMachines() File "/home/ubuntu/PerfKitBenchmarker/perfkitbenchmarker/benchmark_spec.py", line 404, in ConstructVirtualMachines vms = self.ConstructVirtualMachineGroup(group_name, group_spec) File "/home/ubuntu/PerfKitBenchmarker/perfkitbenchmarker/benchmark_spec.py", line 343, in ConstructVirtualMachineGroup vm = self._CreateVirtualMachine(group_spec.vm_spec, os_type, cloud) File "/home/ubuntu/PerfKitBenchmarker/perfkitbenchmarker/benchmark_spec.py", line 659, in _CreateVirtualMachine vm_class = virtual_machine.GetVmClass(cloud, os_type) File "/home/ubuntu/PerfKitBenchmarker/perfkitbenchmarker/virtual_machine.py", line 75, in GetVmClass CLOUD=cloud, OS_TYPE=os_type) File "/home/ubuntu/PerfKitBenchmarker/perfkitbenchmarker/resource.py", line 47, in GetResourceClass (base_class.name, kwargs)) SubclassNotFoundError: No BaseVirtualMachine subclass defined with the attributes: {'OS_TYPE': 'ubuntu1604', 'CLOUD': 'OpenStack'} 2019-03-05 07:15:33,497 6e735b77 MainThread ping(1/1) ERROR Benchmark 1/1 ping (UID: ping0) failed. Execution will continue. 2019-03-05 07:15:33,497 6e735b77 MainThread ping(1/1) INFO Benchmark run statuses:

Name UID Status Failed Substatus

ping ping0 FAILED

Success rate: 0.00% (0/1) 2019-03-05 07:15:33,497 6e735b77 MainThread ping(1/1) INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/6e735b77/pkb.log 2019-03-05 07:15:33,497 6e735b77 MainThread ping(1/1) INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/6e735b77/completion_statuses.json

I am not sure this is bug or not. Maybe I am missing something. Can you help me for solve this issue ?

s-deitz commented 5 years ago

I haven't used OpenStack, but looking in providers/openstack/os_virtual_machine.py, I see a Debian baseclass, but not Ubuntu 16.04. Can you run with --os_type=debian? This code may be stale and need some attention.

Since Ubuntu 16.04 is our default, we may need to add a base class in os_virtual_machine.py as well. If you are able to take a look, we can look at a PR.

jharbott commented 5 years ago

I was seeing the same issue, adding --os_type=debian and modifying providers/openstack/os_virtual_machine.py to use ubuntu-16.04 instead of 14.04 has gotten the iperf test running fine for me. As the debian class seems to be a "debian-based-OS-type" class anyway, defaulting to Ubuntu, adding an ubuntu class directly so that the default parameters work, would seem like the proper solution. Adding a note to the readme about the expected image name might also be helpful for starters.