Error in Synthesis

[HFarahat]

Hi,

I got this error when i synthesis the sample base vm and the overlay vm you have provided in theREADME file:

XML: internal error guest CPU is not compatible with host CPU Error, make sure previous VM is closed and check QEMU_ARGUMENT INFO SUCCESS to launch VM ERROR Warning, Possible VNC port error:'NoneType' object has no attribute 'XMLDesc' INFO Resource is allocated for Session: 98175423644247766 INFO [SOCKET] waiting for client exit message

---

Regards,

[krha]

Hello,

As written in the error message you host machine CPU does not support core2duo CPU model that is used in the example base VM. In this case, you need to create your own base VM.

But I'm wondering why your machine does not support core2duo CPU model because it typically works even with pretty old machines. Could you paste output of those commands?

$ kvm -cpu ? $ virsh capabilities $ cat /proc/cpuinfo | grep flags

[HFarahat]

Hi, Here is the output: $ kvm -cpu ? x86 Opteron_G3 x86 Opteron_G2 x86 Opteron_G1 x86 Nehalem x86 Penryn x86 Conroe x86 [n270] x86 [athlon] x86 [pentium3] x86 [pentium2] x86 [pentium] x86 [486] x86 [coreduo] x86 [kvm32] x86 [qemu32] x86 [kvm64] x86 [core2duo] x86 [phenom] x86 [qemu64]

$ virsh capabilities

82263860-e291-11df-9bb9-bcaec530da6e x86_64 Opteron_G3 AMD tcp apparmor 0 hvm 32 /usr/bin/qemu-system-x86_64 pc-1.0 pc pc-0.14 pc-0.13 pc-0.12 pc-0.11 pc-0.10 isapc /usr/bin/kvm pc-1.0 pc pc-0.14 pc-0.13 pc-0.12 pc-0.11 pc-0.10 isapc hvm 64 /usr/bin/qemu-system-x86_64 pc-1.0 pc pc-0.14 pc-0.13 pc-0.12 pc-0.11 pc-0.10 isapc /usr/bin/kvm pc-1.0 pc pc-0.14 pc-0.13 pc-0.12 pc-0.11 pc-0.10 isapc hvm 32 /usr/bin/qemu-system-arm integratorcp beagle beaglexm collie nuri smdkc210 connex verdex highbank mainstone musicpal n800 n810 n900 sx1 sx1-v1 overo cheetah realview-eb realview-eb-mpcore realview-pb-a8 realview-pbx-a9 akita spitz borzoi terrier lm3s811evb lm3s6965evb tosa versatilepb versatileab vexpress-a9 vexpress-a15 z2 hvm 32 /usr/bin/qemu-system-microblaze petalogix-s3adsp1800 petalogix-ml605 hvm 32 /usr/bin/qemu-system-mips malta magnum pica61 mipssim mips hvm 32 /usr/bin/qemu-system-mipsel malta magnum pica61 mipssim mips hvm 32 /usr/bin/qemu-system-sparc SS-5 leon3_generic SS-10 SS-600MP SS-20 Voyager LX SS-4 SPARCClassic SPARCbook SS-1000 SS-2000 SS-2 hvm 32 /usr/bin/qemu-system-ppc g3beige ref405ep taihu bamboo mac99 prep mpc8544ds virtex-ml507

$ cat /proc/cpuinfo | grep flags flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr npt lbrv svm_lock nrip_save pausefilter flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr npt lbrv svm_lock nrip_save pausefilter flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr npt lbrv svm_lock nrip_save pausefilter flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr npt lbrv svm_lock nrip_save pausefilter flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr npt lbrv svm_lock nrip_save pausefilter flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr npt lbrv svm_lock nrip_save pausefilter flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr npt lbrv svm_lock nrip_save pausefilter flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr npt lbrv svm_lock nrip_save pausefilter flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr npt lbrv svm_lock nrip_save pausefilter flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr npt lbrv svm_lock nrip_save pausefilter flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr npt lbrv svm_lock nrip_save pausefilter flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr npt lbrv svm_lock nrip_save pausefilter flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr npt lbrv svm_lock nrip_save pausefilter flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr npt lbrv svm_lock nrip_save pausefilter flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr npt lbrv svm_lock nrip_save pausefilter flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr npt lbrv svm_lock nrip_save pausefilter

[anilce]

Error after re-installation of cloudlet provision master..

overlay transfer and base VM not starting..

[krha]

Hi @anilce

I have just clone from master repo and found no problem. Can you tell me more information to reproduce your problem?

1) Can you create VM overlay using

$ cloudlet overlay [base vm]

2) Can you perform VM synthesis using

$ cloudlet synthesis [base vm] [VM overlay]

3) log message at /var/tmp/cloudlet/log-synthesis

Thanks.

[anilce]

Error after re-installation of cloudlet provision master.. not working with: |synthesis_server | synthesis_client -s localhost -o Downloads/overlay.zip |

1) Can you create VM overlay using

Ans: YES /successfully

2) Can you perform VM synthesis using

Ans: YES /successfully

3) log message at /var/tmp/cloudlet/log-synthesis

----------------------- New Connection -------------- 05-10 17:24 elijah.provisioning.server INFO New client request /home/kdp/.cloudlet/abda52a61692094b3b7d45c9647d022f5e297d1b788679eb93735374007576b8/precise.raw VM 05-11 14:05 elijah.provisioning.synthesis INFO Decompression time : 0.435351 (s) 05-11 14:05 elijah.provisioning.synthesis INFO Recovering launch VM 05-11 14:05 elijah.provisioning.synthesis INFO Start FUSE 05-11 14:05 elijah.provisioning.synthesis INFO Resume the launch VM 05-11 14:05 elijah.provisioning.synthesis INFO Launch disk: /tmp/cloudlet-launch-disk-gIVcJW 05-11 14:05 elijah.provisioning.synthesis INFO Launch memory: /tmp/cloudlet-launch-mem-XIXt0m 05-11 14:05 elijah.provisioning.cloudletfs INFO start monitoring at /var/tmp/cloudletfs-Z1geWo/disk/streams/chunks_modified 05-11 14:05 elijah.provisioning.cloudletfs INFO start monitoring at /var/tmp/cloudletfs-Z1geWo/disk/streams/chunks_accessed 05-11 14:05 elijah.provisioning.cloudletfs INFO start monitoring at /var/tmp/cloudletfs-Z1geWo/memory/streams/chunks_accessed 05-11 14:05 elijah.provisioning.cloudletfs INFO start monitoring at /tmp/cloudlet-overlay-tjqvpc/qemu-trim-log 05-11 14:07 elijah.provisioning.cloudletfs INFO [FUSE] : (1431333303.11)-(1431333479.85)=(176.741596937)

05-11 14:08 elijah.provisioning.delta INFO [Delta] : (1431333303.1)-(1431333481.32)=(178.213143826), delta 38465 chunks 05-11 14:08 elijah.provisioning.synthesis INFO restoring VM... 05-11 14:08 elijah.provisioning.synthesis INFO VM is restored... 05-11 14:08 elijah.provisioning.synthesis INFO [RESUME] : QEMU resume time (1431333482.336626)~(1431333483.092081)=(0.755455) 05-11 14:08 elijah.provisioning.synthesis INFO waiting for finishing VNC interaction 05-11 14:08 elijah.provisioning.cloudletfs INFO stop monitoring at /var/tmp/cloudletfs-Z1geWo/disk/streams/chunks_modified 05-11 14:08 elijah.provisioning.cloudletfs INFO stop monitoring at /var/tmp/cloudletfs-Z1geWo/disk/streams/chunks_accessed 05-11 14:08 elijah.provisioning.cloudletfs INFO stop monitoring at /var/tmp/cloudletfs-Z1geWo/memory/streams/chunks_accessed 05-11 14:08 elijah.provisioning.cloudletfs INFO close Stream monitoring thread 05-11 14:08 elijah.provisioning.synthesis DEBUG ------------------------------------------------- 05-11 14:08 elijah.provisioning.synthesis DEBUG ## Synthesis Statistics (took 0.451414 seconds) ## 05-11 14:08 elijah.provisioning.synthesis DEBUG Overlay acccess count / total overlay count : 4361 / 38465 = 11.34 % 05-11 14:08 elijah.provisioning.synthesis DEBUG Overlay acccess size / total overlay size : 003 MB/ 6.544 MB= 46.05 % 05-11 14:08 elijah.provisioning.synthesis DEBUG Memory Count: Overlay memory acccess / total memory overlay : 4357 / 30493 = 14.29 % 05-11 14:08 elijah.provisioning.synthesis DEBUG Memory Size: Overlay memory acccess / total overlay : 3158743 / 6861593 = 46.04 % 05-11 14:08 elijah.provisioning.synthesis DEBUG Disk Count: Overlay acccess / total disk overlay : 4 / 7972 = 00.05 % 05-11 14:08 elijah.provisioning.synthesis DEBUG Disk Size: Overlay acccess / total overlay : 985 / 6861593 = 00.01 % 05-11 14:08 elijah.provisioning.synthesis DEBUG EXTRA (count): Overlay memory acccess / VM memory access : 4357 / 12608 = 34.56 % 05-11 14:08 elijah.provisioning.synthesis DEBUG EXTRA (count): Overlay disk acccess / VM disk access : 4 / 13 = 30.77 % 05-11 14:08 elijah.provisioning.synthesis DEBUG 1 blobs (3.799213 MB) are required out of 1 (100.00 %) 05-11 14:08 elijah.provisioning.synthesis DEBUG ------------------------------------------------- 05-11 14:08 elijah.provisioning.cloudletfs INFO Fuse close pipe 05-11 14:08 elijah.provisioning.cloudletfs INFO NO chunks has been waited at FUSE 05-11 14:08 elijah.provisioning.cloudletfs INFO close Fuse Exec thread 05-11 14:08 elijah.provisioning.cloudletfs INFO close File monitoring thread 05-11 14:08 elijah.provisioning.server INFO -------------------------------------------------- 05-11 14:08 elijah.provisioning.server INFO * Base VM Configuration 05-11 14:08 elijah.provisioning.server INFO 0 : /home/kdp/.cloudlet/abda52a61692094b3b7d45c9647d022f5e297d1b788679eb93735374007576b8/precise.raw (Disk 8192 MB, Memory 1040 MB) 05-11 14:08 elijah.provisioning.server INFO -------------------------------------------------- 05-11 14:08 elijah.provisioning.server INFO * Server configuration 05-11 14:08 elijah.provisioning.server INFO - Open TCP Server at ('0.0.0.0', 8021) 05-11 14:08 elijah.provisioning.server INFO - Disable Nagle(No TCP delay) : 1 05-11 14:08 elijah.provisioning.server INFO -------------------------------------------------- 05-11 14:08 elijah.provisioning.server INFO

----------------------- New Connection -------------- 05-11 14:08 elijah.provisioning.server INFO New client request /home/kdp/.cloudlet/abda52a61692094b3b7d45c9647d022f5e297d1b788679eb93735374007576b8/precise.raw VM

[krha]

I don't see any problem from the log file.

In fact, since command line execution works well, I'm suspicious on the mismatch between a client and a server program.

1) make sure that the client program is also update from the git.

2) reinstalling after the clean uninstallation.

% uninstall $ sudo pip uninstall elijah-provisioning $ rm -rf /usr/local/bin/cloudlet*

$ % reinstall $ fab localhost install

3) line by line debugging to find the problematic line at synthesis_server.py

Thanks.

[anilce]

(not working) During reinstalling elijah-provisioning it asks:

Download modified QEMU from https://github.com/cmusatyalab/elijah-qemu/ It enables on-demand VM fetching. Unlike the codes in elijah-provisioning, Modified QEMU is distributed under GPL license. Agree (Y/n) ?

3) line by line debugging to find the problematic line at synthesis_server.py how to debug line by line synthesis_server.py?

[anilce]

~$ pydb --fntrace synthesis_server

--Call level 0 exists(path='../elijah') (/usr/lib/python2.7/genericpath.py:15): exists

• 15 def exists(path): OSError:(2, 'No such file or directory', '../elijah') (/usr/lib/python2.7/genericpath.py:18): exists
• 18 os.stat(path) --Return from level 0 => False (<type 'bool'>) --Call level 0 (/usr/local/lib/python2.7/dist-packages/elijah/init.py:1):
• --Return from level 0 (<type 'NoneType'>) --Call level 0 (/usr/local/lib/python2.7/dist-packages/elijah/provisioning/init.py:21):
• 21 all = ['synthesis', 'server'] --Return from level 0 (<type 'NoneType'>) --Call level 0 (/usr/local/lib/python2.7/dist-packages/elijah/provisioning/server.py:21):
• 21 import os ----Call level 1 (/usr/lib/python2.7/SocketServer.py:120):
• 120 """ ------Call level 2 (/usr/lib/python2.7/socket.py:45):
• 45 """ --------Call level 3 _get_exports_list(module=<module '_socket' (built-in)>) (/usr/lib/python2.7/os.py:35): _get_exports_list
• 35 def _get_exports_list(module): AttributeError:"'module' object has no attribute 'all'" (/usr/lib/python2.7/os.py:37): _get_exports_list
• 37 return list(module.all) --------Return from level 3 (<type 'list'>) --------Call level 3 _closedsocket() (/usr/lib/python2.7/socket.py:167): _closedsocket
• 167 class _closedsocket(object): --------Return from level 3 (<type 'dict'>) --------Call level 3 _socketobject() (/usr/lib/python2.7/socket.py:179): _socketobject
• 179 class _socketobject(object): --------Return from level 3 (<type 'dict'>) --------Call level 3 _fileobject() (/usr/lib/python2.7/socket.py:235): _fileobject
• 235 class _fileobject(object): --------Return from level 3 (<type 'dict'>) ------Return from level 2 (<type 'NoneType'>) ------Call level 2 BaseServer() (/usr/lib/python2.7/SocketServer.py:160): BaseServer
• 160 class BaseServer: ------Return from level 2 (<type 'dict'>) ------Call level 2 TCPServer() (/usr/lib/python2.7/SocketServer.py:358): TCPServer
• 358 class TCPServer(BaseServer): ------Return from level 2 (<type 'dict'>) ------Call level 2 UDPServer() (/usr/lib/python2.7/SocketServer.py:480): UDPServer
• 480 class UDPServer(TCPServer): ------Return from level 2 (<type 'dict'>) ------Call level 2 ForkingMixIn() (/usr/lib/python2.7/SocketServer.py:506): ForkingMixIn
• 506 class ForkingMixIn: ------Return from level 2 (<type 'dict'>) ------Call level 2 ThreadingMixIn() (/usr/lib/python2.7/SocketServer.py:579): ThreadingMixIn
• 579 class ThreadingMixIn: ------Return from level 2 (<type 'dict'>) ------Call level 2 ForkingUDPServer() (/usr/lib/python2.7/SocketServer.py:607): ForkingUDPServer
• 607 class ForkingUDPServer(ForkingMixIn, UDPServer): pass ------Return from level 2 (<type 'dict'>) ------Call level 2 ForkingTCPServer() (/usr/lib/python2.7/SocketServer.py:608): ForkingTCPServer
• 608 class ForkingTCPServer(ForkingMixIn, TCPServer): pass ------Return from level 2 (<type 'dict'>) ------Call level 2 ThreadingUDPServer() (/usr/lib/python2.7/SocketServer.py:610): ThreadingUDPServer
• 610 class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass ------Return from level 2 (<type 'dict'>) ------Call level 2 ThreadingTCPServer() (/usr/lib/python2.7/SocketServer.py:611): ThreadingTCPServer
• 611 class ThreadingTCPServer(ThreadingMixIn, TCPServer): pass ------Return from level 2 (<type 'dict'>) ------Call level 2 UnixStreamServer() (/usr/lib/python2.7/SocketServer.py:615): UnixStreamServer
• 615 class UnixStreamServer(TCPServer): ------Return from level 2 (<type 'dict'>) ------Call level 2 UnixDatagramServer() (/usr/lib/python2.7/SocketServer.py:618): UnixDatagramServer
• 618 class UnixDatagramServer(UDPServer): ------Return from level 2 (<type 'dict'>) ------Call level 2 ThreadingUnixStreamServer() (/usr/lib/python2.7/SocketServer.py:621): ThreadingUnixStreamServer
• 621 class ThreadingUnixStreamServer(ThreadingMixIn, UnixStreamServer): pass ------Return from level 2 (<type 'dict'>) ------Call level 2 ThreadingUnixDatagramServer() (/usr/lib/python2.7/SocketServer.py:623): ThreadingUnixDatagramServer
• 623 class ThreadingUnixDatagramServer(ThreadingMixIn, UnixDatagramServer): pass ------Return from level 2 (<type 'dict'>) ------Call level 2 BaseRequestHandler() (/usr/lib/python2.7/SocketServer.py:625): BaseRequestHandler
• 625 class BaseRequestHandler: ------Return from level 2 (<type 'dict'>) ------Call level 2 StreamRequestHandler() (/usr/lib/python2.7/SocketServer.py:671): StreamRequestHandler
• 671 class StreamRequestHandler(BaseRequestHandler): ------Return from level 2 (<type 'dict'>) ------Call level 2 DatagramRequestHandler() (/usr/lib/python2.7/SocketServer.py:714): DatagramRequestHandler
• 714 class DatagramRequestHandler(BaseRequestHandler): ------Return from level 2 (<type 'dict'>) ----Return from level 1 (<type 'NoneType'>) ----Call level 1 (/usr/lib/python2.7/tempfile.py:18):
• 18 """ ------Call level 2 (/usr/lib/python2.7/io.py:34):
• 34 """ --------Call level 3 IOBase() (/usr/lib/python2.7/io.py:69): IOBase
• 69 class IOBase(_io._IOBase): --------Return from level 3 (<type 'dict'>) --------Call level 3 new(mcls=<class 'abc.ABCMeta'>, name='IOBase', bases=(<type '_io._IOBase'>,), namespace={'return': ...) (/usr/lib/python2.7/abc.py:86): new
• 86 def new(mcls, name, bases, namespace): ----------Call level 4Traceback (most recent call last): File "/usr/bin/pydb", line 765, in main p._runscript(mainpyfile) File "/usr/share/pyshared/pydb/pydbcmd.py", line 94, in _runscript self.run(statement) File "/usr/share/pyshared/pydb/pydbbdb.py", line 374, in run exec cmd in globals, locals File "", line 1, in File "/usr/local/bin/synthesis_server", line 26, in from elijah.provisioning.server import SynthesisServer File "/usr/local/lib/python2.7/dist-packages/elijah/provisioning/server.py", line 28, in import tempfile File "/usr/lib/python2.7/tempfile.py", line 32, in import io as _io File "/usr/lib/python2.7/io.py", line 69, in class IOBase(_io._IOBase): File "/usr/lib/python2.7/abc.py", line 90, in new for name, value in namespace.items() File "/usr/lib/python2.7/abc.py", line 89, in abstracts = set(name File "/usr/share/pyshared/pydb/gdb.py", line 241, in trace_dispatch_gdb return self.trace_dispatch_old(frame, event, arg) File "/usr/lib/python2.7/bdb.py", line 51, in trace_dispatch return self.dispatch_call(frame, arg) File "/usr/lib/python2.7/bdb.py", line 80, in dispatch_call self.user_call(frame, arg) File "/usr/share/pyshared/pydb/pydbbdb.py", line 411, in user_call self.print_call_params(frame) File "/usr/share/pyshared/pydb/pydbbdb.py", line 58, in print_call_params include_location=False)) File "/usr/share/pyshared/pydb/pydbbdb.py", line 316, in format_stack_entry parms=inspect.formatargvalues(args, varargs, varkw, local_vars) File "/usr/lib/python2.7/inspect.py", line 888, in formatargvalues specs.append(strseq(args[i], convert, join)) File "/usr/lib/python2.7/inspect.py", line 841, in strseq return join(map(lambda o, c=convert, j=join: strseq(o, c, j), object)) File "/usr/lib/python2.7/inspect.py", line 841, in return join(map(lambda o, c=convert, j=join: strseq(o, c, j), object)) File "/usr/lib/python2.7/inspect.py", line 843, in strseq return convert(object) File "/usr/lib/python2.7/inspect.py", line 885, in convert return formatarg(name) + formatvalue(locals[name]) KeyError: 'name' *\ Uncaught exception.

[krha]

@anilce

What I mean by line by line test is making a break point at the code where it communicate with a client, then find the blocking point moving line by line. Since every cloudlet functionality is working using command line tool, I assume that it's networking communication problem between a client part and a server part.

Please start from [here]https://github.com/cmusatyalab/elijah-provisioning/blob/master/elijah/provisioning/server.py#L860

You can make a break point using import pdb; pdb.set_trace() (see more at https://docs.python.org/2/library/pdb.html)