Console with modified backend

[hassanjameel1]

Hi,

I am trying to try the ponger example using your modified netback. I cannot seem to make it work. The console output for the ClickOS VM says:

Xen Minimal OS! start_info: 0x1be000(VA) nr_pages: 0x800 shared_inf: 0x8ce6b000(MA) pt_base: 0x1c1000(VA) nr_pt_frames: 0x5 mfn_list: 0x1ba000(VA) mod_start: 0x0(VA) mod_len: 0 flags: 0x0 cmd_line: stack: 0x177040-0x197040 MM: Init _text: 0x0(VA) _etext: 0xe1e1d(VA) _erodata: 0x15d000(VA) _edata: 0x15ebb8(VA) stack start: 0x177040(VA) _end: 0x1b9010(VA) start_pfn: 1c9 max_pfn: 800 Mapping memory range 0x400000 - 0x800000 setting 0x0-0x15d000 readonly skipped 0x1000 MM: Initialise page allocator for 1cb000(1cb000)-800000(800000) MM: done Demand map pfns at 801000-2000801000. Heap resides at 2000802000-4000802000. Initialising timer interface Initialising console ... done. gnttab_table mapped at 0x801000. Initialising scheduler Thread "Idle": pointer: 0x2000802050, stack: 0x1e0000 Thread "xenstore": pointer: 0x2000802800, stack: 0x1f0000 xenbus initialised on irq 1 mfn 0x2222b0 Thread "shutdown": pointer: 0x2000802fb0, stack: 0x200000 Dummy main: start_info=0x197040 Thread "main": pointer: 0x2000803760, stack: 0x210000 sparsing 0MB at 176000 "main" [on_status:205] router id 0 [on_status:206] status change to Running Thread "click": pointer: 0x2000810c50, stack: 0x230000 backend dom 0 Mapping TX rings 0 map errors

Am I doing it right? The example.config file that I am using is as follows:

name = 'clickos' kernel = '/home/nfv1/click/clickos/minios/build/clickos_x86_64' vcpus = '1'

pinning your VCPU helps performance

cpus = '3' memory = '8'

uncoment this line if you wish to use our backend

vif = ['bridge=vale0,script=vif-vale']

vif = ['mac=68:05:ca:29:2e:be,bridge=vale0']

on_poweroff = 'destroy' on_reboot = 'restart' on_crash = 'preserve' click = 'ponger.click'

uncomment this line to enable PVH mode for ClickOS

pvh=1

Best wishes, Hassan

[jpemartins]

Hi,

Does it get stuck there on:

"Mapping TX rings 0 map errors"

It doesn't show anything else?

Could you tell me your hardware specifications? and the output of these commands:

xl vcpu-list xl info xl dmesg dmesg

I indeed have heard of cases where it gets stuck on mapping phase being unable to map the RX rings. But I was unable to reproduce it in my hardware.

Thanks, Cheers, Joao

[hassanjameel1]

Hi Joao,

Thanks for your prompt reply. It does get stuck at:

"Mapping TX rings 0 map errors"

and shows nothing else. The hardware specs are as follows:

CPU: Intel(R) Xeon(R) CPU E5-1607 v2 @ 3.00 GHz (4-core) (3 assigned to Dom-0) RAM: 16 GB (with 8 GB assigned to Dom-0) Network Card: Intel Corporation 82599 10 Gigabit TN Network Connection (dual-port)

Let me know if you require any other hardware specification.

The output to "xl vcpu-list" is:

Name ID VCPU CPU State Time(s) CPU Affinity Domain-0 0 0 0 -b- 511.7 0 Domain-0 0 1 1 r-- 397.9 1 Domain-0 0 2 2 -b- 410.2 2 clickos 7 0 3 r-- 367.0 3

The output to "xl info" is:

Name ID VCPU CPU State Time(s) CPU Affinity Domain-0 0 0 0 -b- 511.7 0 Domain-0 0 1 1 r-- 397.9 1 Domain-0 0 2 2 -b- 410.2 2 clickos 7 0 3 r-- 367.0 3 nfv1@nfv1-Precision-T3610:~/Documents/nfv-project$ sudo xl info host : nfv1-Precision-T3610 release : 3.13.0-37-generic version : #64-Ubuntu SMP Mon Sep 22 21:28:38 UTC 2014 machine : x86_64 nr_cpus : 4 max_cpu_id : 3 nr_nodes : 1 cores_per_socket : 4 threads_per_core : 1 cpu_mhz : 2992 hw_caps : bfebfbff:2c100800:00000000:00006f00:77bee3ff:00000000:00000001:00000281 virt_caps : hvm hvm_directio total_memory : 16328 free_memory : 7908 sharing_freed_memory : 0 sharing_used_memory : 0 outstanding_claims : 0 free_cpus : 0 xen_major : 4 xen_minor : 4 xen_extra : .0 xen_version : 4.4.0 xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 xen_scheduler : credit xen_pagesize : 4096 platform_params : virt_start=0xffff800000000000 xen_changeset : xen_commandline : placeholder dom0_mem=8192M,max:8192M dom0_max_vcpus=3 dom0_vcpus_pin cc_compiler : gcc (Ubuntu 4.8.2-19ubuntu1) 4.8.2 cc_compile_by : stefan.bader cc_compile_domain : canonical.com cc_compile_date : Tue Sep 30 19:19:31 UTC 2014 xend_config_format : 4

The output of "xl dmesg" is:

(XEN) Xen version 4.4.0 (Ubuntu 4.4.0-0ubuntu5.2) (stefan.bader@canonical.com) (gcc (Ubuntu 4.8.2-19ubuntu1) 4.8.2) debug=n Tue Sep 30 19:19:31 UTC 2014 (XEN) Bootloader: GRUB 2.02~beta2-9ubuntu1 (XEN) Command line: placeholder dom0_mem=8192M,max:8192M dom0_max_vcpus=3 dom0_vcpus_pin (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds (XEN) EDID info not retrieved because no DDC retrieval method detected (XEN) Disc information: (XEN) Found 1 MBR signatures (XEN) Found 1 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 0000000000095400 (usable) (XEN) 0000000000095400 - 00000000000a0000 (reserved) (XEN) 00000000000e0000 - 0000000000100000 (reserved) (XEN) 0000000000100000 - 0000000087ed6000 (usable) (XEN) 0000000087ed6000 - 0000000088000000 (reserved) (XEN) 0000000088000000 - 0000000088775000 (usable) (XEN) 0000000088775000 - 0000000088800000 (reserved) (XEN) 0000000088800000 - 0000000088ee9000 (usable) (XEN) 0000000088ee9000 - 0000000089000000 (ACPI data) (XEN) 0000000089000000 - 000000008a5a8000 (usable) (XEN) 000000008a5a8000 - 000000008a800000 (ACPI NVS) (XEN) 000000008a800000 - 000000008c594000 (usable) (XEN) 000000008c594000 - 000000008c743000 (reserved) (XEN) 000000008c743000 - 000000008c766000 (usable) (XEN) 000000008c766000 - 000000008c7ec000 (ACPI NVS) (XEN) 000000008c7ec000 - 000000008d000000 (usable) (XEN) 0000000090000000 - 00000000a0000000 (reserved) (XEN) 00000000fed1c000 - 00000000fed40000 (reserved) (XEN) 00000000ff000000 - 0000000100000000 (reserved) (XEN) 0000000100000000 - 0000000470000000 (usable) (XEN) ACPI: RSDP 000F0490, 0024 (r2 DELL ) (XEN) ACPI: XSDT 88F29088, 008C (r1 DELL CBX3 1072009 AMI 10013) (XEN) ACPI: FACP 88F32130, 010C (r5 DELL CBX3 1072009 AMI 10013) (XEN) ACPI: DSDT 88F291A0, 8F8C (r2 DELL CBX3 23 INTL 20091112) (XEN) ACPI: FACS 8A7F7080, 0040 (XEN) ACPI: APIC 88F32240, 0090 (r3 DELL CBX3 1072009 AMI 10013) (XEN) ACPI: FPDT 88F322D0, 0044 (r1 DELL CBX3 1072009 AMI 10013) (XEN) ACPI: MCFG 88F32318, 003C (r1 DELL OEMMCFG. 1072009 MSFT 97) (XEN) ACPI: HPET 88F32358, 0038 (r1 DELL CBX3 1072009 AMI. 5) (XEN) ACPI: ASF! 88F32390, 00A0 (r32 INTEL HCG 1 TFSM F4240) (XEN) ACPI: SSDT 88F32430, CD380 (r2 INTEL CpuPm 4000 INTL 20051117) (XEN) ACPI: EINJ 88FFF7B0, 0130 (r1 AMI AMI EINJ 0 0) (XEN) ACPI: ERST 88FFF8E0, 0230 (r1 AMIER AMI ERST 0 0) (XEN) ACPI: HEST 88FFFB10, 00A8 (r1 AMI AMI HEST 0 0) (XEN) ACPI: BERT 88FFFBB8, 0030 (r1 AMI AMI BERT 0 0) (XEN) ACPI: DMAR 88FFFBE8, 00B4 (r1 A M I OEMDMAR 1 INTL 1) (XEN) ACPI: SLIC 88FFFCA0, 0176 (r3 DELL CBX3 1072009 MSFT 10013) (XEN) System RAM: 16328MB (16720112kB) (XEN) Domain heap initialised (XEN) ACPI: 32/64X FACS address mismatch in FADT - 8a7f7080/0000000000000000, using 32 (XEN) Processor #0 7:14 APIC version 21 (XEN) Processor #2 7:14 APIC version 21 (XEN) Processor #4 7:14 APIC version 21 (XEN) Processor #6 7:14 APIC version 21 (XEN) IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23 (XEN) IOAPIC[1]: apic_id 2, version 32, address 0xfec01000, GSI 24-47 (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs (XEN) Switched to APIC driver x2apic_cluster. (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Detected 2992.900 MHz processor. (XEN) Initing memory sharing. (XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7 (XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB. (XEN) Intel VT-d Snoop Control enabled. (XEN) Intel VT-d Dom0 DMA Passthrough not enabled. (XEN) Intel VT-d Queued Invalidation enabled. (XEN) Intel VT-d Interrupt Remapping enabled. (XEN) Intel VT-d Shared EPT tables enabled. (XEN) I/O virtualisation enabled (XEN) - Dom0 mode: Relaxed (XEN) Interrupt remapping enabled (XEN) Enabled directed EOI with ioapic_ack_old on! (XEN) ENABLING IO-APIC IRQs (XEN) -> Using old ACK method (XEN) Platform timer is 14.318MHz HPET (XEN) Allocated console ring of 16 KiB. (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Extended Page Tables (EPT) (XEN) - Virtual-Processor Identifiers (VPID) (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) - Unrestricted Guest (XEN) - APIC Register Virtualization (XEN) - Virtual Interrupt Delivery (XEN) - Posted Interrupt Processing (XEN) HVM: ASIDs enabled. (XEN) HVM: VMX enabled (XEN) HVM: Hardware Assisted Paging (HAP) detected (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB (XEN) Brought up 4 CPUs (XEN) verify_tsc_reliability: TSC warp detected, disabling TSC_RELIABLE (XEN) * LOADING DOMAIN 0 * (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2403000 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 000000045c000000->0000000460000000 (2076086 pages to be allocated) (XEN) Init. ramdisk: 000000046edb6000->000000046ffff6ce (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff81000000->ffffffff82403000 (XEN) Init. ramdisk: ffffffff82403000->ffffffff8364c6ce (XEN) Phys-Mach map: ffffffff8364d000->ffffffff8464d000 (XEN) Start info: ffffffff8464d000->ffffffff8464d4b4 (XEN) Page tables: ffffffff8464e000->ffffffff84675000 (XEN) Boot stack: ffffffff84675000->ffffffff84676000 (XEN) TOTAL: ffffffff80000000->ffffffff84800000 (XEN) ENTRY ADDRESS: ffffffff81d361f0 (XEN) Dom0 has maximum 3 VCPUs (XEN) Scrubbing Free RAM: ................................................................................done. (XEN) Initial low memory virq threshold set at 0x4000 pages. (XEN) Std. Loglevel: Errors and warnings (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings) (XEN) Xen is relinquishing VGA console. (XEN) *\ Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen) (XEN) Freed 280kB init memory.

The output of "dmesg" is:

[65497.038538] xen_netback/xen_netback:xenvif_netmap_unmap: vif6.0: revoking 1034 pages [65497.038542] xen:grant_table: WARNING: g.e. 0x207 still in use! [65497.038544] deferring g.e. 0x207 (pfn 0xffffffffffffffff) [65497.038545] xen:grant_table: WARNING: g.e. 0x40e still in use! [65497.038546] deferring g.e. 0x40e (pfn 0xffffffffffffffff) [65497.038547] xen:grant_table: WARNING: g.e. 0x40f still in use! [65497.038548] deferring g.e. 0x40f (pfn 0xffffffffffffffff) [65497.038548] xen:grant_table: WARNING: g.e. 0x410 still in use! [65497.038549] deferring g.e. 0x410 (pfn 0xffffffffffffffff) [65497.038550] xen:grant_table: WARNING: g.e. 0x411 still in use! [65497.038551] deferring g.e. 0x411 (pfn 0xffffffffffffffff) [65497.109658] 085.609299 [ 719] nm_bdg_detach deleting priv [65497.120414] xen:balloon: Cannot add additional memory (-17) [65498.035669] xen:grant_table: freeing g.e. 0x207 [65498.035675] xen:grant_table: freeing g.e. 0x40e [65498.035677] xen:grant_table: freeing g.e. 0x40f [65498.035680] xen:grant_table: freeing g.e. 0x410 [65498.035682] xen:grant_table: freeing g.e. 0x411 [65497.038633] xen_netback/xen_netback:xenvif_netmap_detach: netmap_detach vif6.0 [65522.831735] 111.331376 [2259] netmap_attach success for vif7.0 [65522.844806] IPv6: ADDRCONF(NETDEV_UP): vif7.0: link is not ready [65522.845175] IPv6: ADDRCONF(NETDEV_UP): vif7.0: link is not ready [65522.898519] 111.398160 [1456] netmap_set_ringid deprecated API, old ringid 0x0 -> ringid 0 reg 1 [65522.901717] xen_netback/xen_netback:xenvif_netmap_map: vif7.0: granted 10 1024 (ring/buffers)

Best wishes, Hassan

[jpemartins]

Hey Hassan,

Everything looks ok on the backend side and configurations.

Could you try booting the VM in one of the cores of Domain-0 ? For example in core 2 instead of core 3, then pinning it back to core 3.

Thanks, Cheers

[hassanjameel1]

HI Joao,

Great, it worked. I specified the cpu in the .cfg file and booted the VM. The console this time didn't get stuck. After that I pinned it back to core 3 by:

xl vcpu-pin clickos 0 3

Thanks a lot! Any reason why this worked?

[jpemartins]

Not sure yet but will let you know when I fix the issue, in mini-os. Thanks a lot!