ostroproject / ostro-os

Ostro OS
Other
93 stars 43 forks source link

Using Ostro OS + Serial CDC #214

Closed sysarcher closed 7 years ago

sysarcher commented 7 years ago

Hello Everybody,

In my quest to using Intel's Wireless AC-3160 wifi PCIe card with the Galileo, I compiled the swupd version of Ostro OS and everything seemed to work well except for the Serial port.

My USB CDC device does not appear as an ACMx file interface (/dev/ttyACM0 for example). Has anyone had similar problems with kernel 4.4?

I've put some commands I ran, that might help the context at the end of this post. The device in question is the STMicroelectronics STM32F407. I can confirm that reverting to the standard image for the Galileo fixes this issue.

Thank you...

# lsusb  
    Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub  
    Bus 002 Device 003: ID 8087:07dc Intel Corp.   
    Bus 002 Device 002: ID 0483:5740 STMicroelectronics STM32F407  
    Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub  

# lsusb -d 0483:5740 -v    # for STM32F407  

Bus 002 Device 002: ID 0483:5740 STMicroelectronics STM32F407  
Device Descriptor:  
  bLength                18  
  bDescriptorType         1  
  bcdUSB               2.00  
  bDeviceClass            0   
  bDeviceSubClass         0   
  bDeviceProtocol         0   
  bMaxPacketSize0        64  
  idVendor           0x0483 STMicroelectronics  
  idProduct          0x5740 STM32F407  
  bcdDevice            2.00  
  iManufacturer           1 STMicroelectronics  
  iProduct                2 STM32 Virtual ComPort  
  iSerial                 3 00000000001A  
  bNumConfigurations      1  
  Configuration Descriptor:  
    bLength                 9  
    bDescriptorType         2  
    wTotalLength           67  
    bNumInterfaces          2  
    bConfigurationValue     1  
    iConfiguration          0   
    bmAttributes         0x80  
      (Bus Powered)  
    MaxPower              250mA  
    Interface Descriptor:  
      bLength                 9  
      bDescriptorType         4  
      bInterfaceNumber        0  
      bAlternateSetting       0  
      bNumEndpoints           1  
      bInterfaceClass         2 Communications  
      bInterfaceSubClass      2 Abstract (modem)  
      bInterfaceProtocol      1 AT-commands (v.25ter)  
      iInterface              0   
      CDC Header:  
        bcdCDC               1.10  
      CDC Call Management:  
        bmCapabilities       0x00  
        bDataInterface          1  
      CDC ACM:  
        bmCapabilities       0x02  
          line coding and serial state  
      CDC Union:  
        bMasterInterface        0  
        bSlaveInterface         1   
      Endpoint Descriptor:  
        bLength                 7  
        bDescriptorType         5  
        bEndpointAddress     0x82  EP 2 IN  
        bmAttributes            3  
          Transfer Type            Interrupt  
          Synch Type               None  
          Usage Type               Data  
        wMaxPacketSize     0x0008  1x 8 bytes  
        bInterval              16  
    Interface Descriptor:  
      bLength                 9  
      bDescriptorType         4  
      bInterfaceNumber        1  
      bAlternateSetting       0  
      bNumEndpoints           2  
      bInterfaceClass        10 CDC Data  
      bInterfaceSubClass      0   
      bInterfaceProtocol      0   
      iInterface              0   
      Endpoint Descriptor:  
        bLength                 7  
        bDescriptorType         5  
        bEndpointAddress     0x01  EP 1 OUT  
        bmAttributes            2  
          Transfer Type            Bulk  
          Synch Type               None  
          Usage Type               Data  
        wMaxPacketSize     0x0040  1x 64 bytes  
        bInterval               0  
      Endpoint Descriptor:  
        bLength                 7  
        bDescriptorType         5  
        bEndpointAddress     0x81  EP 1 IN  
        bmAttributes            2  
          Transfer Type            Bulk  
          Synch Type               None  
          Usage Type               Data  
        wMaxPacketSize     0x0040  1x 64 bytes  
        bInterval               0  
can't get device qualifier: Resource temporarily unavailable  
can't get debug descriptor: Resource temporarily unavailable  
Device Status:     0x0000  
  (Bus Powered)

# cat /dev/tty  
    tty     tty11   tty15   tty19   tty22   tty26   tty3    tty33   tty37   tty40   tty44   tty48   tty51   tty55   tty59   tty62   tty9    ttyS2     
    tty0    tty12   tty16   tty2    tty23   tty27   tty30   tty34   tty38   tty41   tty45   tty49   tty52   tty56   tty6    tty63   ttyGS0  ttyS3     
    tty1    tty13   tty17   tty20   tty24   tty28   tty31   tty35   tty39   tty42   tty46   tty5    tty53   tty57   tty60   tty7    ttyS0     
    tty10   tty14   tty18   tty21   tty25   tty29   tty32   tty36   tty4    tty43   tty47   tty50   tty54   tty58   tty61   tty8    ttyS1   
gvancuts commented 7 years ago

Hi @shrmrf , can you clarify what the 'standard image' for Galileo is (which works as expected)?

My first guess is that we may not have all the appropriate drivers enabled in our kernel. Do you know which kernel driver is used in the image that works correctly?

sysarcher commented 7 years ago

Hello @gvancuts ... thanks for your reply. The standard image I was referring to is the one distributed on Intel's getting started site: https://software.intel.com/en-us/iot/hardware/galileo/downloads

Its kernel version 3.8...

# uname -a
Linux galileo 3.8.7-yocto-standard #1 Tue Nov 29 14:38:48 UTC 2016 i586 GNU/Linux

I'm going to look into the kernel drivers enabled. Do you know how I can check the driver that's used?

sysarcher commented 7 years ago

EDIT: Sorry for the incorrect copy-paste before... the output below should be for the correct driver.. EDIT-2: I will confirm whether the driver exists in the ostro image.

Btw, does this help? I think I am referring to the right device on the correct bus.

root@galileo:/sys/bus/usb/devices/2-1# readlink driver
../../../../../bus/usb/drivers/usb
root@galileo:/sys/bus/usb/devices/2-1# /sbin/modinfo `cat 2-1\:1.0/modalias`
filename:       /lib/modules/3.8.7-yocto-standard/kernel/drivers/usb/class/cdc-acm.ko
alias:          char-major-166-*
license:        GPL
description:    USB Abstract Control Model driver for USB modems and ISDN adapters
author:         Armin Fuerst, Pavel Machek, Johannes Erdfelt, Vojtech Pavlik, David Kubicek, Johan Hovold
alias:          usb:v*p*d*dc*dsc*dp*ic02isc02ip06in*
alias:          usb:v*p*d*dc*dsc*dp*ic02isc02ip05in*
alias:          usb:v*p*d*dc*dsc*dp*ic02isc02ip04in*
alias:          usb:v*p*d*dc*dsc*dp*ic02isc02ip03in*
alias:          usb:v*p*d*dc*dsc*dp*ic02isc02ip02in*
alias:          usb:v*p*d*dc*dsc*dp*ic02isc02ip01in*
alias:          usb:v*p*d*dc*dsc*dp*ic02isc02ip00in*
alias:          usb:v04D8p000Bd*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v0694pFF00d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v03EBp0030d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v04E7p6651d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p03CDd*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0335d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0302d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p01D4d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p04CEd*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0154d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p026Cd*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0275d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0223d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p01D0d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p02D9d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p010Ed*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0178d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p02E3d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p01F5d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0108d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p00E9d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p003Ad*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0094d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p007Bd*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p00A0d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p008Fd*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0128d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0099d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p00E9d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0070d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p04F0d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0071d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0007d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0481d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p00ABd*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p00B0d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0042d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p00FCd*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0088d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p002Fd*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p046Ed*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0134d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p04B2d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p04E6d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0420d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p048Ed*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p042Fd*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0445d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p000Ed*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p04DFd*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0486d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0425d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0418d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0508d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0475d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0001d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p044Dd*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p0419d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p04C9d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p04D8d*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v0421p042Dd*dc*dsc*dp*ic02isc02ipFFin*
alias:          usb:v1576p03B1d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v1BBBp0003d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v05F9p4002d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v0572p1340d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v0572p1329d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v22B8p2D9Ad*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v22B8p2D99d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v22B8p2D97d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v22B8p2D96d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v22B8p2D95d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v22B8p2D93d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v22B8p2D92d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v22B8p2D91d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v22B8p6425d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v0572p1328d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v0572p1324d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v0572p1321d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v0803p3095d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v22B8p7000d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v0ACEp1611d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v0ACEp1608d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v0ACEp1602d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v079Bp000Fd*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v0482p0203d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v0E8Dp3329d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v0E8Dp0003d*dc*dsc*dp*ic*isc*ip*in*
alias:          usb:v0870p0001d*dc*dsc*dp*ic*isc*ip*in*
depends:        usbcore
intree:         Y
vermagic:       3.8.7-yocto-standard mod_unload 586TSC
sysarcher commented 7 years ago

@gvancuts ... YUP, as you expected... the driver isn't available on the ostro image:-

what would be the quickest way to resolve this? Any suggestion would be extremely helpful!!

root@galileo:/sys/bus/usb/devices/2-1# /sbin/modinfo `cat 2-1\:1.0/modalias`
modinfo: ERROR: Module usb:v0483p5740d0200dc00dsc00dp00ic02isc02ip01in00 not found.
gvancuts commented 7 years ago

It sounds like you're already able to compile an Ostro OS image. In that case, from your dev machine, use "bitbake linux-yocto -c menuconfig" will allow you to change the kernel config. Once done, proceed with recreating a new image. Let me know if that helps or if you need more accurate steps.

sysarcher commented 7 years ago

Hello @gvancuts thanks for the reply. I would love it if you could specify where the cdc_acm drivers are in menuconfig. can't seem to figure out... (maybe I'm burned out?)

gvancuts commented 7 years ago

It is in 'Device Drivers' -> 'USB support' ('Support for Host-side USB' needs to be selected).

(hint: I never remember where these are myself but you can easily search for a keyword from that menuconfig, hitting '/' will open the search box)

sysarcher commented 7 years ago

Hey @gvancuts ... thanks a lot for the help. I think I can figure out the rest of the story 👍

There was also usb_acm in my search results. Wonder if we need to set it to be included. I'll experiment now and let you know.

gvancuts commented 7 years ago

Sounds good, good luck and keep us posted!

sysarcher commented 7 years ago

Hello @gvancuts ... unfortunately, I'm still stuck. I've tried to generate a .config file with all the necessary drivers selected. I'm not sure how we can include the necessary changes to our configuration.

What I tried to do also was modify meta-ostro-bsp/recipes-kernel/linux-yocto/linux-ostro/quark/defconfig using menuconfig. Interestingly, the following options were already selected:

  2721  CONFIG_USB_ACM=y
  2879  CONFIG_USB_F_ACM=m
  2906  CONFIG_USB_G_ACM_MS=m

Am I going about it the right way? Or, is there something else I need to look at?

Thanks...

sysarcher commented 7 years ago

When I unplug/plug the peripheral, I see these messages...

usb 2-1: USB disconnect, device number 4
usb 2-1: new full-speed USB device number 5 using ohci-pci

Are we maybe looking at the wrong drivers to enable?

here's the output of lsmod (note that ohci_pci seems fine.. (edit))...

root@intel-quark:/sys/bus/usb/devices/2-1# lsmod
Module                  Size  Used by
8021q                  16124  0
xt_connmark             1525  2
bnep                    8995  2
iptable_nat             1463  0
nf_nat_ipv4             4423  1 iptable_nat
nf_nat                 10795  1 nf_nat_ipv4
iptable_mangle          1256  1
arc4                    1728  2
btusb                  27284  0
btrtl                   3808  1 btusb
btbcm                   5783  1 btusb
btintel                 6136  1 btusb
iwlmvm                219949  0
bluetooth             296766  26 bnep,btbcm,btrtl,btusb,btintel
mac80211              479742  1 iwlmvm
iwlwifi               146276  1 iwlmvm
cfg80211              424943  3 iwlwifi,mac80211,iwlmvm
ohci_pci                2496  0                                                # <<<<----------------- HERE
ohci_hcd               28149  1 ohci_pci
adc1x8s102              3646  0
industrialio_triggered_buffer      969  1 adc1x8s102
kfifo_buf               2311  1 industrialio_triggered_buffer
industrialio           33780  3 industrialio_triggered_buffer,adc1x8s102,kfifo_buf
pca9685                 5470  0
at24                    4626  0
regmap_i2c              2792  1 pca9685
nf_conntrack_ipv6       6803  1
nf_conntrack_ipv4      10506  4
nf_defrag_ipv6          9822  1 nf_conntrack_ipv6
nf_defrag_ipv4          1115  1 nf_conntrack_ipv4
xt_tcpudp               1975  6
xt_conntrack            2905  2
nf_conntrack           55816  6 nf_nat,nf_nat_ipv4,xt_conntrack,xt_connmark,nf_conntrack_ipv4,nf_conntrack_ipv6
usb_f_acm               4176  1
u_serial                9217  1 usb_f_acm
ip6table_filter         1132  1
ip6_tables              9874  1 ip6table_filter
g_acm_ms                2682  0
usb_f_mass_storage     26101  2 g_acm_ms
iptable_filter          1160  1
libcomposite           33261  3 usb_f_acm,g_acm_ms,usb_f_mass_storage
configfs               21716  4 usb_f_acm,libcomposite,usb_f_mass_storage
efivarfs                5199  1
ip_tables               9426  3 iptable_filter,iptable_mangle,iptable_nat
x_tables               11209  8 ip6table_filter,ip_tables,xt_tcpudp,xt_conntrack,iptable_filter,xt_connmark,iptable_mangle,ip6_tables
sysarcher commented 7 years ago

btw, I'm doing bitbake -k ostro-image-swupd (not noswupd) ... shouldn't be a problem, right?

mythi commented 7 years ago

On Thu, Dec 15, 2016 at 10:30 AM, T wrote:

btw, I'm doing bitbake -k ostro-image-swupd (not noswupd) ... shouldn't be a problem, right?

Not a problem but you take an unnecessary hit in time when using swupd in debugging.

sysarcher commented 7 years ago

@mythi ... yeah... I'm trying a clean build now. Do you think I should delete the tmp folder and do a clean build? I noticed that image size didn't change because of my changes to defconfig

gvancuts commented 7 years ago

Hi @shrmrf , modifying a kernel config in Yocto/Ostro can be a little tricky. We wrote up some steps to help with this, have a look here: https://ostroproject.org/documentation/howtos/modifying-ostro-kernel.html

The quick summary is there are a couple of steps needed after you've run "bitbake linux-yocto -c menuconfig" and modified the kernel config, namely:

$ bitbake -f linux-yocto -c compile
$ bitbake -f linux-yocto -c compile_kernelmodules

And then you should re-create your image:

$ bitbake ostro-image-noswupd           # or use the ostro-image-swupd target

Having said that, I would also recommend trying this in your image already:

# modprobe cdc-acm

This may help in case the drive is available as a module but wasn't loaded automatically

sysarcher commented 7 years ago

I've already started the build...

# modprobe cdc-acm
modprobe: module cdc-acm not found in modules.dep
sysarcher commented 7 years ago

@gvancuts When I do a bitbake linux-yocto -c menuconfig, it asks me whether I want to save it in a .config file. I leave the default as is. Is it possible that I am not saving it to the correct place? Should I save it to defconfig somewhere for example? My assumption is that the .config file will automatically be recognized by bitbake and will be used. I do not have documentation to assert my claim though - its just an assumption.

sysarcher commented 7 years ago

@gvancuts and @mythi I've the same problem... https://github.com/ostroproject/ostro-os/issues/214#issuecomment-267293379

# modprobe cdc-acm
modprobe: module cdc-acm not found in modules.dep
gvancuts commented 7 years ago

@shrmrf , saving it to .config (as proposed by menuconfig) is the right thing to do. Did you use the -f flag in the next two calls to bitbake?

It's also very important that you issue these bitbake commands right after re-configuring the kernel. Do not do a (simpler) bitbake linux-yocto as this will reset the configuration of the kernel.

sysarcher commented 7 years ago

@gvancuts ... seems like I missed the -f at least in my last attempt. Trying once more...

So I did bitbake ostro-image-noswupd at the end.... does it also need the -f flag?

gvancuts commented 7 years ago

That could explain what you're seeing... if things don't work out as expected, it may be interesting to look inside the log for the last few commands (they're all placed in tmp-glibc/log/cooker/intel-quark/)

sysarcher commented 7 years ago

WOAH!!! Thanks @gvancuts and @mythi ... I think we got there!!!

root@intel-quark:~# usb 2-1: new full-speed USB device number 3 using ohci-pci
cdc_acm 2-1:1.0: ttyACM0: USB ACM device
usbcore: registered new interface driver cdc_acm
cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
gvancuts commented 7 years ago

Nice one, have fun!

sysarcher commented 7 years ago

So this was the build I did by:

Notes

The above is the only way I got it working. Even if I repeat the above steps again, it doesn't matter. bitbake -f ostro-image-noswupd doesn't help either (-f doesn't help).. All -c cleanall magic doesn't help as well. I suspect however that just deleting the tmp-glibc, cache, sstate-cache directories might have been enough.

ALL CREDIT GOES TO: @gvancuts Honorable Mentions: @mythi

Thank you guys!!!

gvancuts commented 7 years ago

So this was the build I did by: • Deleting the build directory • Sourcing the oe-init-build-env again • changing conf/local.conf to enable development images

And I guess you also uncommented the following line in conf/local.conf, didn't you?

# MACHINE ?= "intel-quark"

• doing

Missing a $ bitbake linux-yocto -c menuconfig I guess

o bitbake -f linux-yocto -c compile o bitbake -f linux-yocto -c compile_kernelmodules o bitbake ostro-image-noswupd # note, no -f flag Notes The above is the only way I got it working. Even if I repeat the above steps again, it doesn't matter. bitbake -f ostro-image-noswupd doesn't help either (-f doesn't help).. All -c cleanall magic doesn't help as well. I suspect however that just deleting the tmp-> glibc, cache, sstate-cache directories might have been enough. ALL CREDIT GOES TO: @gvancuts Honorable Mentions: @mythi Thank you guys!!!

You're welcome ;)

sysarcher commented 7 years ago

@gvancuts yes, yes :) I over summarized the cleaning pains and forgot the other important stuff...