armbian / build

Armbian Linux build framework generates custom Debian or Ubuntu image for x86, aarch64, riscv64 & armhf
https://www.armbian.com
GNU General Public License v2.0
4.28k stars 2.35k forks source link

New bcm driver for M2+ and X2 - needs firmware files to be consolidated. #484

Closed spock64 closed 5 years ago

spock64 commented 8 years ago

Igor, As mentioned in my post at: http://forum.armbian.com/index.php/topic/872-beelink-x2-with-armbian-possible/page-8#entry16758 ... some firmware files need to be moved around before the new version of the BCM dhd driver is fully working on Beelink X2. At the moment it works "by chance" on the banana pi M2+. Jason

ThomasKaiser commented 8 years ago

Can confirm that Wi-Fi is now broken on Beelink X2. As I understand the issue we should come up with a single solution for Beelink X2 (also taking into account that newer revisions now use RTL8189ETV instead) and also Banana Pi M2+, Banana Pro, Cubietruck and NanoPi Air using a consolidated firmware, no Wi-Fi modules added to /etc/modules and echo 123 > /proc/driver/wifi-pm/power (maybe added to /etc/rc.local?)

zador-blood-stained commented 8 years ago

Maybe we can run some kind of autodetection to determine Wi-Fi chip IDs? For example this can be done on cubietruck:

➜  ~  % cat /sys/bus/sdio/devices/mmc1:0001:1/vendor
0x02d0
➜  ~  % cat /sys/bus/sdio/devices/mmc1:0001:1/device
0xa962
➜  ~  %

and this on Orange Pi+2E

root@orangepiplus2e:~# cat /sys/bus/sdio/devices/mmc1:0001:1/vendor
0x024c
root@orangepiplus2e:~# cat /sys/bus/sdio/devices/mmc1:0001:1/device
0xf179
root@orangepiplus2e:~#

but only after wireless driver was loaded as far as I understand.

ThomasKaiser commented 8 years ago

My (old) Beelink X2 shares vendor/device info with Cubietruck. Here's dmesg output contained: http://sprunge.us/PIFP

If I understood @jernejsk correctly, all that's needed is config.txt responsible for choosing the correct firmware file (and consolidated paths maybe).

I totally lost track but does Armbian contains all changes from https://github.com/jernejsk/wlan-firmware/commits/master already?

In case anyone deals with this firmware consolidation stuff and taking care of armbian-firmware and armbian-firmware-full package: Adding the three files from https://github.com/lwfinger/rtl8723bs_bt would be great too (BT on Pine64):

root@pine64:/lib/firmware/rtl_bt# ls -la
total 108
drwxr-xr-x 2 root root  4096 Oct 23 14:39 .
drwxr-xr-x 3 root root  4096 Oct 23 14:39 ..
-rw-r--r-- 1 root root    55 Oct 23 14:39 rtlbt_config
-rw-r--r-- 1 root root 44504 Oct 23 14:39 rtlbt_fw
-rw-r--r-- 1 root root 51364 Oct 23 14:39 rtlbt_fw_new
jernejsk commented 8 years ago

From what I know, driver side is covered. Only thing now is to move all bcm FWs to common folder and update paths in kernel config accordingly. However, that may be tedious, if same FWs are used on more architectures.

rhemberger commented 8 years ago

What I did: I copied all firmware files from BeelinkX2-Android /vendor/modules to /lib/firmware/ap6212 and linked nvram_ap6210.txt to nvram.txt. Works immediately. A little bit ugly, and the link has to be renewed after an Armbian firmware update. I think, a proper /lib/firmware/ap6212/config.txt would handle this. But I think, the syntax from https://github.com/jernejsk/wlan-firmware/blob/34d66e85f8c6ce5b0291bc4aa0cc5c0dd88d659d/firmware/brcm/config.txt is wrong. Perhaps something like one line: "nv_by_chip=2 43362 1 nvram_ap6210.txt 43430 0 nvram_ap6212.txt" but i don't understand C.

rhemberger commented 8 years ago

Confirming myself: lib/firmware/ap6212/config.txt gives: `[ 10.359348] dhd_module_init: in [ 10.359370] ======== bcm_wlan_set_plat_data ======== [ 10.359394] bcm_wlan_get_oob_irq enter. [ 10.359529] gpio [202] map to virq [10] ok [ 10.359543] host_oob_irq: 10 [ 10.359557] host_oob_irq_flags=0x414 [ 10.359571] dhd_wifi_platform_load: Enter [ 10.359617] Power-up adapter 'DHD generic adapter' [ 10.359630] wifi_platform_set_power = 1 [ 10.359640] ======== PULL WL_REG_ON HIGH! ======== [ 10.359652] [wifi_pm]: set wl_reg_on 0 ! [ 10.746203] twi_start()434 - [i2c0] START can't sendout! [ 10.746483] twi_start()434 - [i2c0] START can't sendout! [ 10.746754] twi_start()434 - [i2c0] START can't sendout! [ 10.747028] twi_start()434 - [i2c0] START can't sendout! [ 10.747299] twi_start()434 - [i2c0] START can't sendout! [ 10.747570] twi_start()434 - [i2c0] START can't sendout! [ 10.749753] twi_start()434 - [i2c0] START can't sendout! [ 10.750882] twi_start()434 - [i2c0] START can't sendout! [ 10.751168] twi_start()434 - [i2c0] START can't sendout! [ 10.751316] bmp085: probe of 0-0077 failed with error -121 [ 10.892764] [wifi_pm]: set wl_reg_on 1 ! [ 10.963024] vmouse_input_dev_open [ 11.040689] vmouse_input_dev_close [ 11.139435] gmac0: probed [ 11.139822] gmac0 gmac0: eth0: eth0: PHY ID 00441400 at 0 IRQ poll (gmac0-0:00) [ 11.183442] systemd-journald[230]: Received request to flush runtime journal from PID 1 [ 11.750182] wifi_platform_bus_enumerate device present 1 [ 11.750210] ======== Card detection to detect SDIO card! ======== [ 11.750582] dummy_sdmmc: probe of mmc1:0001:1 failed with error -123 [ 11.750940] mmc1: card 0001 removed [ 11.751402] [mmc]: sdc1 power_supply is null [ 11.755769] [mmc]: sdc1 power_supply is null [ 11.807781] mmc1: queuing unknown CIS tuple 0x80 (2 bytes) [ 11.809370] mmc1: queuing unknown CIS tuple 0x80 (3 bytes) [ 11.811035] mmc1: queuing unknown CIS tuple 0x80 (3 bytes) [ 11.813863] mmc1: queuing unknown CIS tuple 0x80 (7 bytes) [ 11.831200] mmc1: new high speed SDIO card at address 0001 [ 11.848345] bcmsdh_register: register client driver [ 11.848545] bcmsdh_sdmmc: bcmsdh_sdmmc_probe Enter [ 11.848700] bcmsdh_sdmmc: bcmsdh_sdmmc_probe Enter [ 11.848717] bus num (host idx)=1, slot num (rca)=1 [ 11.848731] found adapter info 'DHD generic adapter' [ 11.848861] sdioh_attach: set sd_f2_blocksize 128 [ 11.849608] F1 signature read @0x18000000=0x1591a962 [ 11.849996] dhd_conf_set_hw_oob_intr: Enable HW OOB for 43362 [ 11.854091] F1 signature OK, socitype:0x1 chip:0xa962 rev:0x1 pkg:0x9 [ 11.855579] DHD: dongle ram size is set to 245760(orig 245760) at 0x0 [ 11.856903] dhd_conf_set_chiprev: chip=0xa962, chiprev=1 [ 11.857011] dhd_conf_set_conf_path_by_nv_path: config_path=/lib/firmware/ap6212/config.txt [ 11.859484] dhd_conf_read_nv_by_chip: nv_by_chip_count=2 [ 11.859512] dhd_conf_read_nv_by_chip: chip=0xa9a6, chiprev=0, name=nvram_ap6212.txt [ 11.859534] dhd_conf_read_nv_by_chip: chip=0xa962, chiprev=1, name=nvram_ap6210.txt [ 11.861341] tsk Enter, tsk = 0xed841430 [ 11.861379] wl_create_event_handler(): thread:wl_event_handler:1da started [ 11.862062] dhd_attach(): thread:dhd_watchdog_thread:1db started [ 11.862218] dhd_dpc_thread: set dpc_cpucore 0 from config.txt [ 11.862274] dhd_attach(): thread:dhd_dpc:1dc started [ 11.862457] dhd_attach(): thread:dhd_rxf:1dd started [ 11.862487] dhd_deferred_work_init: work queue initialized [ 11.863234] dhd_conf_read_nv_by_chip: nv_by_chip_count=2 [ 11.863263] dhd_conf_read_nv_by_chip: chip=0xa9a6, chiprev=0, name=nvram_ap6212.txt [ 11.863284] dhd_conf_read_nv_by_chip: chip=0xa962, chiprev=1, name=nvram_ap6210.txt [ 11.863881] dhd_conf_set_fw_name_by_chip: firmware_path=/lib/firmware/ap6212/fw_bcm40181a2.bin [ 11.863910] dhd_conf_set_nv_name_by_chip: nvram_path=/lib/firmware/ap6212/nvram_ap6210.txt [ 11.863930] dhdsdio_download_firmware: set use_rxchain 0 [ 11.863944] dhdsdio_download_firmware: set txglomsize 16 [ 11.863959] sdioh_set_mode: set txglom_mode to multi-desc [ 11.863972] Final fw_path=/lib/firmware/ap6212/fw_bcm40181a2.bin [ 11.863986] Final nv_path=/lib/firmware/ap6212/nvram_ap6210.txt [ 11.864005] Final conf_path=/lib/firmware/ap6212/config.txt [ 11.908584] NVRAM version: AP6210_NVRAM_V1.3_08182013 [ 11.909465] dhdsdio_write_vars: Download, Upload and compare of NVRAM succeeded. [ 11.910612] dhd_bus_init: delay 100ms for BCM43362 [ 12.131709] dhd_bus_init: enable 0x06, ready 0x06 (waited 0us) [ 12.131875] bcmsdh_oob_intr_register: Enter [ 12.131889] bcmsdh_oob_intr_register: HW_OOB enabled [ 12.131904] bcmsdh_oob_intr_register OOB irq=10 flags=414 [ 12.132058] bcmsdh_oob_intr_register: disable_irq_wake [ 12.133106] dhd_conf_set_fw_int_cmd: set WLC_SET_BAND 142 0 [ 12.133917] dhd_preinit_ioctls: Set tcpack_sup_mode 0 [ 12.135071] Firmware up: op_mode=0x0001, MAC=e0:76:d0:09:31:2f [ 12.135090] dhd_conf_set_country: set country ALL, revision 0 [ 12.135106] dhd_conf_set_fw_string_struct_cmd: set country [ 12.208801] Country code: ALL (ALL/0) [ 12.209363] dhd_conf_set_fw_string_cmd: set roam_off 1 [ 12.213143] dhd_preinit_ioctls buf_key_b4_m4 set failed -23 [ 12.223449] Firmware version = wl0: Mar 10 2014 14:59:50 version 5.90.195.89.18 FWID 01-b13c09c7 [ 12.223481] Driver: 1.201.59.6 (r506368) [ 12.223488] Firmware: wl0: Mar 10 2014 14:59:50 version 5.90.195.89.18 FWID 01-b13c09c7 [ 12.224014] dhd_txglom_enable: enable 0 [ 12.224030] dhd_conf_set_txglom_params: swtxglom=0, txglom_ext=0 [ 12.224044] dhd_conf_set_txglom_params: txglom_bucket_size=0 [ 12.224060] dhd_conf_set_txglom_params: txglomsize=0, deferred_tx_len=0, bus_txglom=0 [ 12.224078] dhd_conf_set_txglom_params: tx_in_rx=1, tx_max_offset=0 [ 12.224093] dhd_conf_set_disable_proptx: set disable_proptx 1 [ 12.227541] dhd_preinit_ioctls wl ampdu_hostreorder failed -23 [ 12.245055] Dongle Host Driver, version 1.201.59.6 (r506368) [ 12.245069] Compiled in drivers/net/wireless/bcmdhd [ 12.246305] Register interface [wlan0] MAC: e0:76:d0:09:31:2f

[ 12.246467] dhd_module_init: Exit err=0`

spock64 commented 8 years ago

Thomas,

Our messages overlapped - yes I agree that the same device is present on the vintage X2 - but on mmc2 not mmc1. The dhd driver copes with this when it loads and tries to find the right firmware.

The config.txt does need to change I thing - I just need to read the driver code and work out the right syntax.

I think that Igor has taken all the patches from jerenej’s repo - at least last time I looked about 10 days ago he had.

I will try to make something generic and then we could make this work with everything.

Jason

On 23 Oct 2016, at 17:48, Thomas Kaiser notifications@github.com wrote:

My (old) Beelink X2 shares vendor/device info with Cubietruck. Here's dmesg output contained: http://sprunge.us/PIFP http://sprunge.us/PIFP If I understood @jernejsk https://github.com/jernejsk correctly, all that's needed is config.txt http://forum.armbian.com/index.php/topic/872-beelink-x2-with-armbian-possible/?p=16673 responsible for choosing the correct firmware file (and consolidated paths maybe).

I totally lost track but does Armbian contains all changes from https://github.com/jernejsk/wlan-firmware/commits/master https://github.com/jernejsk/wlan-firmware/commits/master already?

In case anyone deals with this firmware consolidation stuff and taking care of armbian-firmware and armbian-firmware-full package: Adding the three files from https://github.com/lwfinger/rtl8723bs_bt https://github.com/lwfinger/rtl8723bs_bt would be great too (BT on Pine64):

root@pine64:/lib/firmware/rtl_bt# ls -la total 108 drwxr-xr-x 2 root root 4096 Oct 23 14:39 . drwxr-xr-x 3 root root 4096 Oct 23 14:39 .. -rw-r--r-- 1 root root 55 Oct 23 14:39 rtlbt_config -rw-r--r-- 1 root root 44504 Oct 23 14:39 rtlbt_fw -rw-r--r-- 1 root root 51364 Oct 23 14:39 rtlbt_fw_new — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/igorpecovnik/lib/issues/484#issuecomment-255599309, or mute the thread https://github.com/notifications/unsubscribe-auth/AViF0x5VZN6xdA7-tqNIo1Fj8pLUgdSPks5q24_PgaJpZM4KME-k.

spock64 commented 8 years ago

Thomas - I reported the “broken” status a few days ago. The issue is that each broadcom device needs its own firmware - jernej had put together a patch that causes the bcmdhd/dhd driver to autodetect the device and load the correct firmware. So, the way I see it, we need to fix the firmware files and the driver / firmware will load properly at boot time. This will avoid a mess in /etc/rc.local I guess ?

Mikhail - we would need a tricky script to get the detection right - on the beelink, the wifi is on mmc2 … Also, the new versions of the dhd driver detects the device itself on load. We should not be trying to second guess this process - all we need to do I think is put the firmware in the right place.

In my tests, pre-comit 47714a8 , I get wifi to work by putting the correct firmware path into /etc/modprobe.d/bcmdhd.conf - so I know the driver is fine.

So my plan for this week is something like:

Then report progress and ask for help for how to get the patches together to make this all work.

Jason

On 23 Oct 2016, at 17:27, Mikhail notifications@github.com wrote:

Maybe we can run some kind of autodetection to determine Wi-Fi chip IDs? For example this can be done on cubietruck:

➜ ~ % cat /sys/bus/sdio/devices/mmc1:0001:1/vendor 0x02d0 ➜ ~ % cat /sys/bus/sdio/devices/mmc1:0001:1/device 0xa962 ➜ ~ % and this on Orange Pi+2E

root@orangepiplus2e:~# cat /sys/bus/sdio/devices/mmc1:0001:1/vendor 0x024c root@orangepiplus2e:~# cat /sys/bus/sdio/devices/mmc1:0001:1/device 0xf179 root@orangepiplus2e:~# but only after wireless driver was loaded as far as I understand.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/igorpecovnik/lib/issues/484#issuecomment-255598082, or mute the thread https://github.com/notifications/unsubscribe-auth/AViF0_s6wpXdAzrWaRXQeb1daRUiaD-qks5q24rcgaJpZM4KME-k.

jernejsk commented 8 years ago

@spock64, why would be tricky script needed? Are you saying that there are two versions of Belink X2 with broadcom wifi, once on mmc1 and secondly on mmc2? I doubt that very much. Even so, which bus to scan for wifi is still script.bin setting.

spock64 commented 8 years ago

@jernej

I was just writing something for you about all of this.

I was saying that the Beelink has the wifi on mmc2 and other boards have it on mmc1. This means that a generic approach is tricky. In any case, I think it would better if the system was set up so that the driver will load the firmware properly during kernel initialisation. If the driver does not load, you can't write a script to probe the bus :-)

Overall, I think the best objective for Armbian is to have the driver download firmware without any configuration if possible. This is important as your patch to rf_pm/wifi_pm.c causes the bus to be probed at kernel initialisation time - and this can cause problems with system starting if the driver fails to load the firmware. The problem then is how to set up the firmware files so that the system loads when the driver has detected the wifi chip.

Right now, the dhd driver detects the chip type and constructs a path for the firmware file but does not construct a unique path for the nvram file. If the driver is patched so that it constructs a path for the nvram file based on the chip id, we can have all the firmware and nvram files in one directory (or use some soft links). The driver will then load the correct firmware and nvram during kernel initialisation.

Do you think this is a good approach ?

On 25 Oct 2016, at 20:02, Jernej Škrabec notifications@github.com wrote:

@spock64 https://github.com/spock64, why would be tricky script needed? Are you saying that there are two versions of Belink X2 with broadcom wifi, once on mmc1 and secondly on mmc2? I doubt that very much. Even so, which bus to scan for wifi is still script.bin setting.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/igorpecovnik/lib/issues/484#issuecomment-256143038, or mute the thread https://github.com/notifications/unsubscribe-auth/AViF0wDtf5Ih3uu_CwJBbP5-X5OA-U-0ks5q3lJJgaJpZM4KME-k.

jernejsk commented 8 years ago

I was saying that the Beelink has the wifi on mmc2 and other boards have it on mmc1. This means that a generic approach is tricky.

Still don't get it why that would be tricky. It is only a single setting in the fex file: https://github.com/igorpecovnik/lib/blob/master/config/fex/beelinkx2.fex#L890

And about your concern for scaning sdio in kernel. Please check dmesg output when this is done. Pretty much at the end. Even then, if FW is not found, driver doesn't crash anymore, now that I fixed it.

Config file is for translation between chip id and nvram file name. You would be solving the problem which was already solved. If you check how it is done in my OE fork, then you would now, that all the files are already in the same folder.

spock64 commented 8 years ago

Yes you are right.

I guess that all that is needed is for a directory to be set up which contains softlinks to all the firmware & nvram files and a config.txt that maps the chip type to the correct nvram file. If this is done properly, there is no need for any scripts ... Apologies - I did check your fork a few weeks back but had missed this part.

I will try this out and report back.

jernejsk commented 8 years ago

I checked both fex files for X2, mine and Armbian, and both have set wifi to sdio1 (a.k.a mmc1). Mine images were tested on both versions of X2, so I think that wifi is at mmc1. Why do you think that it is on mmc2?

spock64 commented 8 years ago

It's mmc2 on my box ... I'm booted off the sd card ...

root@beelinkx2:~# cat /sys/bus/sdio/devices/mmc1:0001:1/vendor cat: /sys/bus/sdio/devices/mmc1:0001:1/vendor: No such file or directory root@beelinkx2:~# cat /sys/bus/sdio/devices/mmc2:0001:1/vendor 0x02d0 root@beelinkx2:~# cat /sys/bus/sdio/devices/mmc2:0001:1/device 0xa962

mmc0 is the sd card, mmc1 is the eMMC ...

spock64 commented 8 years ago

To fix this, the following script needs to be run on the image during the build for the X2. It should then allow the wifi firmware to be downloaded properly. Can someone point me to where this script should go? Should it be part of building the image? ... or should it be part of the first run configuration for the board?

There are a number of improvements that could/should be made to this, but the following should work for now.

--- snip --- cd /lib/firmware/ap6212 ln -s ../ap6210/fw_bcm40181a2.bin . ln -s ../ap6210/fw_bcm40181a2_apsta.bin . ln -s ../ap6210/fw_bcm40181a2_p2p.bin . cp ../ap6210/nvram.txt nvram_ap6210.txt ln -s ../ap6210/bcm20710a1.hcd . ln -s ../ap6210/nvram.txt nvram_ap6212.txt cp ../brcm/config.txt . --- snip ---

rhemberger commented 8 years ago

Why don't copy all bcmdhd files in one directory in https://github.com/ igorpecovnik/lib/tree/master/bin/firmware-overlay as Jernej did: https://github.com/jernejsk/wlan-firmware/tree/ 34d66e85f8c6ce5b0291bc4aa0cc5c0dd88d659d/firmware/brcm ? Adjust CONFIG_BCMDHD_FW_PATH="/lib/firmware/brcm/fw_bcmdhd.bin" to this directory, but let CONFIG_BCMDHD_NVRAM_PATH unset? @Jernej: I could be totally wrong, but shouldn't nv_chip_num in nv_by_chip=[nv_chip_num] in config.txt be the number of following nvram mappings, see https://github.com/jernejsk/linux/blob/master/drivers/net/ wireless/bcmdhd/dhd_config.c#L1540 and https://github.com/jernejsk/ linux/blob/master/drivers/net/wireless/bcmdhd/dhd_config.c#L1555 ?

2016-10-25 21:58 GMT+02:00 spock64 notifications@github.com:

To fix this, the following script needs to be run on the image during the build for the X2. It should then allow the wifi firmware to be downloaded properly. Can someone point me to where this script should go? Should it be part of building the image? ... or should it be part of the first run configuration for the board?

There are a number of improvements that could/should be made to this, but the following should work for now.

--- snip --- cd /lib/firmware/ap6212 ln -s ../ap6210/fw_bcm40181a2.bin . ln -s ../ap6210/fw_bcm40181a2_apsta.bin . ln -s ../ap6210/fw_bcm40181a2_p2p.bin . cp ../ap6210/nvram.txt nvram_ap6210.txt ln -s ../ap6210/bcm20710a1.hcd . ln -s ../ap6210/nvram.txt nvram_ap6212.txt cp ../brcm/config.txt . --- snip ---

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/igorpecovnik/lib/issues/484#issuecomment-256158314, or mute the thread https://github.com/notifications/unsubscribe-auth/AJ4lH_1HghLjCsXMLdqwPLbYl5boV9Tmks5q3l96gaJpZM4KME-k .

jernejsk commented 8 years ago

@ralemb, you are right. I checked driver code and beside that number I also forgot to add "\" at the end of the second line.

rhemberger commented 8 years ago

Just for the record: the firmware files copied from beelinkx2-android and my config.txt are here: https://drive.google.com/drive/folders/0B7tUQydFkK-ATmw3djE4UGowdWM?usp=sharing At a glance the firmware files are the same as you already collected. Perhaps someone can test whether dropping them in the /lib/firmware/ap6212 folder works with the other armbian hardware?

2016-10-26 0:20 GMT+02:00 Jernej Škrabec notifications@github.com:

@ralemb https://github.com/ralemb, you are right. I checked driver code and beside that number I also forgot to add "\" at the end of the second line.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/igorpecovnik/lib/issues/484#issuecomment-256194538, or mute the thread https://github.com/notifications/unsubscribe-auth/AJ4lH7EkE4PK1BpHlRK1883I7iCS12Qyks5q3oCjgaJpZM4KME-k .

spock64 commented 8 years ago

I will check these files against the versions already in Armbian when I get back to my office later today.

The firmware bundle used by Armbian is sourced from kernel.org http://kernel.org/, so assuming nobody has been too clever, they should be the same as yours.

On 26 Oct 2016, at 00:10, ralemb notifications@github.com wrote:

Just for the record: the firmware files copied from beelinkx2-android and my config.txt are here: https://drive.google.com/drive/folders/0B7tUQydFkK-ATmw3djE4UGowdWM?usp=sharing At a glance the firmware files are the same as you already collected. Perhaps someone can test whether dropping them in the /lib/firmware/ap6212 folder works with the other armbian hardware?

2016-10-26 0:20 GMT+02:00 Jernej Škrabec notifications@github.com:

@ralemb https://github.com/ralemb, you are right. I checked driver code and beside that number I also forgot to add "\" at the end of the second line.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/igorpecovnik/lib/issues/484#issuecomment-256194538, or mute the thread https://github.com/notifications/unsubscribe-auth/AJ4lH7EkE4PK1BpHlRK1883I7iCS12Qyks5q3oCjgaJpZM4KME-k .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/igorpecovnik/lib/issues/484#issuecomment-256205491, or mute the thread https://github.com/notifications/unsubscribe-auth/AViF08ei6di5xZ0PYjG1MYH-63bmDZ-sks5q3oxggaJpZM4KME-k.

lanefu commented 5 years ago

closing issue