Windows stuck at boot

[Tabiskabis]

When I install the new beta, Windows won't start anymore and gets stuck in the boot screen. I could recover from that by going into Safe Mode (with networking support) and uninstall OpenVPN.

I'm blaming the DCO driver, as that appears to be the only new and related thing. Also the ntbtlog.txt (Boot Log) ends with BOOTLOG_LOADED \SystemRoot\System32\drivers\ovpn-dco.sys

• OpenVPN version: 2.6_beta1
• OS: Windows 10 22H2 Build 19045.2311 (publicly available "Update Preview")
• Possible conflicting drivers: a whole bunch that accumulated during the last 3 years as a sysadmin... WSL stuff, npcap, wireguard, cisco vpn, forticlient vpn, ...

[lstipakov]

Thanks for reporting. I have seen dco driver caused BSODs, but not this kind of behavior. I assume it works fine if you deselect dco driver in installer dialog? Could you by any chance boot with kernel debugger attached and see if there is anything interesting in logs?

[Tabiskabis]

I'm very confident that would work. But I haven't tested and won't, sorry. This made me lose too many hours already (though mainly company policy's fault) that my bossman won't be happy to hear about.

[cron2]

dealt with in https://github.com/OpenVPN/ovpn-dco-win/issues/24

[bunglex]

Hi, I have same issue and lost two full days until I realised what the cause was, did you resolve this?

[lstipakov]

No, we are unable to reproduce this unfortunately. Can you describe what is happening? Do you get any error message?

[lstipakov]

Can you post your boot log? You may need to enable it first in Start -> Run -> msconfig -> Boot -> Bootlog.

[ArinoJenynof]

I also came across this issue whilst upgrading OpenVPN from 2.5.8 to 2.6.0, wondering why my laptop spent 10 or so minutes in the manufacturer logo when usually not even a minute were needed to boot to Windows. Also can confirm when ovpn-dco.sys is not loaded, laptop booted fine.

Here's what i did:

1. Install openvpn 2.6.0 with ovpn-dco.sys selected / enabled it through Autoruns, also enable bootlog.
2. Reboot to see laptop stuck at manufacturer logo, wait for about 5 minutes, then force shutdown by holding power button.
3. Repeat step 2 until i boot to WinRE, boot to safe mode (basically this)
4. Whilst in safe mode, disable ovpn-dco.sys using Autoruns.
5. Reboot and see that the laptop successfully boot to Windows. Disable bootlog and copy ntbtlog.txt somewhere safe.

Unfortunately i cannot found BOOTLOG_LOADED \SystemRoot\System32\drivers\ovpn-dco.sys line in it like OP, but here it is anyway. Windows 10 version is 22H2 Build 19045.2486.

[lstipakov]

@ArinoJenynof thank you.

Do you happen to have anything special installed on your laptop? Special in terms of drivers / hardware. Could you please post an output of systeminfo?

[ArinoJenynof]

Just a typical entry-level gaming laptop, i think (Acer AN515-54). As far as i know there is nothing special in the configuration, maybe Nvidia Optimus, but that's common in laptop like this. I do notice an "Airplane Mode Driver" that came from Acer themselves, instead of the big three (Intel, Nvidia, AMD) vendors. Here's systeminfo.txt

[lstipakov]

@ArinoJenynof Good news, my colleague was able to reproduce this exact issue on the same laptop. Now I just need to get this model and fix it :)

Here is systeminfo from another Acer AN515-54 which experiences this issue

C:\Users\Brandon>systeminfo

Nombre de host:                            LAPTOP-Q54T2RDV
Nombre del sistema operativo:              Microsoft Windows 10 Home Single Language
Versión del sistema operativo:             10.0.19044 N/D Compilación 19044
Fabricante del sistema operativo:          Microsoft Corporation
Configuración del sistema operativo:       Estación de trabajo independiente
Tipo de compilación del sistema operativo: Multiprocessor Free
Propiedad de:                              Brandon
Organización registrada:                   N/D
Id. del producto:                          xxx
Fecha de instalación original:             17/10/2021, 3:48:36 a. m.
Tiempo de arranque del sistema:            31/01/2023, 8:38:12 a. m.
Fabricante del sistema:                    Acer
Modelo el sistema:                         Nitro AN515-54
Tipo de sistema:                           x64-based PC
Procesador(es):                            1 Procesadores instalados.
                                           [01]: Intel64 Family 6 Model 158 Stepping 10 GenuineIntel ~2400 Mhz
Versión del BIOS:                          Insyde Corp. V1.33, 17/11/2020
Directorio de Windows:                     C:\Windows
Directorio de sistema:                     C:\Windows\system32
Dispositivo de arranque:                   \Device\HarddiskVolume1
Configuración regional del sistema:        es-mx;Español (México)
Idioma de entrada:                         es-mx;Español (México)
Zona horaria:                              (UTC-05:00) Bogotá, Lima, Quito, Rio Branco
Cantidad total de memoria física:          12.125 MB
Memoria física disponible:                 10.205 MB
Memoria virtual: tamaño máximo:            18.269 MB
Memoria virtual: disponible:               16.647 MB
Memoria virtual: en uso:                   1.622 MB
Ubicación(es) de archivo de paginación:    C:\pagefile.sys
Dominio:                                   WORKGROUP
Servidor de inicio de sesión:              \\LAPTOP-Q54T2RDV
Revisión(es):                              21 revisión(es) instaladas.
                                           [01]: KB5020872
                                           [02]: KB4534170
                                           [03]: KB4537759
                                           [04]: KB4545706
                                           [05]: KB4562830
                                           [06]: KB4586864
                                           [07]: KB5003791
                                           [08]: KB5012170
                                           [09]: KB5022282
                                           [10]: KB5006753
                                           [11]: KB5007273
                                           [12]: KB5011352
                                           [13]: KB5011651
                                           [14]: KB5014032
                                           [15]: KB5014035
                                           [16]: KB5014671
                                           [17]: KB5015895
                                           [18]: KB5016705
                                           [19]: KB5018506
                                           [20]: KB5020372
                                           [21]: KB5005699
Tarjeta(s) de red:                         10 Tarjetas de interfaz de red instaladas.
                                           [01]: Realtek Gaming GbE Family Controller
                                                 Nombre de conexión: Ethernet
                                                 Estado:             Medios desconectados
                                           [02]: Intel(R) Wi-Fi 6 AX200 160MHz
                                                 Nombre de conexión: Wi-Fi
                                                 Estado:             Hardware ausente
                                           [03]: TAP-Windows Adapter V9 for OpenVPN Connect
                                                 Nombre de conexión: Conexión de área local
                                                 Estado:             Medios desconectados
                                           [04]: Hyper-V Virtual Ethernet Adapter
                                                 Nombre de conexión: vEthernet (Default Switch)
                                                 DHCP habilitado:    No
                                                 Direcciones IP
                                                 [01]: 172.28.144.1
                                                 [02]: fe80::1734:855c:56ac:1f3d
                                           [05]: Hyper-V Virtual Ethernet Adapter
                                                 Nombre de conexión: vEthernet (External)
                                                 Estado:             Medios desconectados
                                           [06]: Wintun Userspace Tunnel
                                                 Nombre de conexión: OpenVPN Wintun
                                                 Estado:             Medios desconectados
                                           [07]: TAP-Windows Adapter V9
                                                 Nombre de conexión: OpenVPN TAP-Windows6
                                                 Estado:             Medios desconectados
                                           [08]: TAP Adapter OAS NDIS 6.0
                                                 Nombre de conexión: Conexión de área local 2
                                                 Estado:             Medios desconectados
                                           [09]: TAP-Windows Adapter V9
                                                 Nombre de conexión: Conexión de área local 3
                                                 Estado:             Medios desconectados
                                           [10]: OpenVPN Data Channel Offload
                                                 Nombre de conexión: OpenVPN Data Channel Offload
                                                 Estado:             Hardware ausente
Requisitos Hyper-V:                        Extensiones de modo de monitor de VM: Sí
                                           Se habilitó la virtualización en el firmware: Sí
                                           Traducción de direcciones de segundo nivel: Sí
                                           La prevención de ejecución de datos está disponible: Sí

[lstipakov]

@Tabiskabis @bunglex which hardware do you use? Is this also Acer laptop?

[Tabiskabis]

which hardware do you use? Is this also Acer laptop?

Dell Precision 3550 laptop, Intel i7-10510U (Intel64 Family 6 Model 142 Stepping 12) (Hotfixes, if relevant at all, have changed since my initial posting)

Network Card(s):           17 NIC(s) Installed.
                           [01]: Wintun Userspace Tunnel
                           [02]: Intel(R) Ethernet Connection (10) I219-V
                           [03]: WireGuard Tunnel
                           [04]: TAP-Windows Adapter V9
                           [05]: Hyper-V Virtual Ethernet Adapter
                           [06]: Hyper-V Virtual Ethernet Adapter
                           [07]: Intel(R) Wi-Fi 6 AX201 160MHz
                           [08]: Bluetooth Device (Personal Area Network)
                           [09]: Realtek USB GbE Family Controller
                           [10]: Wintun Userspace Tunnel
                           [11]: TAP-Windows Adapter V9
                           [12]: TAP-Windows Adapter V9
                           [13]: VirtualBox Host-Only Ethernet Adapter
                           [14]: Wintun Userspace Tunnel
                           [15]: VirtualBox Host-Only Ethernet Adapter
                           [16]: Fortinet Virtual Ethernet Adapter (NDIS 6.30)
                           [17]: Fortinet SSL VPN Virtual Ethernet Adapter
Hyper-V Requirements:      A hypervisor has been detected. Features required for Hyper-V will not be displayed.
Hotfix(s):                 29 Hotfix(s) Installed.
                           [01]: KB5022405
                           [02]: KB4562830
                           [03]: KB4570334
                           [04]: KB4577266
                           [05]: KB4577586
                           [06]: KB4580325
                           [07]: KB4586864
                           [08]: KB4589212
                           [09]: KB4593175
                           [10]: KB4598481
                           [11]: KB5000736
                           [12]: KB5003791
                           [13]: KB5012170
                           [14]: KB5015684
                           [15]: KB5019275
                           [16]: KB5006753
                           [17]: KB5007273
                           [18]: KB5009636
                           [19]: KB5011352
                           [20]: KB5011651
                           [21]: KB5012677
                           [22]: KB5014032
                           [23]: KB5014035
                           [24]: KB5014671
                           [25]: KB5015895
                           [26]: KB5016705
                           [27]: KB5018506
                           [28]: KB5020372
                           [29]: KB5005699

[lstipakov]

Another list of problematic models:

Model: XPS 15 9500, Windows Version: 10.0.19045.2486
Model: XPS 13 7390, Windows Version: 10.0.19045.2486
Model: XPS 13 9305, Windows Version: 10.0.19045.2486
Model: XPS 13 7390, Windows Version : 10.0.19045.2364

Note that on another XPS 7390 everything works just fine.

[Gulaschcowboy]

Dell Latiude 7400/Latitutde 7520 with Win10 (different builds) also affected

[lstipakov]

Since I am unable to reproduce this on our machine, I would like to ask for assistance in figuring out the cause. I have prepared a new version of the driver with enhanced logging which hopefully sheds some light on what is happening.

ovpn-dco-0.8.4.zip

• Extract this archive and run install.bat from Administrator command prompt. This will install (or update) the driver to the version with improved logging. This will also enable driver logging at boot phase (AutoLogger session)
• Reboot and experience the problem (manufacturer logo and spinning circle, boot not progressing)
• Reboot in Safe Mode
• You should have driver logs in C:\Windows\System32\LogFiles\WMI\ovpn-dco.etl (and etl.001, 002, 003 etc after each reboot)
• Paste those files here. They do not contain any private info, just ovpn-dco driver logs. If you have no network in Safe Mode, disable ovpn-dco driver via Device Manager and reboot (you might want to disable the driver in any case)

[Tabiskabis]

@lstipakov, will devcon remove ovpn-dco.inf suffice to get rid of the driver again?

[lstipakov]

@lstipakov, will devcon remove ovpn-dco.inf suffice to get rid of the driver again?

Should be suffice, yes.

[Tabiskabis]

@lstipakov ovpn-dco.etl.001.zip and ntbtlog.zip (different time, though)

[lstipakov]

@Tabiskabis thanks. Unfortunately the driver log is empty. Just to be sure - is this the log from problematic boot?

I found something interesting in your ntbtlog -

12  5 2022 22:12:40.500
BOOTLOG_LOADED \SystemRoot\system32\ntoskrnl.exe
...
BOOTLOG_LOADED \SystemRoot\System32\drivers\ovpn-dco.sys 

Looks like Windows managed to load DCO driver in last May. Have you used some beta client at that time?

[lstipakov]

also @Tabiskabis can you look into Event Log and see if there is anything about dco driver?

[lstipakov]

and do you happen to have ovpn-dco.etl?

[lstipakov]

Traces from problematic machine:

ovpn-dco.zip

There seem to be no warnings/errors.

[Tabiskabis]

DCO in May? I can't remember, but you're probably right, I must have been trying a beta, then. No ovpn-dco.etl, but ovpn-dco.etl.002.zip from a later boot. Event viewer is full of errors complaining about services that could not be startet. Last log before these start is The vmms service terminated with the following error: Ran out of memory

[lstipakov]

Close when I004 is out.

[lstipakov]

@Tabiskabis @Gulaschcowboy @ArinoJenynof This bug should be fixed in upcoming release. Could you give this build a try? https://github.com/OpenVPN/openvpn-build/suites/10769949141/artifacts/541018299

[Tabiskabis]

~Will test~ the 3550 ~on monday~ [update: boots normally now], ~but~ the new version is looking good on a Precision 3560. Except the connected interface always shows received packets: 0 - is that normal?

Meanwhile, i've tested a Precision 3560 with the same issue. Removed as many drivers as possible, then installed ovpn-dco and rebooted. It worked. Windows Update reinstalled most of them and the next reboot failed. Then tried a binary search (removing fewer and fewer drivers). In the end i thought i had nailed it down to either Intel Thunderbolt, Management Engine, or Wireless Manageability drivers. But either it's a certain driver combination, or the issue is a heisenbug.

[ArinoJenynof]

This bug should be fixed in upcoming release. Could you give this build a try? https://github.com/OpenVPN/openvpn-build/suites/10769949141/artifacts/541018299

Thanks, that build works. No more hanging on my laptop.

[lstipakov]

Thanks, I assume you are (still) able to connect with DCO driver? Lähetetty iPhonestaArino Jenynof @.***> kirjoitti 5.2.2023 kello 17.53:

This bug should be fixed in upcoming release. Could you give this build a try? https://github.com/OpenVPN/openvpn-build/suites/10769949141/artifacts/541018299

Thanks, that build works. No more hanging on my laptop.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you modified the open/close state.Message ID: @.***>

[ArinoJenynof]

Yes, also tried that. No issues connecting.

[Heir93]

This bug should be fixed in upcoming release. Could you give this build a try? https://github.com/OpenVPN/openvpn-build/suites/10769949141/artifacts/541018299

Thanks, that build works. No more hanging on my laptop.

Do you mind resharing the link? It doesn't seem to work anymore.

[Pippin1st]

Hi,

Posted on forum: https://forums.openvpn.net/viewtopic.php?p=110487 ;)

[lstipakov]

And here is direct link - https://build.openvpn.net/downloads/releases/OpenVPN-2.6.0-I004-amd64.msi

[Heir93]

Thank you all. Can confirm that it is working.

[lstipakov]

Fixed in 2.6.0-I004.