bencheeorg / benchee

Easy and extensible benchmarking in Elixir providing you with lots of statistics!
MIT License
1.42k stars 66 forks source link

WMIC has been deprecated in Windows 11 and is now no longer available #442

Open doughsay opened 1 day ago

doughsay commented 1 day ago

This line here: https://github.com/bencheeorg/benchee/blob/741db4d1c282543bca6857204df9ec59c65100dd/lib/benchee/system.ex#L114

Appears to fail on up-to-date Windows 11 systems due to WMIC having been deprecated.

Here's an article announcing the deprecation: https://techcommunity.microsoft.com/blog/windows-itpro-blog/wmi-command-line-wmic-utility-deprecation-next-steps/4039242

It seems the only alternative is to use powershell, according to that article. I'm pretty unfamiliar with windows development, so I don't really know the best way to fix this...

PragTob commented 16 hours ago

:wave:

Hey, thank you for the issue and the announcement! But also... ooof. I think that was mostly done externally back then as well and I also don't have a Windows 11 device to test this right now (yeah there should still be the VMs around).

What's the impact right now? Does benchee crash as a whole or does it just not report the values? I dearly hope the latter otherwise we got some fixing to do :sweat_smile:

PragTob commented 16 hours ago

@OvermindDL1 you still around? Any ideas? :grin:

kshannoninnes commented 10 hours ago

Sorry, he made the issue on my behalf (I brought it up in the Elixir discord but it was 4:30am and I was heading to bed). I'll be happy to run tests/answer any questions about the issue, although given timezones, It'll probably be several hours before I'm awake again.

Running the latest up to date Windows 11 (reinstalled from scratch around a week prior to this issue) with Erlang 27 and Elixir 1.17.3

Minimal Reproducible Example

Mix.install([
    {:benchee, "~> 1.0"}
])

Benchee.run(%{
    "test" => fn -> IO.puts("Hello World") end
})

Resulting Output

C:\Users\*******\Documents\Elixir\aoc2024>elixir test.exs
** (ErlangError) Erlang error: :enoent
    (elixir 1.17.3) lib/system.ex:1114: System.cmd("WMIC", ["COMPUTERSYSTEM", "GET", "TOTALPHYSICALMEMORY"], [])
    (benchee 1.3.1) lib/benchee/system.ex:235: Benchee.System.system_cmd/3
    (benchee 1.3.1) lib/benchee/system.ex:171: Benchee.System.available_memory/1
    (benchee 1.3.1) lib/benchee/system.ex:57: Benchee.System.system/1
    (benchee 1.3.1) lib/benchee.ex:46: Benchee.run/2
    test.exs:6: (file)
PragTob commented 7 hours ago

@kshannoninnes got it, thanks! I should be able to at least stop that from failing and leave retrieving the value for later....

maybe I should join the elixir discord to be pingable there.

OvermindDL1 commented 7 hours ago

Honestly no clue beyond just doing it in powershell, it's not really any slower than wmic too (especially once it's cached).

What are you trying to get information of that BEAM doesn't already expose somehow?

I guess you could go the sledgehammer method and make a rustler crate to link in rusts's sysinfo crate and get whatever system information you want on whatever arch/os you want though since it supports basically everything with a modern OS?