search

NREL / EnergyPlus

EnergyPlus™ is a whole building energy simulation program that engineers, architects, and researchers use to model both energy consumption and water use in buildings.
https://energyplus.net
Other
1.05k stars 379 forks source link

Mac armv8: Docker energyplus input_file: File does not exist: energyplus #10322

Closed rh0dium closed 6 days ago

rh0dium commented 7 months ago

Issue overview

First reported over on https://github.com/NREL/OpenStudio/issues/5057 but narrowed down to energyplus.

When running the energyplus container 23.2.0 on OSX (M1 Max) you get the following error.

>  energyplus -w ../../weather/USA_CO_Denver.Intl.AP.725650_TMY3.epw  in.idf
input_file: File does not exist: energyplus
Run with --help for more information.

Note this same command works fine in on 23.1.0 

>  energyplus -w ../../weather/USA_CO_Denver.Intl.AP.725650_TMY3.epw in.idf
EnergyPlus Starting
EnergyPlus, Version 23.1.0-87ed9199d4, YMD=2023.12.13 16:19
**FATAL:Errors occurred on processing input file. Preceding condition(s) cause termination.
EnergyPlus Run Time=00hr 00min  0.11sec
Program terminated: EnergyPlus Terminated--Error(s) Detected.

_Note: the errors are due to the in.idf generated for 23.2. What is important is that it ran`

@jmarrec - Suggested that E+ 23.2.0 is the first release that uses the revamped CLI based on the modern C++ lib CLI11, https://github.com/CLIUtils/CLI11, I think this is what's causing the issue.

Note: this works fine outside of the container!

Just trying to run simulations ;) I don't think this is related to the OS-ERI as this does work when running outside of the container.

Details

Some additional details for this issue (if relevant):

Context

Add to this list or remove from it as applicable. This is a simple templated set of guidelines.

jmarrec commented 7 months ago

Also note that I have a mac M1 Max with the same exact system version and it works perfectly fine on my machine

jmarrec commented 7 months ago

Porting over some debugging @rh0dium did here: https://github.com/NREL/OpenStudio/issues/5057#issuecomment-1854188955

>  /usr/local/openstudio-3.7.0/EnergyPlus/energyplus -w ../../weather/USA_CO_Denver.Intl.AP.725650_TMY3.epw --debug-cli       

state.dataGlobal->AnnualSimulation = false,
state.dataGlobal->DDOnlySimulation = false,
state.dataStrGlobals->outDirPath = '',
state.dataStrGlobals->inputIddFilePath= '/usr/local/openstudio-3.7.0/EnergyPlus/Energy+.idd',

runEPMacro = false,
prefixOutName = eplus,

state.dataGlobal->runReadVars=false,
state.dataGlobal->outputEpJSONConversion=false,
state.dataGlobal->outputEpJSONConversionOnly=false,

suffixType=L,

state.dataGlobal->numThread=1,
state.files.inputWeatherFilePath.filePath='../../weather/USA_CO_Denver.Intl.AP.725650_TMY3.epw',
state.dataStrGlobals->inputFilePath='/usr/local/openstudio-3.7.0/EnergyPlus/energyplus',

The fact that state.dataStrGlobals->inputFilePath='/usr/local/openstudio-3.7.0/EnergyPlus/energyplus' looks VERY strange to me. It's supposed to be just in.idf when the IDF file is omitted.

On my machine:

$ /usr/local/openstudio-3.7.0/EnergyPlus/energyplus -w ../../weather/USA_CO_Denver.Intl.AP.725650_TMY3.epw --debug-cli

state.dataGlobal->AnnualSimulation = false,
state.dataGlobal->DDOnlySimulation = false,
state.dataStrGlobals->outDirPath = '',
state.dataStrGlobals->inputIddFilePath= '/usr/local/openstudio-3.7.0/EnergyPlus/Energy+.idd',

runEPMacro = false,
prefixOutName = eplus,

state.dataGlobal->runReadVars=false,
state.dataGlobal->outputEpJSONConversion=false,
state.dataGlobal->outputEpJSONConversionOnly=false,

suffixType=L,

state.dataGlobal->numThread=1,
state.files.inputWeatherFilePath.filePath='../../weather/USA_CO_Denver.Intl.AP.725650_TMY3.epw',
state.dataStrGlobals->inputFilePath='in.idf',

Somehow is looks like it understands /usr/local/openstudio-3.7.0/EnergyPlus/energyplus as the first argument itself instead of being the program name?!

jmarrec commented 7 months ago

On my M1 Max, testing the official E+ docker image (https://hub.docker.com/r/nrel/energyplus/tags , from https://github.com/NREL/docker-energyplus)

$ docker run -it --rm --platform linux/amd64 nrel/energyplus:23.2.0 bash

root@32d34102cfbf:/# energyplus --debug-cli

state.dataGlobal->AnnualSimulation = false,
state.dataGlobal->DDOnlySimulation = false,
state.dataStrGlobals->outDirPath = '',
state.dataStrGlobals->inputIddFilePath= '/EnergyPlus-23.2.0-7636e6b3e9-Linux-Ubuntu20.04-x86_64/Energy+.idd',

runEPMacro = false,
prefixOutName = eplus,

state.dataGlobal->runReadVars=false,
state.dataGlobal->outputEpJSONConversion=false,
state.dataGlobal->outputEpJSONConversionOnly=false,

suffixType=L,

state.dataGlobal->numThread=1,
state.files.inputWeatherFilePath.filePath='in.epw',
state.dataStrGlobals->inputFilePath='in.idf',

Then running a simulation works fine.

cd home
apt update && apt install curl
curl -sLO https://raw.githubusercontent.com/NREL/EnergyPlus/develop/weather/USA_CO_Golden-NREL.724666_TMY3.epw
curl -sLO https://raw.githubusercontent.com/NREL/EnergyPlus/develop/testfiles/1ZoneUncontrolled.idf
cp 1ZoneUncontrolled.idf in.idf
energyplus -w USA_CO_Golden-NREL.724666_TMY3.epw in.idf 
[...]
EnergyPlus Completed Successfully.

@rh0dium, can you confirm this also fails for you please?

rh0dium commented 7 months ago

@jmarrec

This fails for me.

❯ docker run -it --rm --platform linux/amd64 nrel/energyplus:23.2.0 bash
root@b3439f0a1bc8:/# energyplus --debug-cli
input_file: File does not exist: energyplus
Run with --help for more information.
root@b3439f0a1bc8:/#

Same with this 

>> cd home
>> apt update && apt install curl
>> curl -sLO https://raw.githubusercontent.com/NREL/EnergyPlus/develop/weather/USA_CO_Golden->> NREL.724666_TMY3.epw
>> curl -sLO https://raw.githubusercontent.com/NREL/EnergyPlus/develop/testfiles/1ZoneUncontrolled.idf
>> cp 1ZoneUncontrolled.idf in.idf
>> energyplus -w USA_CO_Golden-NREL.724666_TMY3.epw in.idf 
input_file: File does not exist: energyplus
Run with --help for more information.

Can you confirm the following?

Docker Desktop 4.25.2 (129061) Engine: 24.0.6 Compose: v2.23.0-desktop.1

This seems to look similar to #10308 ¯_(ツ)_/¯

rh0dium commented 7 months ago

And even more bizzare is that this work natively..

> curl -sLO https://raw.githubusercontent.com/NREL/EnergyPlus/develop/weather/USA_CO_Golden->> NREL.724666_TMY3.epw
> curl -sLO https://raw.githubusercontent.com/NREL/EnergyPlus/develop/testfiles/1ZoneUncontrolled.idf
> cp 1ZoneUncontrolled.idf in.idf
>  /Applications/OpenStudio-3.7.0/EnergyPlus/energyplus -w USA_CO_Golden-NREL.724666_TMY3.epw in.idf
[...]
EnergyPlus Completed Successfully.
ckm47 commented 5 months ago

I was running into similar issues and was able to solve it by changing the following docker settings:

Configuration:

rh0dium commented 4 months ago

@ckm47 I can confirm your docker setting changes will make it work. I was able to narrow this a bit more simply shifting from VirtioFS to gPRC FUSE (which disables Rosetta) will make this work. Then I shifted back to VirtioFS but keeping Rosetta disabled. Still works.

tldr; Uncheck Rosetta for x86/amd64 emulation on Apple Silicon under Docker Desktop Settings will FIX this issue. This is not ideal but will work.

FYI - Verified this using Docker Desktop v4.27.2

rh0dium commented 4 months ago

Couple more tests with Rosetta enabled. I wanted to verify that this was with EnergyPlus and not a Mac docker issue.

cd home
apt update && apt install curl
curl -sLO https://raw.githubusercontent.com/NREL/EnergyPlus/develop/weather/USA_CO_Golden-NREL.724666_TMY3.epw
curl -sLO https://raw.githubusercontent.com/NREL/EnergyPlus/develop/testfiles/1ZoneUncontrolled.idf
cp 1ZoneUncontrolled.idf in.idf
energyplus -w USA_CO_Golden-NREL.724666_TMY3.epw in.idf

Version 22.1.0: WORKS

Version 22.2.0: WORKS

Version 23.1.0 WORKS

Version 23.2.0 FAILS

So this isn't a docker issue this is something that changed between 23.1.0 and 23.2.0.

Only 1600 files changed.

shorowit commented 4 months ago

As @jmarrec said earlier, this seems like the likely culprit: https://github.com/NREL/EnergyPlus/pull/10148

rh0dium commented 3 months ago

@jmarrec is there anything I can do to help this issue? Happy to contribute if you point me in the right direction.

jmarrec commented 3 months ago

I didn't really want to have to debug that stuff to be honest (when you start having to dig into docker and rosetta troubles you're getting pretty far outside of E+ internals), but I still did.

Turns out the issue is that I rely on CLI11's (circa 2.3.2) interpretation of argc/argv, which for linux reads /proc/self/cmdline separated by null terminators.

https://github.com/CLIUtils/CLI11/blob/985a19f3860be0cba87354336f6588494b20111c/include/CLI/impl/Argv_inl.hpp#L141

When you run under rosetta, you end up with a duplicated program path, looks like rosetta is trying to remove itself from the command line but duplicating it. So wheen you call energyplus --help what cmdline has is energyplus energyplus --help

This is essentially the same issue as described here: https://github.com/docker/for-mac/issues/7058

Someone created a tool to fix it, cf comment: https://github.com/docker/for-mac/issues/7058#issuecomment-1928714175 and https://github.com/norio-nomura/re-register-rosetta

This is an implementation that follows what's written in the doc https://developer.apple.com/documentation/virtualization/running_intel_binaries_in_linux_vms_with_rosetta#3978496

After running docker run --platform=linux/arm64 --privileged --rm ghcr.io/norio-nomura/re-register-rosetta I can successfully run docker run -it --rm --platform linux/amd64 -v /Users/julien/Software/OSCLI11:/home/OSCLI11 nrel/energyplus:23.2.0 bash and use energyplus

Mind that the end of the linked issue says it's a bug in Rosetta and it's tracked by rdar://123115850 (that's Apple Radar, their internal bug tracking system, so you have no way to see if that's being addressed or not).

jmarrec commented 3 months ago

@Myoldmopar updating to CLI 2.40+ (and not relying on cmdline) is possible to remove the issue altogether, but not sure if that's a worthwhile time investment or not.

rh0dium commented 2 months ago

@jmarrec Wow! Thanks for digging to this and yeah I was afraid this was going to really be a can of worms. @Myoldmopar What would moving to 2.40+ look like - Can we help in any way?

rh0dium commented 2 months ago

FYI - I can confirm that running this one time command will solve this locally.

> docker run --platform=linux/arm64 --privileged --rm ghcr.io/norio-nomura/re-register-rosetta

Followed then by:

> docker run -it --rm --platform linux/amd64 nrel/energyplus:23.2.0 bash

root@f52c1b898f60:/# energyplus --debug-cli

state.dataGlobal->AnnualSimulation = false,
state.dataGlobal->DDOnlySimulation = false,
state.dataStrGlobals->outDirPath = '',
state.dataStrGlobals->inputIddFilePath= '/EnergyPlus-23.2.0-7636e6b3e9-Linux-Ubuntu20.04-x86_64/Energy+.idd',

So that's a solid work around.

jmarrec commented 3 weeks ago

@rh0dium do you happen to have a mac where you could reproduce this? One where you did not run ghcr.io/norio-nomura/re-register-rosetta. If so, I'd appreciate if you could test with https://github.com/jmarrec/EnergyPlus/releases/tag/v24.2.0-docker_cmdline please?

jmarrec commented 1 week ago

Alright, I tested myself. Nevermind.

rh0dium commented 4 days ago

@jmarrec Sorry for the late reply on this thanks for fixing this.