Conda build pipeline fails for OSX

[asmacdo]

Conda build pipeline is failing on OSX

>       memory_total = os.sysconf("SC_PAGE_SIZE") * os.sysconf("SC_PHYS_PAGES")
E       ValueError: unrecognized configuration name

If I understand correctly, on mac this should be _SC_PAGESIZE and _SC_PHYS_PAGES

IIUC on OSX:

 # E       ValueError: unrecognized configuration name
os.sysconf("SC_PAGE_SIZE")   
os.sysconf("SC_PHYS_PAGES") 
 # should be good?
os.sysconf("_SC_PAGESIZE")  
os.sysconf("_SC_PHYS_PAGES") 

@jwodder you have a mac right? Could you sanity check ^ for me?

In the test pipeline stderr I am also seeing https://github.com/con/duct/issues/82

ps: illegal option -- s

@yarikoptic IMO we should drop support for OSX for now, proceed with the conda-build for linux only. We can address OSX and then cut a new release. wdyt?

[jwodder]

@asmacdo The correct expressions on macOS (It hasn't been OS X since 2016) are os.sysconf("SC_PAGESIZE") and os.sysconf("SC_PHYS_PAGES").

[asmacdo]

@jwodder hmm. thats odd right? The build info looks like it is using up-to-date macOS

2024-07-31T18:48:57.1079870Z ##[section]Starting: Initialize job 2024-07-31T18:48:57.1082990Z Agent name: 'Azure Pipelines 83' 2024-07-31T18:48:57.1083420Z Agent machine name: 'Mac-1722450377722' 2024-07-31T18:48:57.1083640Z Current agent version: '3.242.1' 2024-07-31T18:48:57.1117850Z ##[group]Operating System 2024-07-31T18:48:57.1118100Z macOS 2024-07-31T18:48:57.1118210Z 12.7.5 2024-07-31T18:48:57.1118320Z 21H1222 2024-07-31T18:48:57.1118440Z ##[endgroup] 2024-07-31T18:48:57.1118570Z ##[group]Runner Image 2024-07-31T18:48:57.1118730Z Image: macos-12 2024-07-31T18:48:57.1118880Z Version: 20240721.1 2024-07-31T18:48:57.1119200Z Included Software: https://github.com/actions/runner-images/blob/macos-12/20240721.1/images/macos/macos-12-Readme.md 2024-07-31T18:48:57.1119660Z Image Release: https://github.com/actions/runner-images/releases/tag/macos-12%2F20240721.1 2024-07-31T18:48:57.1119920Z ##[endgroup] 2024-07-31T18:48:57.1120080Z ##[group]Runner Image Provisioner 2024-07-31T18:48:57.1120470Z 2.0.374.1+4097a9592d27ce71de414581a65bffbda888dd1b 2024-07-31T18:48:57.1120680Z ##[endgroup] 2024-07-31T18:48:57.1124320Z Current image version: '20240721.1' 2024-07-31T18:48:57.1943990Z Agent running as: 'runner' 2024-07-31T18:48:57.1994600Z Prepare build directory. 2024-07-31T18:48:57.2391620Z Set build variables. 2024-07-31T18:48:57.2433370Z Download all required tasks. 2024-07-31T18:48:57.2619540Z Downloading task: CmdLine (2.243.1) 2024-07-31T18:48:58.0276100Z Downloading task: PublishPipelineArtifact (1.242.0) 2024-07-31T18:48:58.3375230Z Checking job knob settings. 2024-07-31T18:48:58.3400710Z Knob: DockerActionRetries = true Source: $(VSTSAGENT_DOCKER_ACTION_RETRIES) 2024-07-31T18:48:58.3403190Z Knob: AgentToolsDirectory = /Users/runner/hostedtoolcache Source: ${AGENT_TOOLSDIRECTORY} 2024-07-31T18:48:58.3406790Z Knob: UseGitLongPaths = true Source: $(USE_GIT_LONG_PATHS) 2024-07-31T18:48:58.3413280Z Knob: EnableIssueSourceValidation = true Source: $(ENABLE_ISSUE_SOURCE_VALIDATION) 2024-07-31T18:48:58.3415450Z Knob: AgentEnablePipelineArtifactLargeChunkSize = true Source: $(AGENT_ENABLE_PIPELINEARTIFACT_LARGE_CHUNK_SIZE) 2024-07-31T18:48:58.3422550Z Knob: ContinueAfterCancelProcessTreeKillAttempt = true Source: $(VSTSAGENT_CONTINUE_AFTER_CANCEL_PROCESSTREEKILL_ATTEMPT) 2024-07-31T18:48:58.3423670Z Knob: ProcessHandlerSecureArguments = false Source: $(AZP_75787_ENABLE_NEW_LOGIC) 2024-07-31T18:48:58.3425270Z Knob: ProcessHandlerSecureArguments = false Source: $(AZP_75787_ENABLE_NEW_LOGIC_LOG) 2024-07-31T18:48:58.3428160Z Knob: ProcessHandlerTelemetry = true Source: $(AZP_75787_ENABLE_COLLECT) 2024-07-31T18:48:58.3432250Z Knob: ProcessHandlerEnableNewLogic = true Source: $(AZP_75787_ENABLE_NEW_PH_LOGIC) 2024-07-31T18:48:58.3434560Z Knob: IgnoreVSTSTaskLib = true Source: $(AZP_AGENT_IGNORE_VSTSTASKLIB) 2024-07-31T18:48:58.3435620Z Knob: FailJobWhenAgentDies = true Source: $(FAIL_JOB_WHEN_AGENT_DIES) 2024-07-31T18:48:58.3436980Z Knob: CheckForTaskDeprecation = true Source: $(AZP_AGENT_CHECK_FOR_TASK_DEPRECATION) 2024-07-31T18:48:58.3438300Z Knob: MountWorkspace = true Source: $(AZP_AGENT_MOUNT_WORKSPACE) 2024-07-31T18:48:58.3439600Z Knob: LogTaskNameInUserAgent = true Source: $(AZP_AGENT_LOG_TASKNAME_IN_USERAGENT) 2024-07-31T18:48:58.3441340Z Knob: UseFetchFilterInCheckoutTask = true Source: $(AGENT_USE_FETCH_FILTER_IN_CHECKOUT_TASK) 2024-07-31T18:48:58.3443180Z Knob: Rosetta2Warning = true Source: $(ROSETTA2_WARNING) 2024-07-31T18:48:58.3444440Z Finished checking job knob settings. 2024-07-31T18:48:58.7889900Z Plugin: 'Test Result Parser plugin' is running in background. 2024-07-31T18:48:58.7891400Z Plugin: 'Test File Publisher plugin' is running in background. 2024-07-31T18:48:58.7892880Z Start tracking orphan processes. 2024-07-31T18:48:58.8134680Z ##[section]Finishing: Initialize job

[jwodder]

@asmacdo The problem appears to be that you're requesting SC_PAGE_SIZE when it should be SC_PAGESIZE.

[asmacdo]

  memory_total = os.sysconf("SC_PAGESIZE") * os.sysconf("SC_PHYS_PAGES")

E ValueError: unrecognized configuration name

Quick google suggests that SC_PHYS_PAGES doesn't have a simple fix. @jwodder ideas?

[jwodder]

@asmacdo It works on macOS 14.5 with Python 3.12.4:

>>> import os
>>> os.sysconf("SC_PHYS_PAGES")
4194304

What exactly did you find by Googling?

[asmacdo]

Seems like its python version specific. Could you try on 3.8?

https://stackoverflow.com/questions/22102999/get-total-physical-memory-in-python#comment110705631_28161352

I also found a handful of github issues/prs with similar problems, and various fallbacks, ie psutils, sysctl('HW_REALMEM')

[jwodder]

@asmacdo Also works in Python 3.8.19.

[asmacdo]

Huh. 🤔