Parallel incompatible with new Mac systems?

[ojreuter]

Preliminaries

Before submitting an issue, please check (with x in brackets) that you:

• [ X] Are using the newest release (see here for latest release version number).
• [X ] Have checked that the examples in the help work.
• [ X] Have read the help (HTML version) and the gallery of examples.
• [ X] Have checked that there is not already an existing issues for what you are reporting.

Expected behavior and actual behavior

No parallel commands work, including those using system data in the help file. Am wondering if parallel is incompatible with my system.

Steps to reproduce the problem

Here is all the code from install to command:

net install parallel, from(https://raw.github.com/gvegayon/parallel/master/) replace
mata mata mlib index
sysuse bplong.dta
parallel initialize 4, statapath(/Applications/Stata/StataBE.app/Contents/MacOS/StataBE) 
*Note that parallel was unable to find executable folder on its own and required specification of statapath
sort patient
parallel, by(patient): by patient: egen max_bp = max(bp)

The following error is given:
parallel_write_do():  3001  expected 2 to 12 arguments but received 14
                 <istmt>:     -  function returned error

This same error is returned for all commands, bootstraps or simulations I try in the parallel environment.

System information

• Stata BE Version 18
• Apple M3 Pro chip with 12‑core CPU, 18‑core GPU, 16‑core Neural Engine 18GB unified memory
• Mac OS Sonoma Version 14.4
• Parallel version: Latest

Output from creturn list:

. creturn list

System values

----------------------------------------------------------------------------------------------------------------------------------------------------
    c(current_date) = "27 Aug 2024"
    c(current_time) = "20:05:06"
       c(rmsg_time) = 0                          (seconds, from set rmsg)
----------------------------------------------------------------------------------------------------------------------------------------------------
   c(stata_version) = 18
         c(version) = 18                         (version)
     c(userversion) = 18                         (version)
  c(dyndoc_version) = 2                          (dyndoc)
----------------------------------------------------------------------------------------------------------------------------------------------------
       c(born_date) = "07 Aug 2024"
         c(edition) = "BE"
    c(edition_real) = "BE"
             c(bit) = 64
              c(SE) = 0
              c(MP) = 0
      c(processors) = 1                          (Stata/MP, set processors)
  c(processors_lic) = 1
 c(processors_mach) = .
  c(processors_max) = 1
            c(mode) = ""
         c(console) = ""
----------------------------------------------------------------------------------------------------------------------------------------------------
              c(os) = "MacOSX"
           c(osdtl) = "14.4.0"
        c(hostname) = "Johns-MBP.attloca.."
    c(machine_type) = "Mac (Apple Silicon)"
       c(byteorder) = "lohi"
        c(username) = "john"
----------------------------------------------------------------------------------------------------------------------------------------------------

Directories and paths

----------------------------------------------------------------------------------------------------------------------------------------------------
    c(sysdir_stata) = "/Applications/Sta.."      (sysdir)
     c(sysdir_base) = "/Applications/Sta.."      (sysdir)
     c(sysdir_site) = "/Applications/Sta.."      (sysdir)
     c(sysdir_plus) = "/Users/john/Libra.."      (sysdir)
 c(sysdir_personal) = "/Users/john/Docum.."      (sysdir)
 c(sysdir_oldplace) = "~/ado/"                   (sysdir)
          c(tmpdir) = "/var/folders/l8/m.."
----------------------------------------------------------------------------------------------------------------------------------------------------
         c(adopath) = "BASE;SITE;.;PERSO.."      (adopath)
             c(pwd) = "/Users/john/Downl.."      (cd)
          c(dirsep) = "/"
----------------------------------------------------------------------------------------------------------------------------------------------------

System limits

----------------------------------------------------------------------------------------------------------------------------------------------------
    c(max_N_theory) = 2147483620
    c(max_k_theory) = 2048
c(max_width_theory) = 1048576
----------------------------------------------------------------------------------------------------------------------------------------------------
      c(max_matdim) = 800
----------------------------------------------------------------------------------------------------------------------------------------------------
    c(max_it_cvars) = 64
    c(max_it_fvars) = 8
----------------------------------------------------------------------------------------------------------------------------------------------------
    c(max_macrolen) = 264392
        c(macrolen) = 264392
         c(charlen) = 67783
      c(max_cmdlen) = 264408
          c(cmdlen) = 264408
     c(namelenbyte) = 128
     c(namelenchar) = 32
           c(eqlen) = 1337
----------------------------------------------------------------------------------------------------------------------------------------------------

Numerical and string limits

----------------------------------------------------------------------------------------------------------------------------------------------------
       c(mindouble) = -8.9884656743e+307
       c(maxdouble) = 8.9884656743e+307
       c(epsdouble) = 2.22044604925e-16
  c(smallestdouble) = 2.2250738585e-308
----------------------------------------------------------------------------------------------------------------------------------------------------
        c(minfloat) = -1.70141173319e+38
        c(maxfloat) = 1.70141173319e+38
        c(epsfloat) = 1.19209289551e-07
----------------------------------------------------------------------------------------------------------------------------------------------------
         c(minlong) = -2147483647
         c(maxlong) = 2147483620
----------------------------------------------------------------------------------------------------------------------------------------------------
          c(minint) = -32767
          c(maxint) = 32740
----------------------------------------------------------------------------------------------------------------------------------------------------
         c(minbyte) = -127
         c(maxbyte) = 100
----------------------------------------------------------------------------------------------------------------------------------------------------
    c(maxstrvarlen) = 2045
   c(maxstrlvarlen) = 2000000000
    c(maxvlabellen) = 32000
----------------------------------------------------------------------------------------------------------------------------------------------------

Current dataset

----------------------------------------------------------------------------------------------------------------------------------------------------
           c(frame) = "default"
               c(N) = 240
               c(k) = 5
           c(width) = 7
         c(changed) = 0
        c(filename) = "/Applications/Sta.."
        c(filedate) = " 1 May 2022 11:28"
----------------------------------------------------------------------------------------------------------------------------------------------------

Memory settings

----------------------------------------------------------------------------------------------------------------------------------------------------
          c(memory) = 33554432
          c(maxvar) = 2048
        c(niceness) = 5                          (set niceness)
      c(min_memory) = 0                          (set min_memory)
      c(max_memory) = .                          (set max_memory)
     c(segmentsize) = 33554432                   (set segmentsize)
         c(adosize) = 1000                       (set adosize)
----------------------------------------------------------------------------------------------------------------------------------------------------

Output settings

----------------------------------------------------------------------------------------------------------------------------------------------------
            c(more) = "off"                      (set more)
            c(rmsg) = "off"                      (set rmsg)
              c(dp) = "period"                   (set dp)
        c(linesize) = 152                        (set linesize)
        c(pagesize) = 37                         (set pagesize)
         c(logtype) = "smcl"                     (set logtype)
          c(logmsg) = "on"                       (set logmsg)
         c(noisily) = 1
      c(notifyuser) = "on"                       (set notifyuser)
         c(playsnd) = "off"                      (set playsnd)
  c(include_bitmap) = "on"                       (set include_bitmap)
----------------------------------------------------------------------------------------------------------------------------------------------------
         c(iterlog) = "on"                       (set iterlog)
----------------------------------------------------------------------------------------------------------------------------------------------------
           c(level) = 95                         (set level)
          c(clevel) = 95                         (set clevel)
----------------------------------------------------------------------------------------------------------------------------------------------------
  c(showbaselevels) = ""                         (set showbaselevels)
  c(showemptycells) = ""                         (set showemptycells)
     c(showomitted) = ""                         (set showomitted)
         c(fvlabel) = "on"                       (set fvlabel)
          c(fvwrap) = 1                          (set fvwrap)
        c(fvwrapon) = "word"                     (set fvwrapon)
        c(lstretch) = ""                         (set lstretch)
----------------------------------------------------------------------------------------------------------------------------------------------------
         c(cformat) = ""                         (set cformat)
         c(sformat) = ""                         (set sformat)
         c(pformat) = ""                         (set pformat)
----------------------------------------------------------------------------------------------------------------------------------------------------
  c(coeftabresults) = "on"                       (set coeftabresults)
            c(dots) = "on"                       (set dots)
----------------------------------------------------------------------------------------------------------------------------------------------------
   c(collect_label) = "default"                  (set collect_label)
   c(collect_style) = "default"                  (set collect_style)
     c(table_style) = "table"                    (set table_style)
    c(etable_style) = "etable"                   (set etable_style)
    c(dtable_style) = "dtable"                   (set dtable_style)
    c(collect_warn) = "on"                       (set collect_warn)
----------------------------------------------------------------------------------------------------------------------------------------------------

Interface settings

----------------------------------------------------------------------------------------------------------------------------------------------------
      c(reventries) = 5000                       (set reventries)
     c(revkeyboard) = "on"                       (set revkeyboard)
     c(varkeyboard) = "on"                       (set varkeyboard)
     c(smoothfonts) = "on"                       (set smoothfonts)
         c(linegap) = 1                          (set linegap)
   c(scrollbufsize) = 204800                     (set scrollbufsize)
           c(maxdb) = 50                         (set maxdb)
----------------------------------------------------------------------------------------------------------------------------------------------------

Graphics settings

----------------------------------------------------------------------------------------------------------------------------------------------------
        c(graphics) = "on"                       (set graphics)
          c(scheme) = "stcolor"                  (set scheme)
      c(printcolor) = "asis"                     (set printcolor)
       c(copycolor) = "asis"                     (set copycolor)
   c(maxbezierpath) = 0                          (set maxbezierpath)
   c(min_graphsize) = 1                          (region_options)
   c(max_graphsize) = 100                        (region_options)
----------------------------------------------------------------------------------------------------------------------------------------------------

Network settings

----------------------------------------------------------------------------------------------------------------------------------------------------
       c(httpproxy) = "off"                      (set httpproxy)
   c(httpproxyhost) = ""                         (set httpproxyhost)
   c(httpproxyport) = 80                         (set httpproxyport)
----------------------------------------------------------------------------------------------------------------------------------------------------
   c(httpproxyauth) = "off"                      (set httpproxyauth)
   c(httpproxyuser) = ""                         (set httpproxyuser)
     c(httpproxypw) = ""                         (set httpproxypw)
----------------------------------------------------------------------------------------------------------------------------------------------------

Update settings

----------------------------------------------------------------------------------------------------------------------------------------------------
    c(update_query) = "on"                       (set update_query)
 c(update_interval) = 7                          (set update_interval)
   c(update_prompt) = "on"                       (set update_prompt)
----------------------------------------------------------------------------------------------------------------------------------------------------

Trace (program debugging) settings

----------------------------------------------------------------------------------------------------------------------------------------------------
           c(trace) = "off"                      (set trace)
      c(tracedepth) = 32000                      (set tracedepth)
        c(tracesep) = "on"                       (set tracesep)
     c(traceindent) = "on"                       (set traceindent)
     c(traceexpand) = "on"                       (set traceexpand)
     c(tracenumber) = "off"                      (set tracenumber)
     c(tracehilite) = ""                         (set tracehilite)
----------------------------------------------------------------------------------------------------------------------------------------------------

Mata settings

----------------------------------------------------------------------------------------------------------------------------------------------------
      c(matastrict) = "off"                      (set matastrict)
        c(matalnum) = "off"                      (set matalnum)
    c(mataoptimize) = "on"                       (set mataoptimize)
       c(matafavor) = "space"                    (set matafavor)
       c(matacache) = 2000                       (set matacache)
        c(matalibs) = "lmatabase;lmatamc.."      (set matalibs)
     c(matamofirst) = "off"                      (set matamofirst)
    c(matasolvetol) = .                          (set matasolvetol)
----------------------------------------------------------------------------------------------------------------------------------------------------

Java settings

----------------------------------------------------------------------------------------------------------------------------------------------------
    c(java_heapmax) = "4096m"                    (set java_heapmax)
       c(java_home) = "/Applications/Sta.."      (set java_home)
----------------------------------------------------------------------------------------------------------------------------------------------------

LAPACK settings

----------------------------------------------------------------------------------------------------------------------------------------------------
      c(lapack_mkl) = "off"                      (set lapack_mkl)
----------------------------------------------------------------------------------------------------------------------------------------------------

putdocx settings

----------------------------------------------------------------------------------------------------------------------------------------------------
  c(docx_hardbreak) = "off"                      (set docx_hardbreak)
   c(docx_paramode) = "off"                      (set docx_paramode)
   c(docx_maxtable) = 500                        (set docx_maxtable)
----------------------------------------------------------------------------------------------------------------------------------------------------

putpdf settings

----------------------------------------------------------------------------------------------------------------------------------------------------
    c(pdf_maxtable) = 500                        (set pdf_maxtable)
----------------------------------------------------------------------------------------------------------------------------------------------------

Python settings

----------------------------------------------------------------------------------------------------------------------------------------------------
     c(python_exec) = ""                         (set python_exec)
 c(python_userpath) = ""                         (set python_userpath)
----------------------------------------------------------------------------------------------------------------------------------------------------

RNG settings

----------------------------------------------------------------------------------------------------------------------------------------------------
             c(rng) = "default"                  (set rng)
     c(rng_current) = "mt64"
        c(rngstate) = "XAA00000000000000.."      (set rngstate)
   c(rngseed_mt64s) = 123456789
       c(rngstream) = 1                          (set rngstream)
----------------------------------------------------------------------------------------------------------------------------------------------------

sort settings

----------------------------------------------------------------------------------------------------------------------------------------------------
      c(sortmethod) = "default"                  (set sortmethod)
    c(sort_current) = "fsort"
    c(sortrngstate) = "1060756841XZA1122.."      (set sortrngstate)
----------------------------------------------------------------------------------------------------------------------------------------------------

Unicode settings

----------------------------------------------------------------------------------------------------------------------------------------------------
       c(locale_ui) = "en_US"                    (set locale_ui)
c(locale_functions) = "en_US"                    (set locale_functions)
  c(locale_icudflt) = "en_US"                    (unicode locale)
----------------------------------------------------------------------------------------------------------------------------------------------------

Other settings

----------------------------------------------------------------------------------------------------------------------------------------------------
            c(type) = "float"                    (set type)
         c(maxiter) = 300                        (set maxiter)
   c(searchdefault) = "all"                      (set searchdefault)
       c(varabbrev) = "on"                       (set varabbrev)
      c(emptycells) = "keep"                     (set emptycells)
         c(fvtrack) = "term"                     (set fvtrack)
          c(fvbase) = "on"                       (set fvbase)
         c(odbcmgr) = "iodbc"                    (set odbcmgr)
      c(odbcdriver) = "unicode"                  (set odbcdriver)
         c(fredkey) = ""                         (set fredkey)
  c(collect_double) = "on"                       (set collect_double)
   c(dtascomplevel) = 1                          (set dtascomplevel)
   c(reshape_favor) = "memory"                   (set reshape_favor)
    c(doeditbackup) = "on"                       (set doeditbackup)
----------------------------------------------------------------------------------------------------------------------------------------------------

Other system values

----------------------------------------------------------------------------------------------------------------------------------------------------
              c(pi) = 3.141592653589793
           c(alpha) = "a b c d e f g h i.."
           c(ALPHA) = "A B C D E F G H I.."
            c(Mons) = "Jan Feb Mar Apr M.."
          c(Months) = "January February .."
           c(Wdays) = "Sun Mon Tue Wed T.."
        c(Weekdays) = "Sunday Monday Tue.."
           c(obs_t) = "int"
              c(rc) = 0                          (capture)

[bquistorff]

Have you tried it without manually specifying the statapath() in parallel initialize? If that doesn't work, could you report the full value of c(sysdir_stata)?

[ojreuter]

Thanks for your reply! I have tried it without specifying statapath and I receive the error below.
parallel initialize 4 N Child processes: 4 Can not set Stata directory, try using -statapath()- option r(601);

I can force it to select the directory, as below, but then I receive the same error referenced above. . parallel initialize 4, f N Child processes: 4 Stata dir: /Applications/Stata/Stata.app/Contents/MacOS/stata

Here is c(sysdir_stata) display c(sysdir_stata) /Applications/Stata/

Get the same error when I use that abbeviated file path in parallel initialize.

Thanks again for your help!

[bquistorff]

Looks like you have an old lparallel.mlib, maybe from SSC. Try to completely remove all versions and reinstall. You can use which lparallel.mlib to see what it's reference.

[ojreuter]

Thanks, I uninstalled everything, using both -ado uninstall- and manually in the folders (for whatever reason, ado uininstall does not uninstall all the parallel package components). Unfortunately, I get the same error. I tried both the stable version and the latest release version from github (fully uninstalling each after each failure). Interestingly, I do NOT get the error when I install and use the SSC version, but it freezes and never completes the task (even the simple one from the help file above) Thanks for your help again, but I may need to stick with R for this.

[bquistorff]

Sorry, yeah, Stata can get confused when you install from multiple sources and then won't uninstall everything. Also with windows, you might need to restart to clear dlls from memory.