Applying concat to long lists fails

[rtoy]

Imported from SourceForge on 2024-07-08 01:47:33 Created by fiorent on 2019-07-17 17:33:27 Original: https://sourceforge.net/p/maxima/bugs/3568

---

Applying concat (or sconcat) to long list fails, e.g. issuing

s: apply(concat, makelist(i, i, 1000000))$

gives

Maxima encountered a Lisp error:
 Control stack exhausted (no more space for function call frames).
This is probably due to heavily nested or infinitely recursive function
calls, or a tail call that SBCL cannot or has not optimized away.

on wxmaxima for Windows (64 bits)

(%i4)   build_info();
(%o4)   build_info(version="5.43.0",timestamp="2019-06-01 18:55:31",host="x86_64-w64-mingw32",
lisp_name="SBCL",lisp_version="1.4.14",maxima_userdir="C:/Users/Giuseppe/maxima",maxima_tempdir
="C:/Users/Giuseppe/AppData/Local/Temp",maxima_objdir=
"C:/Users/Giuseppe/maxima/binary/5_43_0/sbcl/1_4_14",maxima_frontend="wxMaxima",
maxima_frontend_version="19.05.7")

[rtoy]

Imported from SourceForge on 2024-07-08 01:47:34 Created by l_butler on 2019-07-17 20:24:43 Original: https://sourceforge.net/p/maxima/bugs/3568/#b82a

---

• Description has changed:

Diff:

--- old
+++ new
@@ -1,9 +1,11 @@
 Applying concat (or sconcat) to long list fails, e.g. issuing

+~~~~
 s: apply(concat, makelist(i, i, 1000000))$
-
+~~~~
 gives 

+~~~~
 Maxima encountered a Lisp error:
  Control stack exhausted (no more space for function call frames).
 This is probably due to heavily nested or infinitely recursive function
@@ -17,3 +19,4 @@
 ="C:/Users/Giuseppe/AppData/Local/Temp",maxima_objdir=
 "C:/Users/Giuseppe/maxima/binary/5_43_0/sbcl/1_4_14",maxima_frontend="wxMaxima",
 maxima_frontend_version="19.05.7")
+~~~~

[rtoy]

Imported from SourceForge on 2024-07-08 01:47:38 Created by l_butler on 2019-07-17 20:24:43 Original: https://sourceforge.net/p/maxima/bugs/3568/#a5d2

---

This is to be expected, I suppose. Maybe concat and sconcat should be using reduce.

Anyhow, Maxima's lreduce will remedy the problem to the extent possible.

Note that on my oldish laptop, it takes Maxima 59sec just to create your list. For fun, I let Maxima chew on lreduce(concat,s) for over an hour before deciding my office is already hot enough...when I reduce the length of the list by 2 orders of magnitude, apply(concat,s) still errors out while lreduce(concat,s) completes in about 8sec.

[rtoy]

Imported from SourceForge on 2024-07-08 01:47:42 Created by robert_dodier on 2019-09-01 05:51:48 Original: https://sourceforge.net/p/maxima/bugs/3568/#22c6

---

• labels: --> apply
• status: open --> wont-fix

[rtoy]

Imported from SourceForge on 2024-07-08 01:47:45 Created by robert_dodier on 2019-09-01 05:51:49 Original: https://sourceforge.net/p/maxima/bugs/3568/#997c

---

The problem here is that SBCL has a limitation on the number of arguments which can be passed to a function, and the limit is less than what was attempted (1000000 arguments). There isn't anything specific about concat here; any other function with a Lisp implementation would run into the same problem.

I'm closing this item as "won't fix" because this isn't a bug. Every Lisp implementation will have a limit, and Maxima will bump into that limit, given long enough argument lists. Maxima doesn't have its own stated limit on the number of arguments, so it defaults to whatever the Lisp implementation imposes. That seems OK.

We could give Maxima its own arguments limit, which would probably require some additional work to ensure that all Lisp implementations are covered. It occurs to me that it would also require additional declarations for functions we want to cover; specifically I think that grouping arguments so that the number is less than the Lisp limit requires that the function be known to be associative.

If anyone would like to go down that road, my advice is to create a feature request (https://sourceforge.net/p/maxima/feature-requests/) and/or raise the issue on maxima-discuss. It seems like an interesting and useful project, so I would be interested to see how it could be done.

I guess if nothing else is done, at least Maxima could give a more informative error message about the number of arguments. I'll take a look at that.

[rtoy]

Imported from SourceForge on 2024-07-08 01:47:48 Created by fiorent on 2019-09-06 23:10:27 Original: https://sourceforge.net/p/maxima/bugs/3568/#0c29

---

I always thought of Maxima as a high-level mathematical system whose only limitations were (or should have been) the available memory in space and user's patience in time. But, according to your answer, I should re-think it as a convenient way to pass parameters to the underlying Lisp. In this way, no multi-precision system would have been written since, sooner or later, it would have clashed with the limits of the processor on which it was implemented. In my opinion, Maxima should know the limitations of the underlying Lisp and circumvent them with a careful coding (in my case it suffices to split the range into slices and concatenate the results). Am I wrong?

[rtoy]

Imported from SourceForge on 2024-07-08 01:47:52 Created by peterpall on 2019-09-07 06:07:31 Original: https://sourceforge.net/p/maxima/bugs/3568/#7cb4

---

I think at this point Roberts Question is relevant: As you have said Maxima is a powerful mathematical system. And If you stay in that domain the limit tends to be the amount of RAM you own. .. What exactly do you want to convert an extremely long list to a string for? If we know the use case we perhaps find a canonical way to do that that isn't affected by the Lisp's limit: Most operations are. On the other hand: rolling our own method of doing things instead using the means the programming language provides tends to make things complicated and slow. But if you provide an use-case that shows that making things complicated and slow is justified perhaps someone will be willing to do all the work needed to raise this limit that as long l am involved in this project never has been reported to have been found. ...normally I am the one who uses maxima for weird things I an convinced it was not designed for and oftentimes am surprised that it works for me, nonetheless.

Having said that: if you give maxima the command-line parameter

  -l clisp

Does this solve your problem?

[rtoy]

Imported from SourceForge on 2024-07-08 01:47:55 Created by tomio-arisaka on 2019-09-07 08:45:14 Original: https://sourceforge.net/p/maxima/bugs/3568/#27e8

---

I don't think that the limitation on the number of arguments is less than 10^6. Because the value of call-arguments-limit is 4611686018427387903 on SBCL.

For example:

$ rmaxima --quiet -X "--control-stack-size 32"
(%i1) build_info();
(%o1) 
Maxima version: "5.43.0"
Maxima build date: "2019-06-05 13:14:43"
Host type: "x86_64-apple-darwin13.4.0"
Lisp implementation type: "SBCL"
Lisp implementation version: "1.5.3"
(%i2) to_lisp();

Type (to-maxima) to restart, ($quit) to quit Maxima.

MAXIMA> call-arguments-limit

4611686018427387903
MAXIMA> 
(progn (msetq $str (apply #'concatenate (cons 'string (cdr #$makelist(string(i),i,10^6)$))))
       (length $str))

5888896
MAXIMA> 

The value of call-arguments-limit depends on an implementation of lisp.

[rtoy]

Imported from SourceForge on 2024-07-08 01:47:59 Created by robert_dodier on 2019-09-22 00:16:54 Original: https://sourceforge.net/p/maxima/bugs/3568/#7ed8

---

In reference to the comment by Tomio Arisaka, it appears the limitation, for SBCL, which is shown by the original example involving concat, is not CALL-ARGUMENTS-LIMIT, but something else (maybe memory needed for the stack frame).

That behavior makes is very difficult to predict whether a call will succeed or fail, in SBCL, as it is not just a simple matter of ensuring that the number of arguments is less than CALL-ARGUMENTS-LIMIT.

I don't know if any other Lisp implementation has a similar behavior, I didn't investigate.