Open ppurka opened 12 years ago
Description changed:
---
+++
@@ -1,3 +1,8 @@
1. `list_plot3d` plots extra points at the horizontal x-y plane. This seems to stem from the default value of 0.0 that is assigned in the code. See [this post](https://groups.google.com/d/topic/sage-devel/RAdolRdjSTk/discussion).
2. This function also does not take in `color` or `rgbcolor` as keywords. Rest of the plot commands including `plot`, `list_plot`, `plot3d` do take in these keywords.
+
+Attached patch fixes both these problems.
+
+---
+Apply [attachment: trac_12798-fix_extraneous_points_and_kwds_in_list_plot3d.patch](https://github.com/sagemath/sage-prod/files/10655277/trac_12798-fix_extraneous_points_and_kwds_in_list_plot3d.patch.gz) to `devel/sage`.
Author: Punarbasu Purkayastha
Changed keywords from list_plot3d to list_plot3d, sd40.5
I like this idea, but have a couple questions.
color
etc. overriding texture
. Should we be explicit about that? Is it even desirable? (Maybe it is, I just want to ask, as I am not sure.)0.0
was supposed to have done in the past (say, for an empty plot)? I'm not suggesting you know, and I did read your very sensible post, but I'm just curious for any ideas you may have.Replying to @kcrisman:
I like this idea, but have a couple questions.
- The way you have it set up currently has
color
etc. overridingtexture
. Should we be explicit about that? Is it even desirable? (Maybe it is, I just want to ask, as I am not sure.)
Actually, color
and the other keywords will never override texture
. color
will be used only when texture
is not present, and color
is present. As it is setup, if texture
is provided, then the others should be ignored. Actually, maybe we should do this explicitly and remove the other keywords if they are present. Something like
if texture == 'automatic':
if 'color' in kwds:
texture = kwds.pop('color')
if 'rgbcolor' in kwds:
del kwds['rgbcolor']
...
texture = rgbcolor(texture)
if 'color' in kwds:
del kwds['color']
...
Does that sound reasonable?
- Do you have any idea what the original default
0.0
was supposed to have done in the past (say, for an empty plot)? I'm not suggesting you know, and I did read your very sensible post, but I'm just curious for any ideas you may have.
I have no idea why this was done. Was it some bug in matplotlib earlier? Was it just an oversight? For an empty plot, I suppose you would not want there to be plot points at 0.0!
Reviewer: Karl-Dieter Crisman
I like this idea, but have a couple questions.
- The way you have it set up currently has
color
etc. overridingtexture
. Should we be explicit about that? Is it even desirable? (Maybe it is, I just want to ask, as I am not sure.)Actually,
color
and the other keywords will never overridetexture
.color
will be used only whentexture
is not present, andcolor
is present.
Sorry for the inaccuracy, of course that is what I meant.
As it is setup, if
texture
is provided, then the others should be ignored. Actually, maybe we should do this explicitly and remove the other keywords if they are present. Something likeif texture == 'automatic': if 'color' in kwds: texture = kwds.pop('color') if 'rgbcolor' in kwds: del kwds['rgbcolor'] ... texture = rgbcolor(texture) if 'color' in kwds: del kwds['color'] ...
Does that sound reasonable?
Yes, as long as we document it. We do similar things lots of other places.
- Do you have any idea what the original default
0.0
was supposed to have done in the past (say, for an empty plot)? I'm not suggesting you know, and I did read your very sensible post, but I'm just curious for any ideas you may have.I have no idea why this was done. Was it some bug in matplotlib earlier? Was it just an oversight? For an empty plot, I suppose you would not want there to be plot points at 0.0!
Hmm, looking at your plots from the post, I think I know. The idea was probably to have the list_plot
cause a list to be plotted and then have the plot be zero elsewhere. Then there is the weird interpolation thing going on. Anyway, this makes things interpretable, though I'm not sure the original was really that useful, due to the output. I wonder who has used this in the past...
Updated the patch.
Okay, I think this idea is okay. William says to defer to what Mathematica appears to do, since this is where the function was inspired, many years ago.
There will be holes in the surface corresponding to array elements that do not represent explicit height values.
And see here, the second example - clearly not continued beyond the three points given.
But I can't get it to work.
sage: list_plot3d([(0,0,1), (2,3,4)], color='black')
sage: list_plot3d([(0,0,1), (2,3,4)], color='black', rgbcolor='#0f0')
sage: list_plot3d([(0,0,1), (2,3,4)], rgbcolor='#0f0')
sage: list_plot3d([(0,0,1), (2,3,4)], rgbcolor='#fff')
sage: list_plot3d([(0,0,1), (2,3,4), (-1,-1,-1)], rgbcolor='#fff')
sage: list_plot3d([(0,0,1), (2,3,4), (-1,-1,-1)], rgbcolor='#00f')
sage: list_plot3d([(0,0,1), (2,3,4), (-1,-1,-1)], color='black')
None of these really seemed to look like what I wanted, or even sometimes to have anything plot. Am I using this incorrectly?
sage: list_plot3d([(0,0,1), (2,3,4)], color='black')
sage: list_plot3d([(0,0,1), (2,3,4)], color='black', rgbcolor='#0f0')
sage: list_plot3d([(0,0,1), (2,3,4)], rgbcolor='#0f0')
sage: list_plot3d([(0,0,1), (2,3,4)], rgbcolor='#fff')
There seems to be a problem with the colors here. This is a bug that is present even without my patch. It seems like the texture is not passed onto the final function which plots the line.
sage: list_plot3d([(0,0,1), (2,3,4), (-1,-1,-1)], rgbcolor='#fff', num_points=100)
sage: list_plot3d([(0,0,1), (2,3,4), (-1,-1,-1)], rgbcolor='#00f', num_points=100)
sage: list_plot3d([(0,0,1), (2,3,4), (-1,-1,-1)], color='black', num_points=100)
Can you check these with the additional arguments num_points=100
or higher numbers? It seems there are not enough interpolation points and so the plot is either missing or very jagged with a small value of num_points
. With the default value of f.default_value = 0.0
(as it was earlier), you do get a plot but the plot is completely incorrect and useless also.
Apply to devel/sage
Attachment: trac_12798-fix_extraneous_points_and_kwds_in_list_plot3d.patch.gz
Ok. I updated the patch to fix the color in 3D lines.
The other problem with interpolation and num_points
is something that I don't know how to fix. Giving a high value of num_points
gives a very good and accurate plot, but is also very slow. I have added some notes regarding this in the documentation of list_plot3d
.
Great, color works now - nice catch.
It turns out that the documentation for what this function actually does is insanely bad. I'm going to upload a patch clarifying what it actually does, which is not intuitive at all.
As for the issue you are seeing, I think it's very interesting. Notice that the box that comes up does have the right dimensions to incorporate all three points, even when you don't specify num_points
!
Here is what is going on - I've printed out the vals
. All those nan
guys will lead to not much of a plot.
sage: sage: list_plot3d([(0,0,1), (2,3,4), (-1,-1,-1)],interpolation_type='linear')
[[ -1.00000000e+00 nan nan nan nan nan]
[ nan 6.66133815e-16 nan nan nan nan]
[ nan nan nan nan nan nan]
[ nan nan nan 2.00000000e+00 nan nan]
[ nan nan nan nan nan nan]
[ nan nan nan nan nan nan]]
So you pick your poison here. Either you get something that does have the right points, but then also has everything else be extended by zero, or you don't do that, but the algorithm fails in certain cases. This will happen whenever the "box" in question is a lot bigger than the projection of the actual points to the xy-plane.
sage: sage: list_plot3d([(0,0,1), (2,3,4), (-1,-1,-1)]) # bad
sage: sage: list_plot3d([(0,5,1), (5,5,4), (-1,-5,-1)]) # good
I'm good with your work. If you like my changes, sign off as a reviewer and we'll get this in.
Patchbot: Apply trac_12798-fix_extraneous_points_and_kwds_in_list_plot3d.patch and trac_12798-more-doc.patch.
Changed author from Punarbasu Purkayastha to Punarbasu Purkayastha, Karl-Dieter Crisman
Description changed:
---
+++
@@ -2,7 +2,7 @@
2. This function also does not take in `color` or `rgbcolor` as keywords. Rest of the plot commands including `plot`, `list_plot`, `plot3d` do take in these keywords.
-Attached patch fixes both these problems.
+First attached patch fixes both these problems, second one fixes more doc and clarifies output.
---
-Apply [attachment: trac_12798-fix_extraneous_points_and_kwds_in_list_plot3d.patch](https://github.com/sagemath/sage-prod/files/10655277/trac_12798-fix_extraneous_points_and_kwds_in_list_plot3d.patch.gz) to `devel/sage`.
+Apply [attachment: trac_12798-fix_extraneous_points_and_kwds_in_list_plot3d.patch](https://github.com/sagemath/sage-prod/files/10655277/trac_12798-fix_extraneous_points_and_kwds_in_list_plot3d.patch.gz) and [attachment: trac_12798-more-doc.patch](https://github.com/sagemath/sage/files/ticket12798/trac_12798-more-doc.patch.gz) to `devel/sage`.
only added #long time to last example
Changed reviewer from Karl-Dieter Crisman to Karl-Dieter Crisman, Punarbasu Purkayastha
Attachment: trac_12798-more-doc.2.patch.gz
Your patch looks good. The documentation was indeed lacking. I added just one small comment (# long time) in the last example in your patch. Positive review from me.
Description changed:
---
+++
@@ -5,4 +5,4 @@
First attached patch fixes both these problems, second one fixes more doc and clarifies output.
---
-Apply [attachment: trac_12798-fix_extraneous_points_and_kwds_in_list_plot3d.patch](https://github.com/sagemath/sage-prod/files/10655277/trac_12798-fix_extraneous_points_and_kwds_in_list_plot3d.patch.gz) and [attachment: trac_12798-more-doc.patch](https://github.com/sagemath/sage/files/ticket12798/trac_12798-more-doc.patch.gz) to `devel/sage`.
+Apply [attachment: trac_12798-fix_extraneous_points_and_kwds_in_list_plot3d.patch](https://github.com/sagemath/sage-prod/files/10655277/trac_12798-fix_extraneous_points_and_kwds_in_list_plot3d.patch.gz) and [attachment: trac_12798-more-doc.2.patch](https://github.com/sagemath/sage-prod/files/10655278/trac_12798-more-doc.2.patch.gz) to `devel/sage`.
Merged: sage-5.1.beta4
This might have caused #13135.
This might have caused #13135.
I suppose it's possible. I don't understand why
sage: list_plot3d([(0,0,1), (2,3,4), (-1,-1,-1)],num_points=400) # long time
doesn't, then. Maybe the interpolation type? It's only
f.default_value=0.0
that seems at all conceivable as a culprit. Weird.
It seems giving interpolation type as nn for list of 3-tuples (with more than 3 points in the list) is the same as giving no interpolation type.
Changed merged from sage-5.1.beta4 to none
Unmerging due to #13135 (and the potential resource leak).
The resource leak seems to be a general problem with doctesting of 3d plots. This ticket simply exposes the defect.
Replying to @ppurka:
The resource leak seems to be a general problem with doctesting of 3d plots. This ticket simply exposes the defect.
So you think that the massive slowdown is caused by the few added doctests here, not by the changes in code in this ticket? That could be tested.
Replying to @jdemeyer:
Replying to @ppurka:
The resource leak seems to be a general problem with doctesting of 3d plots. This ticket simply exposes the defect.
So you think that the massive slowdown is caused by the few added doctests here, not by the changes in code in this ticket? That could be tested.
Yes. It will be good to test if num_points=400
in the same doctest command without this patch still leads to the massive slowdown. I can't test it since it is apparently only visible on the solaris arch. The command that is desired to be doctested is
sage: list_plot3d([(0,0,1), (2,3,4), (-1,-1,-1)],num_points=400) # long time
There is one workaround (even with the resource leak) and that is to decrease the value of num_points
in the current patch.
Replying to @ppurka:
Yes. It will be good to test if
num_points=400
in the same doctest command without this patch still leads to the massive slowdown.
On Solaris SPARC, sage-5.1.beta0:
sage: time list_plot3d([(0,0,1), (2,3,4), (-1,-1,-1)],num_points=400)
Time: CPU 23.91 s, Wall: 34.13 s
same machine, sage-5.1.beta3 + this patch:
sage: time list_plot3d([(0,0,1), (2,3,4), (-1,-1,-1)],num_points=400)
Time: CPU 39.62 s, Wall: 50.75 s
So, there is a slowdown. But perhaps more importantly, the time to doctest the file list_plot3d.py
increased by much more than this. So I'm guessing with this patch, the time to doctest is much more than the sum of the times of the individual commands.
This is befuddling. The main time consuming doctest that is added is the num_points=400
one. And here are the doctest times. The starting point is sage-5.2.alpha0 with these patches added, on an Intel(R) Core(TM)2 CPU 6400 @ 2.13GHz
ubuntu 12.04.
f.default_value = 0.0
missing from the file (this is the change introduced by the patches in this ticket)....age-5.2.alpha0/devel/sage» ../../sage -b >& /dev/null
...age-5.2.alpha0/devel/sage» ../../sage -t -long sage/plot/plot3d/list_plot3d.py
sage -t -long "devel/sage-main/sage/plot/plot3d/list_plot3d.py"
[13.5 s]
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 13.5 seconds
f.default_value = 0.0
. Note that at this point, the extra examples introduced by the patches in this ticket are still there in the file....age-5.2.alpha0/devel/sage» ../../sage -b >& /dev/null
...age-5.2.alpha0/devel/sage» ../../sage -t -long sage/plot/plot3d/list_plot3d.py
sage -t -long "devel/sage-main/sage/plot/plot3d/list_plot3d.py"
[16.3 s]
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 16.3 seconds
f.default_value = float('inf')
, and otherwise the patches in this ticket are still applied, i.e., the extra examples are still present....age-5.2.alpha0/devel/sage» ../../sage -b >& /dev/null
...age-5.2.alpha0/devel/sage» ../../sage -t -long sage/plot/plot3d/list_plot3d.py
sage -t -long "devel/sage-main/sage/plot/plot3d/list_plot3d.py"
[12.8 s]
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 12.8 seconds
...age-5.2.alpha0/devel/sage» hg qpop -a
popping trac_12798-more-doc.2.patch
popping trac_12798-fix_extraneous_points_and_kwds_in_list_plot3d.patch
popping trac_9774-doctests.patch
popping trac_9774-mathjax-try5.patch
patch queue now empty
...age-5.2.alpha0/devel/sage» ../../sage -b >& /dev/null
...age-5.2.alpha0/devel/sage» ../../sage -t -long sage/plot/plot3d/list_plot3d.py
sage -t -long "devel/sage-main/sage/plot/plot3d/list_plot3d.py"
[6.3 s]
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 6.3 seconds
Note that correct plots are generated only by 1. and 3. At least on this machine, both 1. and 3. are faster than the code already present in vanilla sage-5.2.alpha0. It is only the example with num_points=400
that introduces the slowdown.
So, I am attaching an updated patch which reduces the num_points
. Here is the time taken for this new one. Notice that the time taken is lower than the vanilla sage-5.2.alpha0, even though the number of examples doctested has increased.
...age-5.2.alpha0/devel/sage» hg qpush trac_12798-more-doc.2.patch
applying trac_9774-mathjax-try5.patch
applying trac_9774-doctests.patch
applying trac_12798-fix_extraneous_points_and_kwds_in_list_plot3d.patch
applying trac_12798-more-doc.2.patch
now at: trac_12798-more-doc.2.patch
...age-5.2.alpha0/devel/sage» hg qref
...age-5.2.alpha0/devel/sage» ../../sage -b >& /dev/null
...age-5.2.alpha0/devel/sage» ../../sage -t -long sage/plot/plot3d/list_plot3d.py
sage -t -long "devel/sage-main/sage/plot/plot3d/list_plot3d.py"
[4.4 s]
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 4.4 seconds
Attachment: trac_12798-more-doc.3.patch.gz
apply to devel/sage
Description changed:
---
+++
@@ -5,4 +5,4 @@
First attached patch fixes both these problems, second one fixes more doc and clarifies output.
---
-Apply [attachment: trac_12798-fix_extraneous_points_and_kwds_in_list_plot3d.patch](https://github.com/sagemath/sage-prod/files/10655277/trac_12798-fix_extraneous_points_and_kwds_in_list_plot3d.patch.gz) and [attachment: trac_12798-more-doc.2.patch](https://github.com/sagemath/sage-prod/files/10655278/trac_12798-more-doc.2.patch.gz) to `devel/sage`.
+Apply [attachment: trac_12798-fix_extraneous_points_and_kwds_in_list_plot3d.patch](https://github.com/sagemath/sage-prod/files/10655277/trac_12798-fix_extraneous_points_and_kwds_in_list_plot3d.patch.gz) and [attachment: trac_12798-more-doc.3.patch](https://github.com/sagemath/sage-prod/files/10655279/trac_12798-more-doc.3.patch.gz) to `devel/sage`.
Replying to @jdemeyer:
Replying to @ppurka:
This is befuddling.
Thank you, wiktionary :-)
It was befuddling when I started writing out that comment. By the time (about 1 hour) I reached the end of that comment, I had changed my opinion and things became less muddled.
It's less bad, but far from good:
buildbot@mark:~/mark/sage-5.1.beta3$ ./sage -t devel/sage/sage/plot/plot3d/list_plot3d.py
sage -t "devel/sage/sage/plot/plot3d/list_plot3d.py"
[448.8 s]
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 449.2 seconds
Without the patch, it is something like 30 seconds.
I'm going to split the patches into a "change functionality" and "add doctests" part.
Attachment: 12798_change.patch.gz
Attachment: 12798_doc.patch.gz
I can absolutely confirm that the slowdown on Solaris SPARC is caused by attachment: 12798_change.patch, not by the added doctests.
Replying to @jdemeyer:
I can absolutely confirm that the slowdown on Solaris SPARC is caused by attachment: 12798_change.patch, not by the added doctests.
It is confusing alright. I am getting speedups on intel but you are getting slowdowns on sparc. Sorry for bothering you so much. Can you do the following test since you have access to the sparc:
f.default_value = float('inf')
This is the only alternative solution I have for the incorrect plots otherwise generated.
I disagree with the analysis as 'needs work'. I understand that this is annoying, but ppurka is right that this makes something mathematically correct, or at least more so. " plots extraneous points at z=0" - we shouldn't be plotting extraneous points. And how many people are going to be plotting 3d list plots on Sparc? Very unlikely.
There probably is some Numpy internal that causes the interpolations to be very slow without a default value - at least, I wouldn't be surprised. It would be worth having someone who understands numpy (not me, unfortunately) generate a "native example".
That said, it seems like there must be some compromise that we can reach here for this ticket. Is it possible to add even more # long time
flags to get it under a minute on mark? Also note that if it's 30 seconds without the patch, that's five times as long as the six seconds above - so mark would not exactly be a "typical" machine to be talking about timings on. That's sort of like on my old PPC machine - I'm happy as long as some SAGE_TIMEOUT_LONG
value works :)
Replying to @ppurka:
f.default_value = float('inf')
I'm afraid this doesn't really help (it might be just a little bit faster, but still massively slower than before).
Replying to @kcrisman:
Also note that if it's 30 seconds without the patch, that's five times as long as the six seconds above - so mark would not exactly be a "typical" machine to be talking about timings on.
That wasn't my point. My point was that the patch on this ticket multiplies the time (without even adding new doctests!) by a factor 12.
Also note that if it's 30 seconds without the patch, that's five times as long as the six seconds above - so mark would not exactly be a "typical" machine to be talking about timings on.
That wasn't my point. My point was that the patch on this ticket multiplies the time (without even adding new doctests!) by a factor 12.
I understand that. I suppose we have somewhat orthogonal points. My point is that we have often, in the past, lived with slowdowns if it meant correctness could be achieved. (Though in this case there may have been some good reason to have the zero points, but it was never documented so we don't know why.) Especially since there is a much smaller chance that anyone will ever use this function on a Sparc machine, and it's not actually breaking anything, it seems to be less of an issue to me. I'm really not convinced why it's important to have every file test fully in less than six minutes anyway; it seems quite arbitrary. But that is just my opinion :)
Unrelated, but while looking in this file I saw some confusing comments concerning the Delaunay call, I fixed them at #13167. Could any of you review that?
Done.
Replying to @jdemeyer:
Replying to @kcrisman:
Also note that if it's 30 seconds without the patch, that's five times as long as the six seconds above - so mark would not exactly be a "typical" machine to be talking about timings on.
That wasn't my point. My point was that the patch on this ticket multiplies the time (without even adding new doctests!) by a factor 12.
That's the weird part. One point of comment:23 was to establish that there is a speedup on intel cpu, though not by a large margin.
How does a doctest "plot" an object? I mean, what really happens when doctesting
sage: plot(foo)
Because the slow part is the actual plotting, not the computation of the object.
That is a good question. You can check it out in the code, there is some stuff in graphics.py or plot.py with DOCTEST_MODE
. Computing the plot objects is very fast, you are right. And plot3d of course is even different from that.
I guess the method sage.plot.plot3d.parametric_surface.ParametricSurface.triangulate()
should be fixed not to add any faces of which the coordinates are not all finite numbers. However, I'm having a hard time understanding that method.
Because now we are sending a list of faces to be plotted full of NaN
s:
sage: G = list_plot3d([(0,0,1), (2,3,4), (-1,-1,-1)],num_points=100)
sage: G.triangulate()
sage: G.face_list()
[...[(1.9696971120698907, 2.71717269474511, nan), (1.9696971120698907, 2.757576737669652, nan), (2.000000142649374, 2.757576737669652, nan), (2.000000142649374, 2.71717269474511, nan)], [(1.9696971120698907, 2.757576737669652, nan), (1.9696971120698907, 2.7979807805941936, nan), (2.000000142649374, 2.7979807805941936, nan), (2.000000142649374, 2.757576737669652, nan)], [(1.9696971120698907, 2.7979807805941936, nan), (1.9696971120698907, 2.8383848235187354, nan), (2.000000142649374, 2.8383848235187354, nan), (2.000000142649374, 2.7979807805941936, nan)], [(1.9696971120698907, 2.8383848235187354, nan), (1.9696971120698907, 2.8787888664432773, nan), (2.000000142649374, 2.8787888664432773, nan), (2.000000142649374, 2.8383848235187354, nan)], [(1.9696971120698907, 2.8787888664432773, nan), (1.9696971120698907, 2.919192909367819, nan), (2.000000142649374, 2.919192909367819, nan), (2.000000142649374, 2.8787888664432773, nan)], [(1.9696971120698907, 2.919192909367819, nan), (1.9696971120698907, 2.959596952292361, 3.9494949494949494), (2.000000142649374, 2.959596952292361, nan), (2.000000142649374, 2.919192909367819, nan)], [(1.9696971120698907, 2.959596952292361, 3.9494949494949494), (1.9696971120698907, 3.000000995216903, nan), (2.000000142649374, 3.000000995216903, nan), (2.000000142649374, 2.959596952292361, nan)]]
I think this is a true bug (and this might even have been the reason for the old default_value
).
list_plot3d
plots extra points at the horizontal x-y plane. This seems to stem from the default value of 0.0 that is assigned in the code. See this post.This function also does not take in
color
orrgbcolor
as keywords. Rest of the plot commands includingplot
,list_plot
,plot3d
do take in these keywords.First attached patch fixes both these problems, second one fixes more doc and clarifies output.
Apply attachment: trac_12798-fix_extraneous_points_and_kwds_in_list_plot3d.patch, attachment: trac_12798-more-doc.3.patch and attachment: trac_12798-dont_pass_nan_to_ParametricSurface.patch to
devel/sage
.CC: @kcrisman
Component: graphics
Keywords: list_plot3d, sd40.5
Author: Punarbasu Purkayastha, Karl-Dieter Crisman
Reviewer: Karl-Dieter Crisman, Punarbasu Purkayastha, Jeroen Demeyer
Issue created by migration from https://trac.sagemath.org/ticket/12798