search

dereneaton / ipyrad

Interactive assembly and analysis of RAD-seq data sets
http://ipyrad.readthedocs.io
GNU General Public License v3.0
72 stars 40 forks source link

AttributeError: module 'numpy' has no attribute 'asscalar' #498

Open LalitaSimpson opened 1 year ago

LalitaSimpson commented 1 year ago

Hi ipyrad team, thanks for all the tools! I am trying out the PCA cookbook but hit this error at the 'Run PCA and plot results' step: AttributeError Traceback (most recent call last) Cell In[34], line 2 1 # plot PC axes 0 and 2 ----> 2 pca.draw(0, 2);

File ~/miniconda3/lib/python3.10/site-packages/ipyrad/analysis/pca.py:405, in PCA.draw(self, ax0, ax1, cycle, colors, opacity, shapes, size, legend, label, outfile, imap, width, height, axes, kwargs) 385 def draw( 386 self, 387 ax0=0, (...) 400 axes=None, 401 kwargs): 402 """ 403 Draw a scatterplot for data along two PC axes. 404 """ --> 405 self.drawing = Drawing( 406 self, ax0, ax1, cycle, colors, opacity, shapes, size, legend, 407 label, outfile, imap, width, height, axes, 408 **kwargs) 409 return self.drawing.canvas, self.drawing.axes

File ~/miniconda3/lib/python3.10/site-packages/ipyrad/analysis/pca.py:648, in Drawing.init(self, pcatool, ax0, ax1, cycle, colors, opacity, shapes, size, legend, label, outfile, imap, width, height, axes, **kwargs) 646 self.rstyles = {} 647 self.pstyles = {} --> 648 self._get_marker_styles() 649 self._assign_styles_to_marks() 650 self._draw_markers()

File ~/miniconda3/lib/python3.10/site-packages/ipyrad/analysis/pca.py:773, in Drawing._get_marker_styles(self) 770 # get color list repeating in cycles of cycle 771 if not self.colors: 772 self.colors = itertools.cycle( --> 773 toyplot.color.broadcast( 774 toyplot.color.brewer.map("Spectral"), shape=self.cycle, 775 ) 776 ) 777 else: 778 self.colors = itertools.cycle(self.colors)

File ~/miniconda3/lib/python3.10/site-packages/toyplot/color.py:205, in broadcast(colors, shape, default) 202 colors, colormap = colors 204 # Next, convert the supplied colors into a toyplot color array. --> 205 if isinstance(colors, collections.Sequence): 206 colors = numpy.array(colors) 208 if isinstance(colors, numpy.ndarray):

AttributeError: module 'collections' has no attribute 'Sequence'

AttributeError Traceback (most recent call last) File ~/miniconda3/lib/python3.10/site-packages/IPython/core/formatters.py:342, in BaseFormatter.call(self, obj) 340 method = get_real_method(obj, self.print_method) 341 if method is not None: --> 342 return method() 343 return None 344 else:

File ~/miniconda3/lib/python3.10/site-packages/toyplot/canvas.py:229, in Canvas._reprhtml(self) 227 def _reprhtml(self): 228 from . import html --> 229 return toyplot.html.tostring(self, style={"text-align":"center"})

File ~/miniconda3/lib/python3.10/site-packages/toyplot/html.py:370, in tostring(canvas, style) 344 def tostring(canvas, style=None): 345 """Convert a canvas to its HTML string representation. 346 347 Generates HTML markup with an embedded SVG representation of the canvas, plus (...) 368 etc. if the result is intended as a standalone HTML document. 369 """ --> 370 return six.text_type(xml.tostring(render(canvas=canvas, style=style), encoding="utf-8", method="html"), encoding="utf-8")

File ~/miniconda3/lib/python3.10/site-packages/toyplot/html.py:330, in render(canvas, fobj, animation, style) 324 context.define("toyplot/root", ["toyplot/root/id"], factory="""function(root_id) 325 { 326 return document.querySelector("#" + root_id); 327 }""") 329 # Render the canvas. --> 330 _render(canvas, context.copy(parent=root_xml)) # pylint: disable=no-value-for-parameter 332 # Return / write the results. 333 if isinstance(fobj, six.string_types):

File ~/miniconda3/lib/python3.10/site-packages/multipledispatch/dispatcher.py:278, in Dispatcher.call(self, *args, *kwargs) 276 self._cache[types] = func 277 try: --> 278 return func(args, *kwargs) 280 except MDNotImplementedError: 281 funcs = self.dispatch_iter(types)

File ~/miniconda3/lib/python3.10/site-packages/toyplot/html.py:796, in _render(canvas, context) 794 # Render everything on the canvas. 795 for child in canvas._children: --> 796 _render(canvas, child._finalize(), context.copy(parent=svg_xml)) 798 # Create a container for any Javascript code. 799 javascript_xml = xml.SubElement( 800 context.parent, 801 "div", 802 attrib={"class": "toyplot-behavior"}, 803 )

File ~/miniconda3/lib/python3.10/site-packages/multipledispatch/dispatcher.py:278, in Dispatcher.call(self, *args, *kwargs) 276 self._cache[types] = func 277 try: --> 278 return func(args, *kwargs) 280 except MDNotImplementedError: 281 funcs = self.dispatch_iter(types)

File ~/miniconda3/lib/python3.10/site-packages/toyplot/html.py:1944, in _render(canvas, axes, context) 1941 _render(axes, child._finalize(), context.copy(parent=children_xml)) 1943 if axes._show: -> 1944 _render(canvas, axes.x, context.copy(parent=cartesian_xml)) 1945 _render(canvas, axes.y, context.copy(parent=cartesian_xml)) 1946 _draw_text( 1947 root=cartesian_xml, 1948 text=axes.label._text, (...) 1951 style=axes.label._style, 1952 )

File ~/miniconda3/lib/python3.10/site-packages/multipledispatch/dispatcher.py:278, in Dispatcher.call(self, *args, *kwargs) 276 self._cache[types] = func 277 try: --> 278 return func(args, *kwargs) 280 except MDNotImplementedError: 281 funcs = self.dispatch_iter(types)

File ~/miniconda3/lib/python3.10/site-packages/toyplot/html.py:1446, in _render(canvas, axis, context) 1439 ticks_group = xml.SubElement(axis_xml, "g") 1440 for location, label, title, label_style in zip( 1441 axis._tick_locations, 1442 axis._tick_labels, 1443 axis._tick_titles, 1444 axis.ticks.labels.label.styles(axis._tick_locations), 1445 ): -> 1446 x = axis.projection(location) 1448 style = toyplot.style.combine( 1449 { 1450 "-toyplot-vertical-align": vertical_align, (...) 1454 label_style, 1455 ) 1457 _draw_text( 1458 root=ticks_group, 1459 text=label, (...) 1464 title=title, 1465 )

File ~/miniconda3/lib/python3.10/site-packages/toyplot/projection.py:173, in Piecewise.call(self, domain_values) 170 raise Exception("Unknown scale: %s" % (scale,)) # pragma: no cover 172 if range_values.shape == (): --> 173 range_values = numpy.asscalar(range_values) 174 return range_values

File ~/miniconda3/lib/python3.10/site-packages/numpy/init.py:311, in getattr(attr) 308 from .testing import Tester 309 return Tester --> 311 raise AttributeError("module {!r} has no attribute " 312 "{!r}".format(name, attr))

AttributeError: module 'numpy' has no attribute 'asscalar'

Subsampling SNPs

LalitaSimpson commented 1 year ago

I tried out the STRUCTURE cookbook and have hit a snag there as well. It reports the same error: AttributeError: module 'numpy' has no attribute 'asscalar' as well as another: AttributeError: module 'collections' has no attribute 'Sequence'

AttributeError Traceback (most recent call last) Cell In[34], line 2 1 # plot PC axes 0 and 2 ----> 2 pca.draw(0, 2);

File ~/miniconda3/lib/python3.10/site-packages/ipyrad/analysis/pca.py:405, in PCA.draw(self, ax0, ax1, cycle, colors, opacity, shapes, size, legend, label, outfile, imap, width, height, axes, kwargs) 385 def draw( 386 self, 387 ax0=0, (...) 400 axes=None, 401 kwargs): 402 """ 403 Draw a scatterplot for data along two PC axes. 404 """ --> 405 self.drawing = Drawing( 406 self, ax0, ax1, cycle, colors, opacity, shapes, size, legend, 407 label, outfile, imap, width, height, axes, 408 **kwargs) 409 return self.drawing.canvas, self.drawing.axes

File ~/miniconda3/lib/python3.10/site-packages/ipyrad/analysis/pca.py:648, in Drawing.init(self, pcatool, ax0, ax1, cycle, colors, opacity, shapes, size, legend, label, outfile, imap, width, height, axes, **kwargs) 646 self.rstyles = {} 647 self.pstyles = {} --> 648 self._get_marker_styles() 649 self._assign_styles_to_marks() 650 self._draw_markers()

File ~/miniconda3/lib/python3.10/site-packages/ipyrad/analysis/pca.py:773, in Drawing._get_marker_styles(self) 770 # get color list repeating in cycles of cycle 771 if not self.colors: 772 self.colors = itertools.cycle( --> 773 toyplot.color.broadcast( 774 toyplot.color.brewer.map("Spectral"), shape=self.cycle, 775 ) 776 ) 777 else: 778 self.colors = itertools.cycle(self.colors)

File ~/miniconda3/lib/python3.10/site-packages/toyplot/color.py:205, in broadcast(colors, shape, default) 202 colors, colormap = colors 204 # Next, convert the supplied colors into a toyplot color array. --> 205 if isinstance(colors, collections.Sequence): 206 colors = numpy.array(colors) 208 if isinstance(colors, numpy.ndarray):

AttributeError: module 'collections' has no attribute 'Sequence'

AttributeError Traceback (most recent call last) File ~/miniconda3/lib/python3.10/site-packages/IPython/core/formatters.py:342, in BaseFormatter.call(self, obj) 340 method = get_real_method(obj, self.print_method) 341 if method is not None: --> 342 return method() 343 return None 344 else:

File ~/miniconda3/lib/python3.10/site-packages/toyplot/canvas.py:229, in Canvas._reprhtml(self) 227 def _reprhtml(self): 228 from . import html --> 229 return toyplot.html.tostring(self, style={"text-align":"center"})

File ~/miniconda3/lib/python3.10/site-packages/toyplot/html.py:370, in tostring(canvas, style) 344 def tostring(canvas, style=None): 345 """Convert a canvas to its HTML string representation. 346 347 Generates HTML markup with an embedded SVG representation of the canvas, plus (...) 368 etc. if the result is intended as a standalone HTML document. 369 """ --> 370 return six.text_type(xml.tostring(render(canvas=canvas, style=style), encoding="utf-8", method="html"), encoding="utf-8")

File ~/miniconda3/lib/python3.10/site-packages/toyplot/html.py:330, in render(canvas, fobj, animation, style) 324 context.define("toyplot/root", ["toyplot/root/id"], factory="""function(root_id) 325 { 326 return document.querySelector("#" + root_id); 327 }""") 329 # Render the canvas. --> 330 _render(canvas, context.copy(parent=root_xml)) # pylint: disable=no-value-for-parameter 332 # Return / write the results. 333 if isinstance(fobj, six.string_types):

File ~/miniconda3/lib/python3.10/site-packages/multipledispatch/dispatcher.py:278, in Dispatcher.call(self, *args, *kwargs) 276 self._cache[types] = func 277 try: --> 278 return func(args, *kwargs) 280 except MDNotImplementedError: 281 funcs = self.dispatch_iter(types)

File ~/miniconda3/lib/python3.10/site-packages/toyplot/html.py:796, in _render(canvas, context) 794 # Render everything on the canvas. 795 for child in canvas._children: --> 796 _render(canvas, child._finalize(), context.copy(parent=svg_xml)) 798 # Create a container for any Javascript code. 799 javascript_xml = xml.SubElement( 800 context.parent, 801 "div", 802 attrib={"class": "toyplot-behavior"}, 803 )

File ~/miniconda3/lib/python3.10/site-packages/multipledispatch/dispatcher.py:278, in Dispatcher.call(self, *args, *kwargs) 276 self._cache[types] = func 277 try: --> 278 return func(args, *kwargs) 280 except MDNotImplementedError: 281 funcs = self.dispatch_iter(types)

File ~/miniconda3/lib/python3.10/site-packages/toyplot/html.py:1944, in _render(canvas, axes, context) 1941 _render(axes, child._finalize(), context.copy(parent=children_xml)) 1943 if axes._show: -> 1944 _render(canvas, axes.x, context.copy(parent=cartesian_xml)) 1945 _render(canvas, axes.y, context.copy(parent=cartesian_xml)) 1946 _draw_text( 1947 root=cartesian_xml, 1948 text=axes.label._text, (...) 1951 style=axes.label._style, 1952 )

File ~/miniconda3/lib/python3.10/site-packages/multipledispatch/dispatcher.py:278, in Dispatcher.call(self, *args, *kwargs) 276 self._cache[types] = func 277 try: --> 278 return func(args, *kwargs) 280 except MDNotImplementedError: 281 funcs = self.dispatch_iter(types)

File ~/miniconda3/lib/python3.10/site-packages/toyplot/html.py:1446, in _render(canvas, axis, context) 1439 ticks_group = xml.SubElement(axis_xml, "g") 1440 for location, label, title, label_style in zip( 1441 axis._tick_locations, 1442 axis._tick_labels, 1443 axis._tick_titles, 1444 axis.ticks.labels.label.styles(axis._tick_locations), 1445 ): -> 1446 x = axis.projection(location) 1448 style = toyplot.style.combine( 1449 { 1450 "-toyplot-vertical-align": vertical_align, (...) 1454 label_style, 1455 ) 1457 _draw_text( 1458 root=ticks_group, 1459 text=label, (...) 1464 title=title, 1465 )

File ~/miniconda3/lib/python3.10/site-packages/toyplot/projection.py:173, in Piecewise.call(self, domain_values) 170 raise Exception("Unknown scale: %s" % (scale,)) # pragma: no cover 172 if range_values.shape == (): --> 173 range_values = numpy.asscalar(range_values) 174 return range_values

File ~/miniconda3/lib/python3.10/site-packages/numpy/init.py:311, in getattr(attr) 308 from .testing import Tester 309 return Tester --> 311 raise AttributeError("module {!r} has no attribute " 312 "{!r}".format(name, attr))

AttributeError: module 'numpy' has no attribute 'asscalar'

isaacovercast commented 1 year ago

What version of toyplot do you have installed? conda list | grep toyplot. It looks like you're using an older version as these are both errors inside of toyplot.

The 'asscalar' error is coming from toyplot and this has been fixed already (https://github.com/sandialabs/toyplot/pull/205) , so make sure your toyplot version is > 1.0.2.

The Collections.Sequence error is also fixed in the most recent versions of toyplot (https://github.com/sandialabs/toyplot/pull/202). Please update your toyplot version and try again and let me know how it goes.

LalitaSimpson commented 1 year ago

Thanks. Yes i had an older version of toyplot. I've now done a fresh install of ipyrad in a clean environment and have got past that error and onto another... Im still using the STRUCTURE jupyter notebook and everything seems ok now up till the "Analyze results: Barplots" step. There seems to be some kind of an issue with clumpp? I do have clumpp installed v 1.1.2. Any advice is much appreciated. many thanks:

k = 3 table = struct.get_clumpp_table(k)

OSError Traceback (most recent call last) Cell In[9], line 2 1 k = 3 ----> 2 table = struct.get_clumpp_table(k)

File ~/miniconda3/envs/ipyrad1/lib/python3.8/site-packages/ipyrad/analysis/structure.py:584, in Structure.get_clumpp_table(self, kvalues, max_var_multiple, quiet) 581 raise ValueError('max_var_multiple must be >1') 583 if isinstance(kvalues, int): --> 584 return _get_clumpp_table(self, kvalues, max_var_multiple, quiet) 585 else: 586 tabledict = {}

File ~/miniconda3/envs/ipyrad1/lib/python3.8/site-packages/ipyrad/analysis/structure.py:873, in _get_clumpp_table(self, kpop, max_var_multiple, quiet) 862 cmd = [ 863 self.CLUMPP, clumphandle, 864 "-i", indfile, (...) 869 "-k", str(kpop), 870 ] 872 # call clumpp --> 873 proc = sps.Popen(cmd, stderr=sps.STDOUT, stdout=sps.PIPE) 874 comm = proc.communicate()
876 # cleanup

File ~/miniconda3/envs/ipyrad1/lib/python3.8/subprocess.py:858, in Popen.init(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, encoding, errors, text) 854 if self.text_mode: 855 self.stderr = io.TextIOWrapper(self.stderr, 856 encoding=encoding, errors=errors) --> 858 self._execute_child(args, executable, preexec_fn, close_fds, 859 pass_fds, cwd, env, 860 startupinfo, creationflags, shell, 861 p2cread, p2cwrite, 862 c2pread, c2pwrite, 863 errread, errwrite, 864 restore_signals, start_new_session) 865 except: 866 # Cleanup if the child failed starting. 867 for f in filter(None, (self.stdin, self.stdout, self.stderr)):

File ~/miniconda3/envs/ipyrad1/lib/python3.8/subprocess.py:1704, in Popen._execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session) 1702 if errno_num != 0: 1703 err_msg = os.strerror(errno_num) -> 1704 raise child_exception_type(errno_num, err_msg, err_filename) 1705 raise child_exception_type(err_msg)

OSError: [Errno 86] Bad CPU type in executable: '/Users/Lalita/miniconda3/envs/ipyrad1/bin/CLUMPP'

isaacovercast commented 1 year ago

I will guess that you are working on a mac with an M1 processor? The clumpp binary is only available at the moment for intel macs, so i will have to take a closer look to figure out how to fix this. Let me know if what I think about your mac processor is correct.

LalitaSimpson commented 1 year ago

Hi Isaac, I am on a mac but i have one of the last with an intel processor as I wanted to avoid compatibility issues with the M processors. I am on a macbook pro, 2 GHz Quad-Core Intel Core i5 processor, 16BG memory. Any other ideas? Thanks.

isaacovercast commented 1 year ago

Can you try this and show me the result?

file /Users/Lalita/miniconda3/envs/ipyrad1/bin/CLUMPP

Mine works and mine looks like this: Mach-O executable i386

Can you also show results for this? sysctl -a | grep brand? Mine looks like this

machdep.cpu.brand_string: Intel(R) Xeon(R) CPU           E5645  @ 2.40GHz
machdep.cpu.brand: 0
alexkrohn commented 1 year ago

Hi @isaacovercast. I too am running into problems using CLUMPP with an M2 silicon chip. I also tried installing Rosetta 2 and specifying the 64 bit architecture, as suggested here. No effect.

% conda activate ipyrad
% arch -x86_64 python  
Python 3.10.10 | packaged by conda-forge | (main, Mar 24 2023, 20:17:34) [Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ipyrad.analysis as ipa
>>> rerun = ipa.structure(data = "bogs.snps.hdf5", name = "143inds-no-lake-plain", workdir = "analysis-structure", load_only = True)
100 previous results loaded for run [143inds-no-lake-plain]
>>> rerun.get_clumpp_table(3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/alexkrohn/anaconda3/envs/ipyrad/lib/python3.10/site-packages/ipyrad/analysis/structure.py", line 584, in get_clumpp_table
    return _get_clumpp_table(self, kvalues, max_var_multiple, quiet)
  File "/Users/alexkrohn/anaconda3/envs/ipyrad/lib/python3.10/site-packages/ipyrad/analysis/structure.py", line 873, in _get_clumpp_table
    proc = sps.Popen(cmd, stderr=sps.STDOUT, stdout=sps.PIPE)
  File "/Users/alexkrohn/anaconda3/envs/ipyrad/lib/python3.10/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/Users/alexkrohn/anaconda3/envs/ipyrad/lib/python3.10/subprocess.py", line 1847, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 86] Bad CPU type in executable: '/Users/alexkrohn/anaconda3/envs/ipyrad/bin/CLUMPP'

As requested:

% file /Users/alexkrohn/anaconda3/envs/ipyrad/bin/CLUMPP
/Users/alexkrohn/anaconda3/envs/ipyrad/bin/CLUMPP: Mach-O executable i386
% sysctl -a | grep brand
machdep.cpu.brand_string: Apple M2 Pro
isaacovercast commented 1 year ago

Hi Alex,

Well, it looks like Rosetta doesn't support 32bit binaries, so that's going to be hard to work around. I will make a new issue to build and push a 64bit binary, but it won't happen any time soon, sorry to say. Best workaround now is to find an intel box unfortunately. -i

isaacovercast commented 1 year ago

https://github.com/dereneaton/ipyrad/issues/511

alexkrohn commented 1 year ago

Thanks for making the issue. I figured it'd be low priority, no worries at all.

At the very least I can run Structure on the fast M2 chips with the parallelization that ipa offers, then take the results and reload them on a computer with intel-based chips to do the CLUMPP functions (as here).