numpy linspace not working on u.Quanttity on iceberg

[fredgent]

When calling Z = np.linspace(xyz[4],xyz[5],Nxyz[2]), where xyz is a 1D array in astropy u.Mm I get

Traceback (most recent call last):
  File "spruit_atmosphere.py", line 100, in <module>
    coords = atm.model_pars.get_coords(model_pars['Nxyz'], u.Quantity(model_pars['xyz']))
  File "/home/sm1fg/pysac/pysac/mhs_atmosphere/parameters/model_pars.py", line 150, in get_coords
    xyz[5],Nxyz[2])
  File "/home/sm1fg/anaconda2/lib/python2.7/site-packages/numpy/core/function_base.py", line 115, in linspace
    y[-1] = stop
ValueError: setting an array element with a sequence.
./src/common/module.cpp(194): assert "module->m_state == State_Initialized" failed in DoCleanUpModules(): not initialized module being cleaned up

On iceberg numpy version is 1.10.4 whereas on my workstation it is 1.8.2 astropy 1.1.1 1.0.2 It works fine on my workstation and when I last tried it on my laptop and the Finnish cluster. Any ideas?

[Cadair]

see astropy/astropy#4543 looks likek you need numpy 1.10.5 or greater.

[fredgent]

Thanks Stuart.

It doesn't look hopeful. 1.10.4 seems to be the latest version downloadable. I've tried rolling back to an earlier version, but that hasn't worked and my installation is a mess, so I'll start again from scratch tomorrow.

Cheers

Fred

On 09/02/16 20:42, Stuart Mumford wrote:

see astropy/astropy#4543 https://github.com/astropy/astropy/issues/4543 looks likek you need numpy 1.10.5 or greater.

— Reply to this email directly or view it on GitHub https://github.com/SWAT-Sheffield/pysac/issues/10#issuecomment-181998136.

[Cadair]

@fredgent I have done some serious upgrades on the Python install on iceberg: http://iceberg.readthedocs.org/en/latest/software/apps/python.html

let me know if you need a hand.

[Cadair]

don't miss this section: http://iceberg.readthedocs.org/en/latest/software/apps/python.html#using-python-with-mpi

[fredgent]

Thanks.

Which module do you recommend I activate, conda or anaconda2-2.4.0? Does astropy still have issues with Python3?

Do I still need to install the development version of astropy/sunpy/yt or can I use these out of the box versions?

Regards,

Fred

On 02/10/2016 12:14 AM, Stuart Mumford wrote:

@fredgent https://github.com/fredgent I have done some serious upgrades on the Python install on iceberg: http://iceberg.readthedocs.org/en/latest/software/apps/python.html

let me know if you need a hand.

— Reply to this email directly or view it on GitHub https://github.com/SWAT-Sheffield/pysac/issues/10#issuecomment-182103066.

[Cadair]

You probably want to activate apps/python/conda and then create your own environment. You should be ok with the out of the box versions of everything.

[fredgent]

It's ok it's working. I have my own conda and need to set it up in the qrsh environment then load the environment in the job script. Works fine for serial. Haven't tested mayavi yet, but expect that will be fine. I may get back to you about how to use mpi in the batch script, but so far looking hopeful

Fred

On 02/11/2016 04:45 PM, Stuart Mumford wrote:

You probably want to activate apps/python/conda and then create your own environment. You should be ok with the out of the box versions of everything.

— Reply to this email directly or view it on GitHub https://github.com/SWAT-Sheffield/pysac/issues/10#issuecomment-182893289.

[fredgent]

Hi Stuart, Mike,

When I launch an interactive shell with qsh there is no module environment. sm1fg@[11:31:07]> module load apps/python/conda bash: module: command not found

Yesterday pysac worked, but today it breaks when np.linspace(u.Quantity,...) is called. The modules are loaded in the batch script, but I had to set up myconda in a qrsh terminal.

I would like to try again from scratch to build the environment in the qsh session, so have deleted my existing local packages apart from pysac. The python instructions imply that the module environment should be accessible, but it isn't. This leads me to infer there is an error in my account settings.

Could you check this for me?

Incidently module appears to exist in the login node, although it is not functional. Does this perhaps support adding modules to the .bashrc to automatically load the preferred packages.

Cheers,

Fred

On 02/11/2016 04:45 PM, Stuart Mumford wrote:

You probably want to activate apps/python/conda and then create your own environment. You should be ok with the out of the box versions of everything.

— Reply to this email directly or view it on GitHub https://github.com/SWAT-Sheffield/pysac/issues/10#issuecomment-182893289.

[Cadair]

@fredgent I modified your .bashrc file to source the global /etc/profile file, it should be working now.

[fredgent]

Thanks Stuart.

I still haven't been able to resolve the problem with np.linspace not handling u.Quantity - it did work yesterday and now it will not! I have rebuilt my conda installation from scratch - mayavi reverts to numpy 1.9.*, but then scipy.interpolate will not work and scipy requires 1.10.4

I have no idea what has changed since yesterday!

The on line chat I can see from astropy talks about it, but does not as far as I can see offer a fix. Does your pysac installation on iceberg work on the mhs_atmosphere examples?

Fred

On 02/12/2016 02:04 PM, Stuart Mumford wrote:

@fredgent https://github.com/fredgent I modified your bashrc file to source the global |/etc/profile| file, it should be working now.

— Reply to this email directly or view it on GitHub https://github.com/SWAT-Sheffield/pysac/issues/10#issuecomment-183296752.

[Cadair]

@fredgent It might be easier to work around the bug by changing the pysac code, it should just be a case of doing .value in the linspace then multiplying by the unit on the outside.

conda recently changed the way it handles numpy versions, and it is more of a pain to roll back now.

[fredgent]

Rewritten with quantities applied to numpy arrays rather than numpy.inspace generated on astropy quantities.