leginonbot
commented
7 months ago Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2015-09-04T17:56:20Z
I think I am using memmap incorrectly.
http://stackoverflow.com/questions/1661986/why-doesnt-pythons-mmap-work-with-large-files
We need to scrap memmap, I do not think 1 particle per iteration is ideal, but we can check system memory and do a chunk of particles at a time.
I already have the tools do implement the functionality, but I do not have the time. Probably a good idea to create a class for handling stack files.
Neil
P.S. Scott you should add yourself as a watcher on these bugs as well.
Author Name: Scott Stagg (Scott Stagg) Original Redmine Issue: 3529, https://emg.nysbc.org/redmine/issues/3529 Original Date: 2015-09-03
I got the following error when I tried to make a substack: ... averaging stack for summary web page Traceback (most recent call last): File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/bin/alignSubStack.py", line 293, in
subStack.start()
File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/bin/alignSubStack.py", line 279, in start
apStack.averageStack(stack=oldstack,outfile=outavg,partlist=includeParticle)
File "/panfs/storage.local/imb/stagg/software/myami_dev/appion/appionlib/apStack.py", line 328, in averageStack
particles = imagic.read(stack)
File "/panfs/storage.local/imb/stagg/software/myami_dev/pyami/imagic.py", line 73, in read
a = readImagicData(pair['img'], header_dict, frame)
File "/panfs/storage.local/imb/stagg/software/myami_dev/pyami/imagic.py", line 43, in readImagicData
a = numpy.memmap(filename, dtype=dtype, mode='r', offset=start, shape=shape, order='C')
File "/usr/lib64/python2.7/site-packages/numpy/core/memmap.py", line 257, in new
mm = mmap.mmap(fid.fileno(), bytes, access=acc, offset=start)
mmap.error: [Errno 12] Cannot allocate memory