MazinLab / MKIDPipeline

The MKID Data Pipeline
http://web.physics.ucsb.edu/~bmazin/
6 stars 3 forks source link

Getting a pixel spectrum of xkid data created with (presumably) `build_hdf.py` #91

Open ld-cd opened 8 months ago

ld-cd commented 8 months ago

necompiler appears to be used to build queries, we blow out the query stack when requesting the pixel spectrum of unwavecaled pixels, but only data from build_hdf.py, that same data passed through h5py works fine. Unclear if this is just because of the lack of a wavecal but given @strampelligiovanni has had issues getting the xkid data through the pipeline its possible we have just not yet had the opportunity to hit this

Traceback, likely not consistent:

---------------------------------------------------------------------------
MemoryError                               Traceback (most recent call last)
<ipython-input-14-e835e05cf115> in <module>
      1 for i in range(80):
----> 2     print(o.get_pixel_spectrum(i))
      3

~/Workspace/analysis/MKIDPipeline/mkidpipeline/photontable.py in get_pixel_spectrum(self, pixel, start, duration, weight, wave_start, wave_stop, bin_width, bin_edges, bin_type)
    828         """
    829
--> 830         photons = self.query(pixel=pixel, start=start, intt=duration, startw=wave_start, stopw=wave_stop)
    831
    832         weights = photons['weight'] if weight else None

~/Workspace/analysis/MKIDPipeline/mkidpipeline/photontable.py in query(self, startw, stopw, start, stopt, resid, intt, pixel, column)
    661             tic = time.time()
    662             try:
--> 663                 q = self.photonTable.read_where(query, field=column)
    664             except SyntaxError:
    665                 raise

~/.conda/envs/wheatley/pipeline/lib/python3.7/site-packages/tables/table.py in read_where(self, condition, condvars, field, start, stop, step)
   1512         self._g_check_open()
   1513         coords = [p.nrow for p in
-> 1514                   self._where(condition, condvars, start, stop, step)]
   1515         self._where_condition = None  # reset the conditions
   1516         if len(coords) > 1:

~/.conda/envs/wheatley/pipeline/lib/python3.7/site-packages/tables/table.py in _where(self, condition, condvars, start, stop, step)
   1479         if compiled.index_expressions:
   1480             chunkmap = _table__where_indexed(
-> 1481                 self, compiled, condition, condvars, start, stop, step)
   1482             if not isinstance(chunkmap, np.ndarray):
   1483                 # If it is not a NumPy array it should be an iterator

~/.conda/envs/wheatley/pipeline/lib/python3.7/site-packages/tables/table.py in _table__where_indexed(self, compiled, condition, condvars, start, stop, step)
    198
    199     # Compute the final chunkmap
--> 200     chunkmap = ne.evaluate(strexpr, cmvars)
    201     if not chunkmap.any():
    202         # The chunkmap is all False, so the result is empty

~/.conda/envs/wheatley/pipeline/lib/python3.7/site-packages/numexpr/necompiler.py in evaluate(ex, local_dict, global_dict, out, order, casting, **kwargs)
    807     expr_key = (ex, tuple(sorted(context.items())))
    808     if expr_key not in _names_cache:
--> 809         _names_cache[expr_key] = getExprNames(ex, context)
    810     names, ex_uses_vml = _names_cache[expr_key]
    811     arguments = getArguments(names, local_dict, global_dict)

~/.conda/envs/wheatley/pipeline/lib/python3.7/site-packages/numexpr/necompiler.py in getExprNames(text, context)
    694
    695 def getExprNames(text, context):
--> 696     ex = stringToExpression(text, {}, context)
    697     ast = expressionToAST(ex)
    698     input_order = getInputOrder(ast, None)

~/.conda/envs/wheatley/pipeline/lib/python3.7/site-packages/numexpr/necompiler.py in stringToExpression(s, types, context)
    272         else:
    273             flags = 0
--> 274         c = compile(s, '<expr>', 'eval', flags)
    275         # make VariableNode's for the names
    276         names = {}

MemoryError: