drvdputt
commented
8 months ago I realize now that this is probably the same bug as #266, but in a different context
Closed drvdputt closed 4 months ago
drvdputt
commented
8 months ago I realize now that this is probably the same bug as #266, but in a different context
jdtsmith
commented
5 months ago One thing I had to do to get the format function here to work correctly was to lie (temporarily) about the underlying shape of the column data. That is because a table format function is used on the "smallest unit", which for multi-dim and structured array data in a column, is the individual value. So you see I turn on a class variable _omit_shape while table formatting, which omits the final dimension of the shape (e.g. 100,3 will just have shape 100). Then it turns _omit_shape off. It would appear that this dirty hack is not universally successful, so that individual values are getting through. Your tracebacks show the value [5.27, masked, masked] originally, but the formatter is just getting 5.27.
jdtsmith
commented
5 months ago How about this radical idea: we give up entirely on using masking for bounds as our underlying representation, and instead, use structured arrays for each variable (val, min, max), and use np.nan to represent "no bound" (just as in the YAML!), with the interpretation of . We can still then use masking to mask out the entire entry (needed for when it's "inapplicable"). This eliminates pretty much all this printing hackery, and should "just work" since you can access structured arrays by index or name. The representation on disk will be a bit different, but no matter.
Our pretty printing is then just a simple column format function, assigned to all bounded param columns, or we can use the same trick we did before to override format and substitute this function for the real format (so users can throw '%8.2g' or whatever in there).
def _fmt(x):
ret = f"{x['val']:6.3f}"
if np.isnan(x['min']) and np.isnan(x['max']):
return ret + " (fixed)"
else:
mn = "-∞" if np.isnan(x['min']) else f"{x['min']:6.3f}"
mx = "∞" if np.isnan(x['max']) else f"{x['max']:6.3f}"
return f"{ret} ({mn}, {mx})"So:
t=Table()
PAR_DTYPE = np.dtype([("val", "f"), ("min", "f"), ("max", "f")])
a = np.ma.array([(np.pi, 2, 4.5), (np.pi / 2, 1, np.nan), (4, np.nan, np.nan)], dtype=PAR_DTYPE, mask=[1, 0, 0])
t['power'] = a
t['power'].info.format = _fmt
print(t)produces
power [val, min, max]
---------------------
--
1.571 ( 1.000, ∞)
4.000 (fixed)Then we must remember to temporarily strip these functions from info.format(unless we go the override route).
jdtsmith
commented
5 months ago See #283 for the idea (untested). We only need to provide the dtype on construction of the table, and can simplify all our hackery. See if that fixes your issue?
jdtsmith
commented
4 months ago
In
Features, theTableFormatterclass is overridden by a new class definition:BoundedParTableFormatterfromfeatures_format.py. It overrides the_pformat_tablefunction, to pretty-print the 3-tuples representing (value, min, max) in each BoundedMaskedColumn of the table.It does this by setting col.info.format to a custom formatting function, which assumes that it will receive a 3-tuple. This happens in this code block
after this, the super()._pformat_table is invoked to apply these formatters.
I'm still not sure how to reproduce this, but printing the features table throws an exception sometimes. For some reason the formatting function
_fmtreceives a scalar value instead of a 3-tuple. This happens during the assignmentcol.info.format = .... AstropyColumnuses a custom assigment function for this property, that tries to format a test value. I think there is some sort of automatic unpacking going on in that test function, that results in an element of the tuple being passed, instead of the intended actual 3-tuple in the column. I have no solution for this, except guarding for this in our_fmtfunction.Here's the series of exceptions