Some of the logging/probe functions will embed a carriage return in the middle of a row for CSV output

[markcoletti]

E.g., this should have been one row, but a carriage return was embedded in the middle of the list:

1999,3,Combined Individual,,"[False False False  True  True False  True  True  True  True  True  True
  True  True  True  True False False  True  True]",14

[lukepmccombs]

This is an issue with numpy np.printoptions. When numpy converts an array to a string, explicitly or implicitly for writing to streams, it uses a predetermined line width to insert breaks. You can bypass this by setting an arbitrarily large linewidth parameter while using np.printoptions as a context manager, or np.set_printoptions to change it thereafter.

[markcoletti]

@lukepmccombs is working on this now. He's exploring native numpy solutions for emitting CSV rows.

[lukepmccombs]

Here is my proposed solution: In numpy you can override the printing function with another of your choosing via np.set_string_function. We can use this functionality to make a print function that doesn't insert newlines and is usable for loading the arrays later. A good option for this is:

def _to_quoted_list(arr):
    f'"{json.dumps(arr.tolist())}"'

which returns a quoted, single line json representation for any size array, and can be restored using json.loads on the contents of the cell.

[lukepmccombs]

This should be closable now. :+1: