extend libmadx.py API to access sequence information

[coldfix]

I am planning to add and alter some functionality in libmadx to make the madx module more useful on its own. My idea is the following:

Create a class Sequence that internally just stores the name of the sequence. It has @propertys to lazily obtain more information when/if desired:

• beam (ex, ey, ...)
• last twiss table (directly via the sequence)
• access to elements (as you started in your branch)
• more ideas?
• there will be a classmethod get_sequences returning a list of instances. scrub that, it can't be a classmethod, since:
• Sequence objects are instanciated via Madx.get_sequence(name), Madx.get_sequences() or similar (they need to be specific to a Madx instance)

Most of this should be possible to add without touching any backward compatibility issues. Changes to the current libmadx.py:

• add get_twiss(sequence) -> numpy.array
• add get_beam(sequence) -> dict
• add get_elements(sequence) -> list(Element) (saving this for #38, though)
• add get_expanded_elements(sequence) -> list(Element)
• add get_active_sequence() -> name making it possible to use Madx.* methods without specifying the sequence name
• modify get_sequences() -> list(names) (incompatible, but much nicer!)

I am not fully decided whether to store a pointer to the C-struct or the name of the sequence within class Sequence. I tend towards storing the name, because

• although the pointer is slightly better in terms of performance
• there may be memory issues with obsolete pointers, when a sequence gets destroyed or moved (is that possible? anyway, it might be someday)
• storing the pointer as class member means it must be a cdef class, instances must be created from libmadx. This in turn creates additional complexity when marshaling the instances between processes.

Furthermore, I'd like to remove the print statements from _libmadx.gettable and replace them with raise RuntimeError or similar. The print statements in _libmadx.getsequences will be obsolete anyway, the properties can then be examined via the class.

By the way, with this and #38, my GUI application will probably become useful for plain MADX files, very soon.

Any thoughts?

[coldfix]

I am heavily reworking the cern.cpymad.madx API as well to return Table proxy objects, that can then be used to retrieve columns/summary if desired, but won't allocate any unnecessary numpy arrays if the columns are not used. It is no problem for the returned value to retain backward compatibility. This may sound complex, but the code is actually quite simple and short.

[will be handled in a separate PR]: I want to remove the retdict parameters from most of the function signatures, as it just creates additional complexity. Do you think this is an acceptible back-ward incompatible change?

[will be handled in separate PR]: Furthermore, I'm not sure whether the columns parameter becomes unnecessary. Apart from determining which columns to retrieve, it was also used in a SELECT call. Does the SELECT call have any influence, apart from setting which variables to output if a file was specified? What is the meaning of the pattern parameter?

Best regards,

Thomas

[coldfix]

Added element inspection to this branch as well, This will close #38. Since I used quite a lot of steps, I recommend not to look at the individual changes, but rather at the overall diff.

If you want to check it out, you could use a test.madx like so:

qp: quadrupole, k1=2, l=1;
sb: sbend, l=2, angle=10;

tt: sequence, l=4, refer=center;
qp, at=0.5;
qp, at=1.5, l=2;
sb, at=3;
endsequence;

and the following python code:

from cern.cpymad.madx import Madx

m = Madx()
m.call('test.madx')
el = m.get_sequence('tt').get_elements()

from cern.cpymad.types import Expression
import yaml

def element_dump(data, stream=None, Dumper=yaml.Dumper, **kwds):
    class OrderedDumper(Dumper):
        pass
    def _expr_representer(dumper, data):
        return dumper.represent_mapping(
            yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG,
            {'expr': data.expr, 'value': data.value})
    def unicode_representer(dumper, uni):
        node = yaml.ScalarNode(tag=u'tag:yaml.org,2002:str', value=uni)
        return node
    OrderedDumper.add_representer(unicode, unicode_representer)
    OrderedDumper.add_representer(Expression, _expr_representer)
    return yaml.dump(data, stream, OrderedDumper, **kwds)

element_dump(el, open("data.yml", 'w'))

This might already be ready for merging, but I will take a few days to see if I need any more features, that should be directly integrated into this PR. And I should really also add some tests. Note, that most of this is still downward compatible and shouldn't make it necessary to change using code (except when that relies on the exact return type of .twiss()).

EDIT: added the little example above as test.

[Eothred]

This is a big change, will take me some time to look through it all. In principle I have no objections, thanks once again for all your work!

[coldfix]

Oh, it looks like some commits were missed in the merge to master. We should merge these as well. We can put these in testing tomorrow. I recommend merging with the "Merge pull request" button, to be sure to merge the most up-to-date version.