Simplify brick selectors

[bartvm]

A question for @janchorowski and @rizar, because I don't know: Is there any reason that we chose to parse strings for selectors instead of using pure Python? Are there any features it offers of what could be done using Python predicates with a few helper functions? If something along the lines of this could work, I guess it could get rid of a lot of string parsing code while also saving people from learning yet another syntax, while probably being more powerful and flexible on the long term.

def brick_predicate(brick):
    return all([
        isinstance(getattr(brick, 'weights_init', None), IsotropicGaussian),
        'foo' in brick.name,
        has_parent(brick, some_other_brick),
        ...
    ])
bricks = select_bricks(mlp, predicate=brick_predicate)

Where has_parent is a simple helper function of which we could write a few.

[janchorowski]

It's because you will get an awfully long list of predicates to get to something a few levels down. It is much more coscise to write
"/encoder/forward_net/embedder" than .child('encoder').child('forward_net').child...

IMHO Python is not that good a using it for domain specific languages, (even something as simple as matrix algebra feels nicer in Matlab or R which offer custom operators) and when you want to have a DSL in python, you end up parsing strings (like e.g. statsmodels). I thought about using pure python expressions - in fact you can, my selectors do just what you propose - they filter lists. The select routine can take a tring, that will be parsed into a list of Selector instances, but it can also use directly this list, or even a mix of strings to parse and Selectors. However, not having the parser makes things much more verbose.

On 1/23/2015 6:06 PM, Bart van Merriënboer wrote:

A question for @janchorowski https://github.com/janchorowski and @rizar https://github.com/rizar, because I don't know: Is there any reason that we chose to parse strings for selectors instead of using pure Python? Are there any features it offers of what could be done using Python predicates with a few helper functions? If something along the lines of this could work, I guess it could get rid of a lot of string parsing code while also saving people from learning yet another syntax, while probably being more powerful and flexible on the long term.

def brick_predicate(brick): return all([ isinstance(getattr(brick,'weights_init',None), IsotropicGaussian), 'foo' in brick.name, has_parent(brick, some_other_brick), ... ]) bricks= select_bricks(mlp,predicate=brick_predicate)

Where |has_parent| is a simple helper function of which we could write a few.

— Reply to this email directly or view it on GitHub https://github.com/bartvm/blocks/issues/147.

[rizar]

I thing the main reason is the lifecycle of machine translation and speech recognition projects. You end up with configuration files in which you want to see as little Python code as possible. You want to do this:

dict(
   ....
   'weighs_init': [('//', 0.001), ('/translator/decoder/attention/projector' : 0.01)]
)

[janchorowski]

That's a very good point too - plain text config files are good

On 1/23/2015 6:15 PM, Dmitry Bogdanov wrote:

I thing the main reason is the lifecycle of machine translation and speech recognition projects. You end up with configuration files in which you want to see as little Python code as possible. You want to do this:

dict( .... 'weighs_init': [('//',0.001), ('/translator/decoder/attention/projector' :0.01)] )

— Reply to this email directly or view it on GitHub https://github.com/bartvm/blocks/issues/147#issuecomment-71228053.

[bartvm]

Okay, so it seems it's mostly a matter of syntactical preference and usability. Ideally, I would like it if the raw Python selectors are relatively usable as well, and nicely separated from the syntactic sugar. That way I can do something like this instead of Jan's example.

def brick_predicate(root, brick):
    if brick not in children(root.c('encoder').c('forward_net').c('embedder')):
        return False
    return all([
        isinstance(getattr(brick, 'weights_init', None), IsotropicGaussian),
        'foo' in brick.name,
        ...
    ])
bricks = select_bricks(mlp, predicate=brick_predicate)

This is something I would have no idea how to write in Jan's syntax, but it takes me a second to do in Python. As a user I wouldn't want a particular syntax imposed on me if it stops me from being productive at the cost of a bit of verbosity (but also clarity, since I know Python, not the selector language).

I appreciate the importance of configuration files for very large scale experiments, it's just that right now I feel that the brick selector syntax is nice, but unnecessary for smaller-scale stuff, and as such just an entrance barrier. Ideally I would like people to start with a simple predicate=lambda brick: 'bla' in brick.name before scaling up to a new selector language that makes large-scale, long-term experiments more manageable.

[janchorowski]

But this is already there - you have a filter Selector that is unparsable and does just this.

BTW, I'm looking at my failed nets on TIMIT and my dev set starts with: "bricks are an alterative" :)

On 1/23/2015 6:32 PM, Bart van Merriënboer wrote:

Okay, so it seems it's mostly a matter of syntactical preference and usability. Ideally, I would like it if the raw Python selectors are relatively usable as well, and nicely separated from the syntactic sugar. That way I can do something like this instead of Jan's example.

def brick_predicate(root,brick): if bricknot in children(root.c('encoder').c('forward_net').c('embedder')): return False return all([ isinstance(getattr(brick,'weights_init',None), IsotropicGaussian), 'foo' in brick.name, ... ]) bricks= select_bricks(mlp,predicate=brick_predicate)

This is something I would have no idea how to write in Jan's syntax, but it takes me a second to do in Python. As a user I wouldn't want a particular syntax imposed on me if it stops me from being productive at the cost of a bit of verbosity (but also clarity, since I know Python, not the selector language).

I appreciate the importance of configuration files for very large scale experiments, it's just that right now I feel that the brick selector syntax is nice, but unnecessary for smaller-scale stuff, and as such just an entrance barrier. Ideally I would like people to start with a simple |predicate=lambda brick: 'bla' in brick.name| before scaling up to a new selector language that makes large-scale, long-term experiments more manageable.

— Reply to this email directly or view it on GitHub https://github.com/bartvm/blocks/issues/147#issuecomment-71230618.

[rizar]

I think that by carefully merging Jan's selectors we can also make an interface like Bart likes. But first let's get it into master, for which it should be rebased and reviewed. I think it would be nice to have it for the presentation Bart will give on February 12.