two loops

[vsraptor]

when i try to list() the result :

I'm trying to do nested iterators ...

In [108]: list( file('text/622_lines.txt') | twofor((sents,words))  )                                                                                                        
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-108-6a273f2a1594> in <module>
----> 1 list( file('text/622_lines.txt') | twofor((sents,words))  )

/......./lib/tools/text.py in twofor(it, its)
     28 @Pipe
     29 def twofor(it,its):
---> 30         for it1 in it | its[0]:
     31                 for item in it1 | its[1]:
     32                         yield item

TypeError: 'Pipe' object is not iterable

[JulienPalard]

I don't really get what you're trying to do, what is file, sents and words?

Is that really different from list( file('text/622_lines.txt') | sents | words)) )?

[vsraptor]

you are right.. i was trying to squeeze too much stuff into pipes.. i have something like this

  for words in sentences : 
       ....code ...
        filtered = words | longer(2) | is_word | isnt_stop | lwcase | lemma
        .... code ..
       process(filltered)

the exit of the pipe (process) requires sentences rather than stream of words

[JulienPalard]

i was trying to squeeze too much stuff into pipes

Yes, bad idea, keep your code easily modifiable.

Also maybe try to stick to "one line do one thing", it helps readability too.

filtered = words | longer(2) | is_word | isnt_stop | lwcase | lemma

is OK, it filters, but don't try to stuff more in this line, better keep it in a for loop as you showed.

It's like trying to put list comprehension inside list comprehension: it can nicely fit on a single line, but can't be (easily) modified.