e_ is assigned to the scalar dolp

[Twenkid]

@boris-kz

Edit for clarity: that's _1_2D, just an edit box for comments. See the master repository.

def Le1(f): # last "_" denotes array vs. element, first "_" denotes higher-line array, pattern, variable

    #...

    pri_s, I, D, Dy, M, My, Vq, p_, olp, olp_ = 0,0,0,0,0,0,0,[],0,[]
    vP = pri_s, I, D, Dy, M, My, Vq, p_, olp, olp_
    pri_sd, Id, Dd, Ddy, Md, Mdy, Dq, d_, dolp, dolp_ = 0,0,0,0,0,0,0,[],0,[]
    dP = pri_sd, Id, Dd, Ddy, Md, Mdy, Dq, d_, dolp, dolp_

#   ...

  dP = pri_sd, Id, Dd, Ddy, Md, Mdy, Dq, d_, dolp, dolp_
  #         0, 1, ...                       , [8] = dolp = 0, [9] - dolp_ []

#  ...

pri_sd, Id, Dd, Ddy, Md, Mdy, Dq, d_, dolp, dolp_ = dP  # dP tuple

#dolp = dP[8]

_P_, next_P_ = comb_P(dP, len(dP_), _dP_, A, _x, x, y, Y, _P_, next_P_)

#
def comb_P(P, nP, _P_, A, _x, x, y, Y, P_, next_P_):  
#...                                                  
 s, I, D, Dy, M, My, Q, r, e_, olp_ = P  # M vs. V: eval per quadrant only, V = M - 2a * W?
 w = len(e_); ix = x - w  # w: P' width, ix: P' initial coordinate

 #e_ = dolp

In that code isn't "e_" assigned to dolp, which is a single value scalar, not a list.

Why "е"? еval?

[boris-kz]

Todor, this function is generic for both vP and dP. e (for element) is p in vP or d_ in dP.

[boris-kz]

Ah, yes, I forgot to add olp in P, thanks!

[boris-kz]

I am confused Todor, why are you mixing level_1 with level_1_2D? level_1_2D is a self-contained alternative to level_1, the former doesn't use the latter.

[Twenkid]

I am not mixing anything, that's just an edit box to put a comment, I'm taking about 2D of course, and it's not a real pull-request with edition of code from my side.

I don't want to create a mess of forks, my forked version is outdated anyway, too.

[boris-kz]

Got it. I just published level_1q.py q for quadrant. It is a minor update of level_1_2D, except that I am not packing vP and dP in ycomp(). I feel their variables need to be exposed at this point

On Sun, Jun 25, 2017 at 6:31 PM, Todor Arnaudov notifications@github.com wrote:

I am not mixing anything, that's just an edit box to put a comment, I'm taking about 2D of course, and it's not a real pull-request with edition of code from my side.

I don't want to create a mess of forks, my forked version is outdated anyway, too.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/pull/2#issuecomment-310932926, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGVppKTSmJnsnaOdSzL36nT7fEaVwks5sHt-mgaJpZM4OEoGz .

[Twenkid]

 _P_, next_P_ = comb_P(vP, len(vP_), _vP_, r, A, _x, x, y, Y, _P_, next_P_)
#(...)
 while x >= _x:  # horizontal overlap between P and next _P

         _P = _P_.pop(); n_P += 1  # n_P is _P counter to sync Fork_ with _P_, better than len(P_) - len(_P_)?

  _P_.reverse(); _P_ += buff_; _P_.reverse() # front concat for next-P comp_P()

I don't have a clear mental picture of that forking process yet, but are you sure that's the correct scanning sequence?

.pop() reads the last item, while x is incremented regularly.

Maybe you want to keep it simple with lists, but that adds "reverse".

For double-side appends there is a deque structure, which doesn't require reversal.

https://docs.python.org/2/library/collections.html#deque-objects

[boris-kz]

Yes, good catch. Deque looks interesting, though not sure it's better than reversing, I would guess that underlying operations are the same.

I should probably have:

<

P.reverse() # at the start of each line

while x >= _x: # horizontal overlap between P and next _P #(...)

buff_ += P # instead of P.reverse(); P += buff_; P.reverse()

On Mon, Jun 26, 2017 at 6:17 PM, Todor Arnaudov notifications@github.com wrote:

P, nextP = combP(vP, len(vP), vP, r, A, _x, x, y, Y, P, nextP)

(...)

while x >= _x: # horizontal overlap between P and next _P

     _P = _P_.pop(); n_P += 1  # n_P is _P counter to sync Fork_ with _P_, better than len(P_) - len(_P_)?

P.reverse(); P += buff_; P.reverse() # front concat for next-P comp_P()

I don't have a clear mental picture of that forking process yet, but are you sure that's the correct scanning sequence?

.pop() reads the last item, while x is incremented regularly.

Maybe you want to keep it simple with lists, but that adds "reverse".

For double-side appends there is a deque structure, which doesn't require reversal.

https://docs.python.org/2/library/collections.html#deque-objects

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/pull/2#issuecomment-311197882, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGRZVUs1_tMvSoK-27S6RjxBXVLaWks5sIC4AgaJpZM4OEoGz .

[boris-kz]

BTW, the process is not forking, it is P scanning higher-line patterns for x overlap.

fork_ contains potentially multiple partial overlaps per P, and root is a duplicate of these overlap per _P.

On Mon, Jun 26, 2017 at 8:14 PM, Boris Kazachenko boris.kz@gmail.com wrote:

Yes, good catch. Deque looks interesting, though not sure it's better than reversing, I would guess that underlying operations are the same.

I should probably have:

<

P.reverse() # at the start of each line

while x >= _x: # horizontal overlap between P and next _P #(...)

buff_ += P # instead of P.reverse(); P += buff_; P.reverse()

On Mon, Jun 26, 2017 at 6:17 PM, Todor Arnaudov notifications@github.com wrote:

P, nextP = combP(vP, len(vP), vP, r, A, _x, x, y, Y, P, nextP)

(...)

while x >= _x: # horizontal overlap between P and next _P

     _P = _P_.pop(); n_P += 1  # n_P is _P counter to sync Fork_ with _P_, better than len(P_) - len(_P_)?

P.reverse(); P += buff_; P.reverse() # front concat for next-P comp_P()

I don't have a clear mental picture of that forking process yet, but are you sure that's the correct scanning sequence?

.pop() reads the last item, while x is incremented regularly.

Maybe you want to keep it simple with lists, but that adds "reverse".

For double-side appends there is a deque structure, which doesn't require reversal.

https://docs.python.org/2/library/collections.html#deque-objects

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/pull/2#issuecomment-311197882, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGRZVUs1_tMvSoK-27S6RjxBXVLaWks5sIC4AgaJpZM4OEoGz .

[Twenkid]

Reg. deque - I doubt it's the same representation. Reversing is confusing and prone to mistakes if done just in order to use "pop".

I don't know how it's in Python, in a lower level lang. it could be a double-linked list, the first and the last elements pointers are at hand ("roots") and they are accessed directly, the other elements are traversed, or one could keep records of particular intermediate pointers.

Reversing can be done cheaply with an abstract data structure holding a simple array with an iterator and a "direction", thus just switching the direction +1 -1 and the respective "begin" element would reverse the content when iterated.

[boris-kz]

I meant the same operations on a hardware level, unless they have double-linked registers. Reversing is confusing only if you have to reverse it back after access, I don't have that. Note that I added in Le1(): <vP.reverse(); dP.reverse() # for pop(), at the start of each line>

On Tue, Jun 27, 2017 at 7:40 AM, Todor Arnaudov notifications@github.com wrote:

Reg. deque - I doubt it's the same representation. Reversing is confusing and prone to mistakes if done just in order to use "pop".

I don't know how it's in Python, in a lower level lang. it could be a double-linked list, the first and the last elements pointers are at hand ("roots") and they are accessed directly, the other elements are traversed, or one could keep records of particular intermediate pointers.

Reversing can be done cheaply with an abstract data structure holding a simple array with an iterator and a "direction", thus just switching the direction +1 -1 and the respective "begin" element would reverse the content when iterated.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/pull/2#issuecomment-311332920, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGe4JfBQBNPESKmlISncw9fhitM6Dks5sIOoqgaJpZM4OEoGz .

[Twenkid]

I noticed.

What about .appendleft() to a deque, then .pop() from it, without reverse.

BTW, that "<" as literal in the editor also keeps the _, but I meant the button on the toolbar after selecting text:

a = b

It's between single back quotes in the editor text.

[boris-kz]

On Jun 27, 2017 10:28 AM, "Todor Arnaudov" notifications@github.com wrote:

I noticed.

What about .appendleft() to a deque, then .pop() from it, without reverse.

I thought about it but mixing lists with deques is messy, probably not worth it.

BTW, that "<" as literal in the editor also keeps the _, but I meant the button on the toolbar after selecting text:

a = b

It's between single back quotes in the editor text.

Ok. I reply by email, so there are no buttons.

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/pull/2#issuecomment-311375643, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGffaEsdWWpD-iMhPQAv3aYBtJp6rks5sIRFzgaJpZM4OEoGz .

[Twenkid]

As of performance:

https://stackoverflow.com/questions/32543608/deque-popleft-and-list-pop0-is-there-performance-difference

https://stackoverflow.com/questions/6256983/how-are-deques-in-python-implemented-and-when-are-they-worse-than-lists

https://stackoverflow.com/questions/23487307/python-deque-vs-list-performance-comparison

From the last one:

https://stackoverflow.com/a/23487658

[Twenkid]

In general I'd avoid reverse, that probably creates a new list, iterates over it and frees the previous instance. You want it for cheap "pop", at the price of a traversal over all the elements before that.

[boris-kz]

Ok, you are right. Could you do a fork replacing lists with dequeues, where appropriate ?:)

BTW, I realized that a proper term for my unit is vertex gradient, vs. quadrant gradient.

So I renamed level_1q - > level_1_2D_draft, and level_1_2D_draft -> level_1_2D_old.

On Tue, Jun 27, 2017 at 11:30 AM, Todor Arnaudov notifications@github.com wrote:

In general I'd avoid reverse, that probably creates a new list, iterates over it and frees the previous instance. You want it for cheap "pop", at the price of a traversal over all the elements before that.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/pull/2#issuecomment-311395321, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGbCZ57l5wTrxNM7rCYQQRpnGRGXYks5sISAVgaJpZM4OEoGz .

[Twenkid]

OK, I'll check regarding the deque.

Why vertex? A vertex on 4-pixel grid? Vertical? Just a fancy term? Vertices imply being part of polygons or graphs.

Did you receive my other fork/pull request:

https://github.com/boris-kz/CogAlg/compare/master...Twenkid:Twenkid-patch-2_level_1q

(There's some mess with the links/comments)

https://github.com/Twenkid/CogAlg/pull/1/commits/90e85018256a67302a5884bb7ad6692dde19b900

> Refactoring for readability and Id - not defined in comb_P (I saw it's defined later)
>  ...
> 
> I suggest using n_dPP, n_vPP instead of i_ ... to aling it with the style of nP and do not confuse i with "input". 
> Also, when calling functions with identifiers which are different to the function definitions, it's useful to use named assigns which make it more clear and readable, like:
> 
> n_vPP, _P_, next_P_ = comb_P(P=vP, nP = len(vP_), _P_ = _vP_, r, A, _x, x, y, Y, _P_, next_P_) #n_vPP instead of i_vPP
> n_vPP, _P_, next_P_ = comb_P(P=vP, nP = len(vP_), _P_ = _vP_, r, A, _x, x, y, Y, _P_, next_P_) #n_vPP instead of i_vPP
>                vP = vP, n_vPP
> 
>   
>     ` n_dPP, _P_, next_P_ = comb_P(P = dP, nP = len(dP_),  _P_ =_dP_, r, A, _x, x, y, Y, _P_, next_P_) 
>       dP = dP, n_dPP` 
> 
> And similarly for the vP.

[boris-kz]

OK, I'll check regarding the deque.

Thanks.

Why vertex? A vertex on 4-pixel grid? Vertical? Just a fancy term? Vertices imply being part of polygons or graphs.

Yes, the polygon here is the quadrant. Vertex doesn't represent the whole quadrant, only the first corner of it.

Did you receive my other fork/pull request:

master...Twenkid:Twenkid-patch-2_level_1q https://github.com/boris-kz/CogAlg/compare/master...Twenkid:Twenkid-patch-2_level_1q

No, and there is no option to comment on it: Can’t automatically merge. Don’t worry, you can still create the pull request.

Refactoring for readability and Id - not defined in comb_P (I saw it's defined later)

That's because Id (and any var.d) is specific for dP, while comb_P is generic for vP and dP.

I suggest using n_dPP, nvPP instead of i ... to aling it with the style of nP and do not confuse i with "input".

Ok, that's probably right. I went back and forth on that.

Also, when calling functions with identifiers which are different to the function definitions, it's useful to use named assigns which make it more clear and readable, like:

n_vPP, P, nextP = combP(P=vP, nP = len(vP), P = vP, r, A, _x, x, y, Y, P, nextP) #n_vPP instead of i_vPP

Ok. This is a work in progress, I will probably change it to: P, rootP, altP, nextP = comb_P... , where Ps are renamed as vP or dP.

Thanks!

[Twenkid]

Why vertex? A vertex on 4-pixel grid? Vertical? Just a fancy term? Vertices imply being part of polygons or graphs.

Yes, the polygon here is the quadrant. Vertex doesn't represent the whole quadrant, only the first corner of it.

To me the combined derivative feels more like a diagonal. Overall, a corner/vertex in 2x2 is kind of perverse to me, there's no "material"/space for variations. (I understand the "space" is in the pixel values and the comparison to the average.)

Indeed, shouldn't the average be computed per each coordinate of the sensory space? I.e. for a[0][0], a[1][0], a[2][0] ... ?

[boris-kz]

Pixels are only samples, the space is continuous. That's why vertex gradient is supposed to represent the whole 0-90 degree 2D span, not a specific 45 degree 1D value.

On Wed, Jun 28, 2017 at 9:54 AM, Todor Arnaudov notifications@github.com wrote:

Why vertex? A vertex on 4-pixel grid? Vertical? Just a fancy term? Vertices imply being part of polygons or graphs.

Yes, the polygon here is the quadrant. Vertex doesn't represent the whole quadrant, only the first corner of it.

To me the combined derivative feels more like a diagonal. Overall, a corner/vertex in 2x2 is kind of perverse to me, there's no "material"/space for variations. (I understand the "space" is in the pixels values and the comparison to the average.)

Indeed, shouldn't the average be computed per each coordinate of the sensory space? I.e. for a[0][0], a[1][0], a[2][0] ... ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/pull/2#issuecomment-311667018, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGUCJZM5eDNcpYfln80igJuGxoAgiks5sIlsfgaJpZM4OEoGz .

[Twenkid]

What about the average?

[boris-kz]

It's the average of two samples per vertex.

On Wed, Jun 28, 2017 at 10:08 AM, Todor Arnaudov notifications@github.com wrote:

What about the average?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/pull/2#issuecomment-311670889, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGTGgAv3z2w0Zs1zHjHj9bgsUr1EFks5sIl5hgaJpZM4OEoGz .

[Twenkid]

But how it's computed, is it per each respective coordinates (0,1) (2,3) ... for the history of samples.

[boris-kz]

Yes, it's per pixel. Pixel tuple: pri_p, m, d, my, dy. I don't actually average, gradient = m+my - A, or d+dy, because I only care about sign, which would be same.

On Wed, Jun 28, 2017 at 10:16 AM, Todor Arnaudov notifications@github.com wrote:

But how it's computed, is it per each respective coordinates (0,1) (2,3) ... for the history of samples.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/pull/2#issuecomment-311673113, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGRn4RJBg07HbMt-vzIAnYFv7bPb5ks5sImAhgaJpZM4OEoGz .

[Twenkid]

Deque

Rule: If a traversal with pop will follow, create the structure as a deque (var = deque(), instead of var = []), and fill it with var.appendleft(item) instead of var.append(item) - the firstly added elements will appear last in incremental traversal.

However, pop removes the element, therefore the structure can be used only once.*

In ycomp P is used in both branches vP, dP. _P_is traversed, items are extracted, then pushed to another array and then returned in the other arrays or something, but I still do partly "lazy evaluation".

(For such purposes call graphs and other AST-builders and code-visualisation tools would be useful, unfortunately I've been busy and haven't worked this out yet. I may do it by hand initially if I don't manage to automate it soon.)

By the way, do you know that for straight traversal of the whole list you may use just:

for _P in List:
      ...  
    #_P will get _P[0], _P[1] etc.

Instead of:

for i in range(len(olp_)) etc. The "range" is appropriate if only a part of the list would be traversed, or there would be skipped elements/step != 1 etc., asrange(start, end, step) ...

[boris-kz]

Rule: If a traversal with pop will follow, create the structure as a deque (var = deque(), instead of var = []), and fill it with var.appendleft(item) instead of var.append(item) - the firstly added elements will appear last in incremental traversal.

Ok. This is faster than list only for popleft? Can I do deque += list, or assign list = deque?

By the way, do you know that for straight traversal of the whole list you

may use just:

for _P in List: ...

_P will get _P[0], _P[1] etc.

Thanks!

[Twenkid]

Rule: If a traversal with pop will follow, create the structure as a deque (var = deque(), instead of var = []), and fill it with var.appendleft(item) instead of var.append(item) - the firstly added elements will appear last in incremental traversal.

Ok. This is faster than list only for popleft?

And there's appendleft, which doesn't exist for list and would require to shift all the elements right. I suppose traversal in both directions is similarly faster for deque.

Also, I'll remind that that "pop"/reading of an element is a tiny operation out of many others inside the loops.

Can I do deque += list, or assign list = deque?

Yes.

In general you can assign anything to any variable, but that'd change its type.

An import is needed:

from collections import deque

>>> d = deque([1,2,3])
>>> d
deque([1, 2, 3])
>>> list = [5,6,7]
>>> list
[5, 6, 7]
>>> d+=list
>>> d
deque([1, 2, 3, 5, 6, 7])
>>> list = d
>>> list
deque([1, 2, 3, 5, 6, 7])
>>>

[boris-kz]

Ok. But I really need to focus on the algorithm itself.

I am defining computation of matches between 1D P vars, PM: combined match between 1D patterns, and redundancy, all in comb_P.

Do you want to talk about that?

On Thu, Jun 29, 2017 at 7:25 AM, Todor Arnaudov notifications@github.com wrote:

Rule: If a traversal with pop will follow, create the structure as a deque (var = deque(), instead of var = []), and fill it with var.appendleft(item) instead of var.append(item) - the firstly added elements will appear last in incremental traversal.

Ok. This is faster than list only for popleft?

And there's appendleft, which doesn't exist for list and would require to shift all the elements right. I suppose traversal in both directions is similarly faster for deque, but this pop is a tiny operation out of many others inside the loops.

Can I do deque += list, or assign list = deque?

Yes.

In general you can assign anything to any variable, but that'd change its type.

An import is needed:

from collections import deque

d = deque([1,2,3]) d deque([1, 2, 3]) list = [5,6,7] list [5, 6, 7] d+=list d deque([1, 2, 3, 5, 6, 7]) s = d s deque([1, 2, 3, 5, 6, 7])

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/pull/2#issuecomment-311938383, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGXBauLSphhhJWnwtL-mdZMTVkE2wks5sI4magaJpZM4OEoGz .

[Twenkid]

OK, let's talk.

[boris-kz]

In 30 min?

On Jun 30, 2017 5:53 AM, "Todor Arnaudov" notifications@github.com wrote:

OK, let's talk.

On Friday, June 30, 2017, boris-kz notifications@github.com wrote:

Ok. But I really need to focus on the algorithm itself.

I am defining computation of matches between 1D P vars, PM: combined match between 1D patterns, and redundancy, all in comb_P.

Do you want to talk about that?

On Thu, Jun 29, 2017 at 7:25 AM, Todor Arnaudov < notifications@github.com> wrote:

Rule: If a traversal with pop will follow, create the structure as a deque (var = deque(), instead of var = []), and fill it with var.appendleft(item) instead of var.append(item) - the firstly added elements will appear last in incremental traversal.

Ok. This is faster than list only for popleft?

And there's appendleft, which doesn't exist for list and would require to shift all the elements right. I suppose traversal in both directions is similarly faster for deque, but this pop is a tiny operation out of many others inside the loops.

Can I do deque += list, or assign list = deque?

Yes.

In general you can assign anything to any variable, but that'd change its type.

An import is needed:

from collections import deque

d = deque([1,2,3]) d deque([1, 2, 3]) list = [5,6,7] list [5, 6, 7] d+=list d deque([1, 2, 3, 5, 6, 7]) s = d s deque([1, 2, 3, 5, 6, 7])

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/pull/2#issuecomment-311938383, or mute the thread < https://github.com/notifications/unsubscribe-auth/AUAXGXBauLSphhhJWnwtL- mdZMTVkE2wks5sI4magaJpZM4OEoGz

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.< https://ci4.googleusercontent.com/proxy/5Bw79BUoQdUjFkuIbDt2CF8rSAMQUS EasQ5EuRTpZMNg0tbLZsuYc_zMmEBX92P0eZGLvld147583rISNJvt FuLtmzVQcYXAFdpr3Fgq0CoIIYL96EYnbHgEO1xsrNr0xy- WCqYPNv1BCIlMU4irsPXQpQr60Q=s0-d-e1-ft#https://github.com/ notifications/beacon/AWSP2Lt_HjrACo2rsiaH-z_XDJ-6c-- Aks5sJGS_gaJpZM4OEoGz.gif

-- Tosh, http://artificial-mind.blogspot.com

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/pull/2#issuecomment-312226622, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGZPGBLenL_4nyufiN8foqaOgq1l2ks5sJMWpgaJpZM4OEoGz .

[Twenkid]

More likely in the evening.

[boris-kz]

?

On Fri, Jun 30, 2017 at 7:44 AM, Todor Arnaudov notifications@github.com wrote:

More likely in the evening.

On Friday, June 30, 2017, boris-kz notifications@github.com wrote:

In 30 min?

On Jun 30, 2017 5:53 AM, "Todor Arnaudov" notifications@github.com wrote:

OK, let's talk.

On Friday, June 30, 2017, boris-kz notifications@github.com wrote:

Ok. But I really need to focus on the algorithm itself.

I am defining computation of matches between 1D P vars, PM: combined match between 1D patterns, and redundancy, all in comb_P.

Do you want to talk about that?

On Thu, Jun 29, 2017 at 7:25 AM, Todor Arnaudov < notifications@github.com> wrote:

Rule: If a traversal with pop will follow, create the structure as a deque (var = deque(), instead of var = []), and fill it with var.appendleft(item) instead of var.append(item) - the firstly added elements will appear last in incremental traversal.

Ok. This is faster than list only for popleft?

And there's appendleft, which doesn't exist for list and would require to shift all the elements right. I suppose traversal in both directions is similarly faster for deque, but this pop is a tiny operation out of many others inside the loops.

Can I do deque += list, or assign list = deque?

Yes.

In general you can assign anything to any variable, but that'd change its type.

An import is needed:

from collections import deque

d = deque([1,2,3]) d deque([1, 2, 3]) list = [5,6,7] list [5, 6, 7] d+=list d deque([1, 2, 3, 5, 6, 7]) s = d s deque([1, 2, 3, 5, 6, 7])

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/pull/2#issuecomment-311938383, or mute the thread < https://github.com/notifications/unsubscribe- auth/AUAXGXBauLSphhhJWnwtL- mdZMTVkE2wks5sI4magaJpZM4OEoGz

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.< https://ci4.googleusercontent.com/proxy/5Bw79BUoQdUjFkuIbDt2CF8rSAMQUS EasQ5EuRTpZMNg0tbLZsuYc_zMmEBX92P0eZGLvld147583rISNJvt FuLtmzVQcYXAFdpr3Fgq0CoIIYL96EYnbHgEO1xsrNr0xy- WCqYPNv1BCIlMU4irsPXQpQr60Q=s0-d-e1-ft#https://github.com/ notifications/beacon/AWSP2Lt_HjrACo2rsiaH-z_XDJ-6c-- Aks5sJGS_gaJpZM4OEoGz.gif

-- Tosh, http://artificial-mind.blogspot.com

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/pull/2#issuecomment-312226622, or mute the thread < https://github.com/notifications/unsubscribe-auth/AUAXGZPGBLenL_ 4nyufiN8foqaOgq1l2ks5sJMWpgaJpZM4OEoGz

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.< https://ci4.googleusercontent.com/proxy/FQmeNc_3V0cxTQOlpvS9s0xFPqUup8L- 4nl99eSF-yTQLBdmzCZdrDRDpiRHAhoMbvO4rndZtzXqLbkcSdsS8mVpQ7s30LKiQP44V FCddCeo3RPA65d5tMCIG1pvNPNpB2t63RUcHBWr8JHPBj15Fi7DAGmbuA= s0-d-e1-ft#https://github.com/notifications/beacon/ AWSP2NXIc3rY8XgZ1nsHkpY5rPhDfcZvks5sJNHagaJpZM4OEoGz.gif

-- Tosh, http://artificial-mind.blogspot.com

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/pull/2#issuecomment-312246582, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGSiUEVcgbCc00rb1aoNYf-uxizi1ks5sJN-vgaJpZM4OEoGz .

[Twenkid]

I can't now, maybe 17:xx-18 or later , I'll be in the hangouts.

[Twenkid]

Update: @boris-kz

#after refactoring, deque is not needed for comp and ycomp - iterator reads both t, _t in the for loop
def comp(p_):  # comparison of consecutive pixels in a scan line forms tuples: pixel, match, difference

    t_ = [] #.deque()
    pri_p = p_[0] #was p_.poplelf(), but p_ is a list, not deque # no d, m at x=0
    t = pri_p; 
    t_.append(t) #right append

    for p in p_[1:] :  # new pixel, comp to prior pixel, vs. for x in range(1, X)
        d = p - pri_p  # lateral difference between consecutive pixels
        m = min(p, pri_p)  # lateral match between consecutive pixels
        t = p, d, m 
        t_.append(t) #right append, deque; 
        pri_p = p
    return t_

(...)

 pri_p = t_[0] #.popleft()  # no d, m at x=0
    for t, _t in zip(t_, _t_):  # compares vertically consecutive tuples, resulting derivatives end with 'y' and 'g':
    #straight traversal for both lists - now pop or popleft are not needed

        p, d, m = t
        _p, _d, _m = _t

zip example:

t_ = [(1,5,11),(2,10, 15), (3,4,5)]
_t_ = [(124,15, 5), (199, 20, 4), (240, 45, 31)]
for t, _t in zip(t_, _t_):
  print(t, _t)
  p, d, m = t
  _p, _d, _m = _t
  print(p,d,m)
  print(_p,_d,_m)

https://docs.python.org/3.3/library/functions.html#zip

This is again just an edit box for my commit, sorry but this GUI feels awkward. I didn't find a simple button for synchronizing my branch with your master, I copied the code... Now when trying to do pull-request I couldn't compare your "master" to my "master", but only to "Twenkid-patch".

(I'll have to use the command line or PyCharm, but not yet)

[boris-kz]

On Sun, Jul 2, 2017 at 4:39 PM, Todor Arnaudov notifications@github.com wrote:

Update: @boris-kz https://github.com/boris-kz

after refactoring, deque is not needed for comp and ycomp

I meant level_1 Todor. level_1_2D is work in progress, so it's difficult to simulate.

• iterator reads both t, t in the for loop def comp(p):

Yes, but is it faster than deque?

was p_.poplelf(), but p_ is a list, not deque

Yes, I left it as a list to try, and PyCharm was fine with it. I think it inferred the type.

zip example:

t_ = [(1,5,11),(2,10, 15), (3,4,5)] t = [(124,15, 5), (199, 20, 4), (240, 45, 31)] for t, t in zip(t, t): print(t, _t) p, d, m = t _p, _d, _m = _t print(p,d,m) print(_p,_d,_m)

Do I need zip or for p in p_: will do?

(I'll have to use the command line or PyCharm, but not yet)

Right.

[Twenkid]

On Sun, Jul 2, 2017 at 4:39 PM, Todor Arnaudov notifications@github.com wrote:

Update: @boris-kz https://github.com/boris-kz

after refactoring, deque is not needed for comp and ycomp

I meant level_1 Todor. level_1_2D is work in progress, so it's difficult to simulate.

OK

• iterator reads both t, t in the for loop def comp(p):

Yes, but is it faster than deque?

I assume it's similar or faster, I'll do tests later with different kinds of iterations. deque was critical if .pop was used due to the reverse order of traversal.

t_ = [(1,5,11),(2,10, 15), (3,4,5)] t = [(124,15, 5), (199, 20, 4), (240, 45, 31)] for t, t in zip(t, t): print(t, _t) p, d, m = t _p, _d, _m = _t print(p,d,m) print(_p,_d,_m)

Do I need zip or for p in p_: will do?

zip is for iterating over many sources, read in the same loop, for one input it's not needed

As of PyCharm it's possible to be doing lazy evaluation sometimes (partial eval), I'll try to test some cases of deliberately wrong usages to see how it would report within the IDE.

-- Tosh, http://artificial-mind.blogspot.com

[Twenkid]

Performance tests: @boris-kz

https://github.com/Twenkid/CogAlg/commit/e11bfb65e245fee8e4c7a89ac0cb24c0f583b174

Conclusion: avoid "pop" and use simple iterators: "for i in X" and zipped: "for i,j in zip(X,Y)"

The iteration with zip(t_, _t_)in the testbench was two times faster than if one of the lists was read by .pop

[boris-kz]

Thanks!

On Mon, Jul 3, 2017 at 1:01 PM, Todor Arnaudov notifications@github.com wrote:

Performance tests:

Twenkid/CogAlg@e11bfb6 https://github.com/Twenkid/CogAlg/commit/e11bfb65e245fee8e4c7a89ac0cb24c0f583b174

Conclusion: avoid "pop" and use simple iterators: "for i in X" and zipped: "for i,j in zip(X,Y)"

The iteration with zip(t_, t) in the testbench was two times faster than if one of the lists was read by .pop

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/pull/2#issuecomment-312694629, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGTSKwLVnaBVw8wUEmnGtfezUaYzFks5sKR5TgaJpZM4OEoGz .

[Twenkid]

@boris-kz

Is this code In level_1, correct?

def inc_rng(...):
 ...
 ip_ = p_  # to differentiate from new p_
...
 for x in range(r+1, X):
        p, fd, fv = ip_[x]       # compared to a pixel at x-r-1:
        pp, pfd, pfv = ip_[x-r]  # previously compared p(ignored), its fd, fv to next p

ip_ is supposed to be an array containing a line of pixels, 1 sample is one item, isn't it?

If so I don't understand why fd, fv and pfd, pfv are read from there?

Also, it's confusing that it runs - now that I try to read that way in interactive editor (expecting to read a sequence of consecutive values), it fails, because it expects tuples.

This fails:

>>> data = [0,1,2,3,4,5,6]
>>> x=3
>>> p, fd, fv = data[x]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'int' object is not iterable

While this passes:

>>> t = (1,2,3)
>>> ip_ = []
>>> ip_.append(t)
>>> ip_.append( (4,5,6))
>>> ip_.append( (7,8,9))
>>> x = 1
>>> p, fd,fv = ip_[x]
>>> p, fd, fv
(4, 5, 6)

Another important note is that in Python lists shouldn't be directly assigned, because they are treated as pointers and the new identifier becomes just an alias to the same data structure.

That's why:

ip_ = p_  

Should be:

ip_ = p_.copy()  

(Also for ip_ = d_  in inc_der)

Or else the following happens:

>>> p_=[1,2,3]
>>> ip_=p_
>>> ip_
[1, 2, 3]
#So far it seems OK, but not until we change the list:
>>> ip_.append(9)
>>> p_
[1, 2, 3, 9]
>>> ip_
[1, 2, 3, 9]
#That's the right way to copy a list:
>>> new_ = ip_.copy()
>>> ip_.append(7)
>>> ip_
[1, 2, 3, 9, 7]
>>> new_
[1, 2, 3, 9]

[boris-kz]

Is this code In level_1, correct?

def incrng(...): ... ip = p # to differentiate from new p ... for x in range(r+1, X): p, fd, fv = ip[x] # compared to a pixel at x-r-1: pp, pfd, pfv = ip[x-r] # previously compared p(ignored), its fd, fv to next p

ip_ is supposed to be an array containing a line of pixels, 1 sample is one item, isn't it?

If so I don't understand why fd, fv and pfd, pfv are read from there?

Because this is a recursion within vP, not an initial comp ip is produced prior recursions of comp (vs. p produced by current comp), thus it contains tuples of pixel + fd + fv.

Also, it's confusing that it runs - now that I try to read that way in

interactive editor (expecting to read a sequence of consecutive values), it fails, because it expects tuples.

It should.

Another important note is that in Python *lists shouldn't be directly

assigned, because they are treated as pointers and the new identifier becomes just an alias to the same data structure.*

That's what I want. This is not copying, I simply rename the list so I can use the same comp (which uses the original name to generate new p) over it. That's recursion. ip should not be modified / appended again.

[Twenkid]

This style of coding is intentionally confusing, especially type changing within the same container. It should be more explicitly marked.

What do you mean "to differentiate":

p_ ... #input parameter

ip = p # to differentiate from new p_ ... pris, I, D, V, rv, olp, p, olp_ = 0, 0, 0, 0, 0, 0, [], [] # tuple vP=0

Well, the test below approves that by aliasing p, the new p = [] is assigned to another variable, although before that ip and p are supposed to point to the same container. That might be correct for the algorithm, but has confusing semantics, one expects that input parameters map to the input.

This code:

def inc_rng(a,b, p_):
    print("inc_rng")    
    ip_ = p_  # to differentiate from new p_
    print("after aliasing")
    print("p_ = ", p_)
    print("ip_ = ", ip_)
    pri_s, I, D, V, rv, olp, p_, olp_ = 0, 0, 0, 0, 0, 0, [], [] 
    print("inc_rng")
    print("p_ = ", p_)
    print("ip_ = ", ip_)
    return p_, ip_

x = 5; y = 1;
p_ = [1,2,3,4,5] 
print(p_)
p1_ , ip1_= inc_rng(x, y , p_)
print("caller scope")
print(p1_)
print(ip1_)

Suggests that ip_ is preserved

inc_rng_1.py
[1, 2, 3, 4, 5]
inc_rng
after aliasing
p_ =  [1, 2, 3, 4, 5]
ip_ =  [1, 2, 3, 4, 5]
inc_rng
p_ =  []
ip_ =  [1, 2, 3, 4, 5]
caller scope
[]
[1, 2, 3, 4, 5]

[boris-kz]

This style of coding is intentionally confusing, especially type changing within the same container.

It is intentionally concise

It should be more explicitly marked.

Do that

What do you mean "to differentiate":

p_ ... #input parameter

ip = p # to differentiate from new p_

Rename to allow for recursion: reuse of comp.