comp_p_draft - aggregation of variables from a different type and norm deriv?

[Twenkid]

I assume that the summations of the vars in CogAlg in general maps to the old concept of "aggregation" for reduction of the resolution of the input, losing the lowest bits?

In compP You are adding match and difference for the variables of the current patterns, however after transforming the initial structure coming from the blobs:

https://github.com/boris-kz/CogAlg/blob/bf91d04197960db7e39d06d3e7643d1c63f4a83f/frame_2D_alg/comp_P_draft.py

s, [min_x, max_x, min_y, max_y, xD, abs_xD, Ly], [L, I, G, Dx, Dy, abs_Dx, abs_Dy], root_ = blob

Adding m,d to the new pattern generates variables of the style mX, dX... :

Pm, Pd, mx, dx, mL, dL, mI, dI, mD, dD, mDy, dDy, mM, dM, mMy, dMy, div_f, nvars = P_ders

Extended with two auxilliary variables, div_f and nvars (normalized vars).

Also, I see the "normalized" and a "norm" in general are supposed to be something about division or fitting within a "normal"/canonical range; also, in a broader view, that's related to the vars related to redundancy rdn and filter (ave) adjustments with multipliers, which also "normalize" - compensate for different lengths, number of vars etc.

That makes sense for "incrementallity", "projecting" the aggregated lower-resolution variables to lower level patterns, especially if same-type vars are compared.

You mention "match deviation" in the header, but currently I see some ratios (some in comments), the rest seems normal comparison operators between two variables of two patterns, but I don't see the base line to which a deviation is computed?

Also I assume a meaningful sum of deviation would count if all variables from all types are within the same range/scale, but are they supposed to be? Thus the question:

How the summation of variables from different types is justified and what it means?

Such as:

   Pnm = xm + nmI + nmDx + nmDy  # defines norm_mPP, no ndx: single, but nmx is summed

        if Pm > Pnm: nmPP_rdn = 1; mPP_rdn = 0  # added to rdn, or diff alt, olp, div rdn?
        else: mPP_rdn = 1; nmPP_rdn = 0

        Pnd = xdd + ndI + ndDx + ndDy  # normalized d defines norm_dPP or ndPP

They resemble a sum of some variation of the basic 4-tuples of patterns, but this is a scalar.

If xm, xdd were also "normalized" and say all were within the same range of (min,max) as the other in the expression, then summing them and comparing them to a "norm sum" would make sense (say 1 + 1 + 1 +1 is max, or some range+range...).

However the first is xmatch (of coordinates?...), the other mDx etc.

Also, I assume that the spawning and collection of all those variables is in order to compare them in another step, to allow each of them to be compared and used for another partitioning etc., until the magnitudes of the variables or the lengths of the containers run out of "valuable" magnitude.

If it keeps going like that and doubling the number, it is asking for at least some partial code generation anyway.

Or/and/eventually starting to compare some "normalized" matches of the different types of variables between each-other and eliminating or not comparing some with lower match in the following levels which fan-out more variables. That however would require additional "maps" and checks per variable.

[boris-kz]

I assume that the summations of the vars in CogAlg in general maps to the old concept of "aggregation" for reduction of the resolution of the input, losing the lowest bits?

It's a reduction of positional resolution, but not per bit. Bit-wise resolution will be controlled by bit filters, which we don't use yet.

In compP You are adding match and difference for the variables of the current patterns, however after transforming the initial structure coming from the blobs:

https://github.com/boris-kz/CogAlg/blob/bf91d04197960db7e39d06d3e7643d1c63f4a83f/frame_2D_alg/comp_P_draft.py

s, [min_x, max_x, min_y, max_y, xD, abs_xD, Ly], [L, I, G, Dx, Dy, abs_Dx, absDy], root = blob

That structure is out of date

Adding m,d to the new pattern generates variables of the style mX, dX... :

Pm, Pd, mx, dx, mL, dL, mI, dI, mD, dD, mDy, dDy, mM, dM, mMy, dMy, div_f, nvars = P_ders

Extended with two auxilliary variables, div_f and nvars (normalized vars).

Also, I see the "normalized" and a "norm" in general are supposed to be something about division or fitting within a "normal"/canonical range; also, in a broader view, that's related to the vars related to redundancy rdn and filter (ave) adjustments with multipliers, which also "normalize" - compensate for different lengths, number of vars etc.

norm here refers to normalizing for length L only

You mention "match deviation" in the header, but currently I see some ratios (some in comments), the rest seems normal comparison operators between two variables of two patterns, but I don't see the base line to which a deviation is computed?

The base is ave, line 61 in frame_blobs: g = np.abs(dy) + np.abs(dx__) - ave # deviation of gradient, initially approximated as |dy| + |dx|

How the summation of variables from different types is justified and what it means?

Such as:

Pnm = xm + nmI + nmDx + nmDy # defines norm_mPP, no ndx: single, but nmx is summed

   if Pm > Pnm: nmPP_rdn = 1; mPP_rdn = 0  # added to rdn, or diff alt, olp, div rdn?
   else: mPP_rdn = 1; nmPP_rdn = 0

   Pnd = xdd + ndI + ndDx + ndDy  # normalized d defines norm_dPP or ndPP

They resemble a sum of some variation of the basic 4-tuples of patterns, but this is a scalar.

All variables are derived from pixels with conserved resolution, so their predictive value corresponds to magnitude, to the same extent as in pixels Match is a reduction of represented magnitude, and difference is a complementary. So, their value is also proportional to magnitude, regardless of type.

If xm, xdd were also "normalized" and say all were within the same range of (min,max) as the other in the expression, then summing them and comparing them to a "norm sum" would make sense (say 1 + 1 + 1 +1 is max, or some range+range...).However the first is xmatch (of coordinates?...), the other mDx etc.

Resolution of coordinates is supposed to be adjusted so that their average magnitude and predictive value approximates that of inputs Initial xm is the extent of overlap (full x match), but I will probably change that to inverse deviation normalized for L: ave_dx - overlap_L / offset_L, or something like that.

Also, I assume that the spawning and collection of all those variables is in order to compare them in another step, to allow each of them to be compared and used for another partitioning etc., until the magnitudes of the variables or the lengths of the containers run out of "valuable" magnitude.

Yes

If it keeps going like that and doubling the number, it is asking for at least some partial code generation anyway.

The only code generation I can justify in principle is macro-recursion, which will increment algorithm of higher levels, and corresponding feedback, which will prune | extend algorithm of lower levels. Micro-recursion within a level is only reusing the same code.

Or/and/eventually starting to compare some "normalized" matches of the different types of variables between each-other and eliminating or not comparing some with lower match in the following levels which fan-out more variables. That however would require additional "maps" and checks per variable.

That's a feedback, we don't have higher levels to send it from it yet.

[Twenkid]

I assume that the summations of the vars in CogAlg in general maps to the old concept of "aggregation" for reduction of the resolution of the input, losing the lowest bits?

It's a reduction of positional resolution, but not per bit. Bit-wise resolution will be controlled by bit filters, which we don't use yet.

Is it possible to give an example related to current code?

norm here refers to normalizing for length L only

i.e. var/L? (L, Ly, _L, _Ly) ...

You mention "match deviation" in the header, but currently I see some ratios (some in comments), the rest seems normal comparison operators between two variables of two patterns, but I don't see the base line to which a deviation is computed? The base is ave, line 61 in frame_blobs: g = np.abs(dy) + np.abs(dx__) - ave # deviation of gradient, initially approximated as |dy| + |dx|

OK, I've seen this usage, but expected another concept.

How the summation of variables from different types is justified and what it means?

Such as:

Pnm = xm + nmI + nmDx + nmDy # defines norm_mPP, no ndx: single, but nmx is summed

if Pm > Pnm: nmPP_rdn = 1; mPP_rdn = 0 # added to rdn, or diff alt, olp, div rdn? else: mPP_rdn = 1; nmPP_rdn = 0

Pnd = xdd + ndI + ndDx + ndDy # normalized d defines norm_dPP or ndPP

They resemble a sum of some variation of the basic 4-tuples of patterns, but this is a scalar.

(1)

All variables are derived from pixels with conserved >resolution, so their predictive value corresponds to magnitude, to the same extent as in pixels

(2)

Match is a reduction of represented magnitude, and difference >is a complementary. So, their value is also proportional to magnitude, regardless of type.

(3)

If xm, xdd were also "normalized" and say all were within the same range of (min,max) as the other in the expression, then summing them and comparing them to a "norm sum" would make sense (say 1 + 1 + 1 +1 is max, or some range+range...).However the first is xmatch (of coordinates?...), the other mDx etc.

Resolution of coordinates is supposed to be adjusted so that >their average magnitude and predictive value approximates that >of inputs

OK, I think these are good insights overall for CogAlg, clarified like that. Is that related to the section:

"As distinct from autoencoders (current mainstay in unsupervised learning), there is no need for decoding: comparison is done on each level, whose output is also fed back to filter lower levels."

i.e. that the process of unfolding of the higher level patterns eventually produces the pixel-representations.

Since the aggregated variables, big letters, cover wider span etc., when converted to smaller spans (divided) they can be "projected" as shorter span variables or eventually as a single-pixel one. The bigger-span variables correspond to a bigger predictive value both as magnitude and as they are produced by comparing a wider range, they have higher certainty than a single, current pixel.

Also, does "predictive value" correspond to the literal value - a specific magnitude (1.46, 2.955, 124.56...) of the variables of the respective lower level?

And/or as their part/fraction of the whole expanded/unfolded expression which eventually computes the magnitude of the respective lower level/higher level/feedback variable?

Initial xm is the extent of overlap (full x match), but I will probably change that to inverse deviation normalized for L: ave_dx - overlap_L / offset_L, or something like that.

Thus (just marking it):

1) deviation = var - ave "inverse deviation" = ave - var

2) "normalized for " is the ratio

norm = var/normalized_to

3) extent of overlap (full x match) etc.

That code?

xm = (x0 + L-1) - _x0   # x olp, ave - xd -> vxP: low partial
if x0 > _x0: xm -= x0 - _x0  # vx only for distant-P comp? no if mx > ave, only PP termination by comp_P? distance, or relative: olp_L / min_L (dderived)?

min_L is min(L, _L), but what are overlap_L and offset_L? (since they are lenghts - 1D) and the above formula is x olp?

Is this correct:

Coordinate(overlap, offset) maps to Input(match, miss)?

At least related to, because overlap is one length, while offset could be one both sides.

If following the logic of match (the smaller comparand), offset is supposed to be the smaller one between ((x0,_x0), (xn,_xn))? Overlap is one value.

If it keeps going like that and doubling the number, it is asking for at least some partial code generation anyway.

The only code generation I can justify in principle is macro-recursion, which will increment algorithm of higher levels, and corresponding feedback, which will prune | extend algorithm of lower levels. Micro-recursion within a level is only reusing the same code.

In this post I meant mostly tools for the developers. For the other thread also that would be a start.

Here, like taking this:

Pm, Pd, mx, dx, mL, dL, mI, dI, mD, dD, mDy, dDy, mM, dM, mMy, dMy, div_f,

And generating:

mPm, dPm, mPd, dPd, mmx, dmx, mdx, ddx, ... = ...

also summing etc.

Also assisted checking whether all variables are compared etc.

It's simple, while may save some mistakes and tediousness.

[boris-kz]

It's a reduction of positional resolution, but not per bit. Bit-wise resolution will be controlled by bit filters, which we don't use yet.

Is it possible to give an example related to current code?

Bit filters will adjust most and least significant bits per input parameter. All summed params have positional resolution of their structure (P, blob, etc.), which lower than that of constituent pixels

norm here refers to normalizing for length L only

i.e. var/L?

Yes.

Resolution of coordinates is supposed to be adjusted so that >their average magnitude and predictive value approximates that >of inputs

OK, I think these are good insights overall for CogAlg, clarified like that. Is that related to the section:

"As distinct from autoencoders (current mainstay in unsupervised learning), there is no need for decoding: comparison is done on each level, whose output is also fed back to filter lower levels."

i.e. that the process of unfolding of the higher level patterns eventually produces the pixel-representations.

I don't see a relationship. Feedback (ave and other filters) is applied to pixel comp, but it is level-wide, not pixel-specific.

Since the aggregated variables, big letters, cover wider span etc., when converted to smaller spans (divided) they can be "projected" as shorter span variables or eventually as a single-pixel one. The bigger-span variables correspond to a bigger predictive value both as magnitude and as they are produced by comparing a wider range, they have higher certainty than a single, current pixel.

They are definitely more stable.

Also, does "predictive value" correspond to the literal value - a specific magnitude (1.46, 2.955, 124.56...) of the variables of the respective lower level?

not for brightness, but definitely for derivatives.

And/or as their part/fraction of the whole expanded/unfolded expression which eventually computes the magnitude of the respective lower level/higher level/feedback variable?

No, feedback is necessarily very coarse.

Thus (just marking it):

1.

deviation = var - ave "inverse deviation" = ave - var

yes

1.

"normalized for " is the ratio

norm = var/normalized_to

yes

1.

extent of overlap (full x match) etc.

That code?

xm = (x0 + L-1) - _x0 # x olp, ave - xd -> vxP: low partial if x0 > _x0: xm -= x0 - _x0 # vx only for distant-P comp? no if mx > ave, only PP termination by comp_P? distance, or relative: olp_L / min_L (dderived)?

min_L is min(L, _L), but what are overlap_L and offset_L? (since they are lenghts - 1D) and the above formula is x olp?

ok, it should be like this:

 xn = x0 + L-1
_xn = _x0 + _L-1
off_L = abs(x0 - _x0) + abs(xn - _xn)
olp_L = min(L, _L) - off_L
Xm = olp_L / off_L
Xd = (x0 + (L-1)//2) - (_x0 + (_L-1)//2)    # Xm and Xd are

L-normalized because individual pixel x m|d is binary

Is this correct:

Coordinate(overlap, offset) maps to Input(match, miss)?

Right

If following the logic of match (the smaller comparand), offset is supposed to be the smaller one between ((x0,_x0), (xn,_xn))? Overlap is one value.

it's the opposite, xm is overlap: common subset of x_ and x

If it keeps going like that and doubling the number, it is asking for at least some partial code generation anyway.

The only code generation I can justify in principle is macro-recursion, which will increment algorithm of higher levels, and corresponding feedback, which will prune | extend algorithm of lower levels. Micro-recursion within a level is only reusing the same code.

In this post I meant mostly tools for the developers.

There are lots of people working on that.

Here, like taking this:

Pm, Pd, mx, dx, mL, dL, mI, dI, mD, dD, mDy, dDy, mM, dM, mMy, dMy, div_f,

I did that for clarity, actual code will be different

And generating:

mPm, dPm, mPd, dPd, mmx, dmx, mdx, ddx, ... = ...

also summing etc.

Also assisted checking whether all variables are compared etc.

It's simple, while may save some mistakes and tediousness.

You are talking about using generic names and operations, nothing special. We do that as needed, let me know if you see a way to simplify existing code.

[Twenkid]

ok, it should be like this:

xn = x0 + L-1 _xn = _x0 + _L-1 off_L = abs(x0 - _x0) + abs(xn - _xn) olp_L = min(L, _L) - off_L Xm = olp_L / off_L Xd = (x0 + (L-1)//2) - (_x0 + (_L-1)//2) # Xm and Xd are L-normalized because individual pixel x m|d is binary

So for the offset you sum the margins on the both sides.

That implies possible olp < 0 if I interpret it correctly - is this OK, what it means? Just that the overlap is smaller than the margins (offset)?

Say

Ooooooooooooo Oohhhhhhhhhho Ooooooaaooooo

off_L = 4+4 olp_L = min(2, 10) - off_L = -6 Xm = -6/8

What is the purpose of the //? The reason I see is that integer divisions reflect input coordinates and respective pixel/input comparisons by iterations, like the case here for x, L.

While gradients, angles, hypot are computed, compared and sorted only numerically, these magnitudes are not mapped to iterable integer coordinates.

And generating:

mPm, dPm, mPd, dPd, mmx, dmx, mdx, ddx, ... = ...

also summing etc.

Also assisted checking whether all variables are compared etc.

It's simple, while may save some mistakes and tediousness.

You are talking about using generic names and operations, nothing special.

I didn't claim it was special, but noticing patterns in the code.

We do that as needed, let me know if you see a way to simplify existing

code.

OK

—

You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/issues/29#issuecomment-482897246, or mute the thread https://github.com/notifications/unsubscribe-auth/AWSP2Dv5DZDnj0rW_aSRWEKsiToLjK9oks5vgmRSgaJpZM4cttV_ .

[boris-kz]

xn = x0 + L-1 _xn = _x0 + _L-1 off_L = abs(x0 - _x0) + abs(xn - _xn) olp_L = min(L, _L) - off_L Xm = olp_L / off_L Xd = (x0 + (L-1)//2) - (_x0 + (_L-1)//2) # Xm and Xd are L-normalized because individual pixel x m|d is binary

So for the offset you sum the margins on the both sides.

That implies possible olp < 0 if I interpret it correctly - is this OK, what it means? Just that the overlap is smaller than the margins (offset)?

Actually, my calculation above is incorrect How would you compute overlap?

What is the purpose of the //? The reason I see is that integer divisions reflect input coordinates and respective pixel/input comparisons by iterations, like the case here for x, L.

No special reason, just that precision is not important here.

[Twenkid]

What about just: olp = min(xn,_xn) - max(x0, _x0)

It seems to work correctly (as number of overlapping pixels, if that's the goal) for: x0 = 1; xn = 12; _x0 = 5; _xn =7 x0 = 5; xn = 7; _x0 = 1; _xn = 12 //replaced lines x0 = 2; xn = 3; _x0 = 2; _xn = 3 //one pixel exact match x0 = 2; xn = 4; _x0 = 2; _xn = 3 //one pixel, one margin

On Sun, Apr 14, 2019 at 4:48 AM Boris Kazachenko notifications@github.com wrote:

xn = x0 + L-1 _xn = _x0 + _L-1 off_L = abs(x0 - _x0) + abs(xn - _xn) olp_L = min(L, _L) - off_L Xm = olp_L / off_L Xd = (x0 + (L-1)//2) - (_x0 + (_L-1)//2) # Xm and Xd are L-normalized because individual pixel x m|d is binary

So for the offset you sum the margins on the both sides.

That implies possible olp < 0 if I interpret it correctly - is this OK, what it means? Just that the overlap is smaller than the margins (offset)?

Actually, my calculation above is incorrect How would you compute overlap?

What is the purpose of the //? The reason I see is that integer divisions reflect input coordinates and respective pixel/input comparisons by iterations, like the case here for x, L.

No special reason, just that precision is not important here.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/issues/29#issuecomment-482911126, or mute the thread https://github.com/notifications/unsubscribe-auth/AWSP2K_DCYwNvjt0bqoOUSJXoHbdEPZPks5vgojwgaJpZM4cttV_ .

[boris-kz]

Great, thanks Todor!

On Sat, Apr 13, 2019, 10:09 PM Todor Arnaudov notifications@github.com wrote:

What about just: olp = min(xn,_xn) - max(x0, _x0)

It seems to work correctly (as number of overlapping pixels, if that's the goal) for: x0 = 1; xn = 12; _x0 = 5; _xn =7 x0 = 5; xn = 7; _x0 = 1; _xn = 12 //replaced lines x0 = 2; xn = 3; _x0 = 2; _xn = 3 //one pixel exact match x0 = 2; xn = 4; _x0 = 2; _xn = 3 //one pixel, one margin

On Sun, Apr 14, 2019 at 4:48 AM Boris Kazachenko <notifications@github.com

wrote:

xn = x0 + L-1 _xn = _x0 + _L-1 off_L = abs(x0 - _x0) + abs(xn - _xn) olp_L = min(L, _L) - off_L Xm = olp_L / off_L Xd = (x0 + (L-1)//2) - (_x0 + (_L-1)//2) # Xm and Xd are L-normalized because individual pixel x m|d is binary

So for the offset you sum the margins on the both sides.

That implies possible olp < 0 if I interpret it correctly - is this OK, what it means? Just that the overlap is smaller than the margins (offset)?

Actually, my calculation above is incorrect How would you compute overlap?

What is the purpose of the //? The reason I see is that integer divisions reflect input coordinates and respective pixel/input comparisons by iterations, like the case here for x, L.

No special reason, just that precision is not important here.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/issues/29#issuecomment-482911126, or mute the thread < https://github.com/notifications/unsubscribe-auth/AWSP2K_DCYwNvjt0bqoOUSJXoHbdEPZPks5vgojwgaJpZM4cttV_

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/issues/29#issuecomment-482912262, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGTYZD4YzYvPyowbmCdMo_pGxOkMEks5vgo3bgaJpZM4cttV_ .

[Twenkid]

Bit filters will adjust most and least significant bits per input parameter. All summed params have positional resolution of their structure (P, blob, etc.), which lower than that of constituent pixels

What do you mean technically? A method for adjusting the effective and, more importantly, the interpreted range of the magnitudes of the variables?

What I understand is:

A) Bit-wise shifts (if integers): Shift Left multiplies by 2, Shift Right divides by 2. It would require additional range-adjusting parameter.

B) Floats can't be shifted, they can be divided and multiplied as for extending the range of E+... of the represented magnitudes by a range-adjusting parameter.

The highest resolution is of the raw input, so the maximum is having all bits legal. Reducing the lower bits is by Shift left/MUL

[boris-kz]

Yes, it will be done by shifting, or switching to different data types. Floats can be scaled too, it's just more complex. This is not a near-term concern.

On Sun, Apr 14, 2019 at 8:10 PM Todor Arnaudov notifications@github.com wrote:

Bit filters will adjust most and least significant bits per input parameter. All summed params have positional resolution of their structure (P, blob, etc.), which lower than that of constituent pixels

What do you mean technically? A method for adjusting the effective and, more importantly, the interpreted range of the magnitudes of the variables?

What I understand is:

A) Bit-wise shifts (if integers): Shift Left multiplies by 2, Shift Right divides by 2. It would require additional range-adjusting parameter.

B) Floats can't be shifted, they can be divided and multiplied as for extending the range of E+... of the represented magnitudes by a range-adjusting parameter.

The highest resolution is of the raw input, so the maximum is having all bits legal. Reducing the lower bits is by Shift left/MUL

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/issues/29#issuecomment-483070781, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGQ4fEgOfkJ3LQGHcEPBL3CYVnhMLks5vg8NrgaJpZM4cttV_ .

[Twenkid]

In special interface levels between the substantial ones - "Type converting levels"?

I understand that it is meaningful theoretically, that the ranges should be adjustable for the generality of the definitions and extension of the levels, and not to be limited by an initial format.

I guess in practice it may be needed when the 32-bit runs out for the integers. Then it has to adjust with Shift/Mul or/and to jump to 64-bit. It could also just start with 64-bit eventually, as with a Double.

The magnitudes of the derivatives will grow or shrink after many operations, but they lose precision after the span is expanded anyway. There are "long long long" types like __int128 in C++, but AFAIK the range and precision of double or 64-bit int is still considered big enough for scientific calculations, the scientific GPUs use 64-bit doubles.

[boris-kz]

It doesn't matter for predictable future Todor, we are constrained by hardware and languages, and will be working with canned input.

On Sun, Apr 14, 2019 at 9:01 PM Todor Arnaudov notifications@github.com wrote:

In special interface levels between the substantial ones - "Type converting levels"?

I understand that it is meaningful theoretically, that the ranges should be adjustable for the generality of the definitions and extension of the levels, and not to be limited by an initial format.

I guess in practice it may be needed when the 32-bit runs out for the integers. Then it has to adjust with Shift/Mul or/and to jump to 64-bit. It could also just start with 64-bit eventually, as with a Double.

The magnitudes of the derivatives will grow or shrink after many operations, but they lose precision after the span is expanded anyway. There are "long long long" types like __int128 in C++, but AFAIK the range and precision of double or 64-bit int is still considered big enough for scientific calculations, the scientific GPUs use 64-bit doubles.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/issues/29#issuecomment-483075560, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGeD4vN_W0-1lX2rYLuL0UlFDQC6Oks5vg8-EgaJpZM4cttV_ .

[Twenkid]

OK. @khanh93vn @boris-kz BTW, may I join a live discussion of yours some time? I don't mind being just a reader and listener at first, in order not to interfere your flow if I'm not prepared to join.

[Twenkid]

An error in the order in intra_comp_debug, form_P:

https://github.com/boris-kz/CogAlg/blob/888b7f775b6b6aef05178f765239498736785aa2/frame_2D_alg/intra_comp_debug.py#L81

  i, ncomp, dy, dx, g = derts_[1][0]  # first derts
  x0, L, I, Dy, Dx, G = x_start, 1, i, g, dy // ncomp, dx // ncomp # P params

==>

x0, L, I, Dy, Dx, G = x_start, 1, i, dy // ncomp, dx // ncomp, g # P params

[boris-kz]

Thanks Todor, I just posted a new version on my repo

On Mon, Apr 15, 2019 at 8:04 PM Todor Arnaudov notifications@github.com wrote:

An error in the order in intra_comp_debug, form_P:

https://github.com/boris-kz/CogAlg/blob/888b7f775b6b6aef05178f765239498736785aa2/frame_2D_alg/intra_comp_debug.py#L81

i, ncomp, dy, dx, g = derts_[1][0] # first derts x0, L, I, Dy, Dx, G = x_start, 1, i, g, dy // ncomp, dx // ncomp # P params

==>

x0, L, I, Dy, Dx, G = x_start, 1, i, dy // ncomp, dx // ncomp, g # P params

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/issues/29#issuecomment-483462087, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGXXWnYVQGD9hji-MrjYXkrnGuPCUks5vhROBgaJpZM4cttV_ .

[boris-kz]

So, the problem with live discussions is that we don't usually schedule them, and I don't think it's an option in messenger. But I will email you the text, as on the other thread. Then there is that fb group I just set up. Don't know about slack, I'd rather stick with platforms we already have. Skype has group chats, but scheduling is too much trouble.

On Mon, Apr 15, 2019 at 6:33 PM Todor Arnaudov notifications@github.com wrote:

OK. @khanh93vn https://github.com/khanh93vn @boris-kz https://github.com/boris-kz BTW, may I join a live discussion of yours some time? I don't mind being just a reader and listener at first, in order not to interfere your flow if I'm not prepared to join.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/issues/29#issuecomment-483443412, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGbzZ84MqwAE41D-kSgIKqpu1FnHKks5vhP5MgaJpZM4cttV_ .

[Twenkid]

Skype - I may join when I find you both active in the channel, whenever we cross and I could, just such a channel has to exist, I'll be notified if I am online etc.

One problem with fb is that it is suited for plain text, no code blocks (if there are no special groups for that). To me the Github issues are also a decent chat platform.

On Tuesday, April 16, 2019, Boris Kazachenko notifications@github.com wrote:

So, the problem with live discussions is that we don't usually schedule them, and I don't think it's an option in messenger. But I will email you the text, as on the other thread. Then there is that fb group I just set up. Don't know about slack, I'd rather stick with platforms we already have. Skype has group chats, but scheduling is too much trouble.

On Mon, Apr 15, 2019 at 6:33 PM Todor Arnaudov notifications@github.com wrote:

OK. @khanh93vn https://github.com/khanh93vn @boris-kz https://github.com/boris-kz BTW, may I join a live discussion of yours some time? I don't mind being just a reader and listener at first, in order not to interfere your flow if I'm not prepared to join.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/issues/29#issuecomment-483443412, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGbzZ84MqwAE41D- kSgIKqpu1FnHKks5vhP5MgaJpZM4cttV_ .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/issues/29#issuecomment-483476714, or mute the thread https://github.com/notifications/unsubscribe-auth/AWSP2KvseYYCuIrAbmb3x2B0kb15a9Cgks5vhSfigaJpZM4cttV_ .

[boris-kz]

Ok, I just created skype conversation CogAlg and added you and Khanh to it

On Mon, Apr 15, 2019 at 11:07 PM Todor Arnaudov notifications@github.com wrote:

Skype - I may join when I find you both active in the channel, whenever we cross and I could, just such a channel has to exist, I'll be notified if I am online etc.

One problem with fb is that it is suited for plain text, no code blocks (if there are no special groups for that). To me the Github issues are also a decent chat platform.

On Tuesday, April 16, 2019, Boris Kazachenko notifications@github.com wrote:

So, the problem with live discussions is that we don't usually schedule them, and I don't think it's an option in messenger. But I will email you the text, as on the other thread. Then there is that fb group I just set up. Don't know about slack, I'd rather stick with platforms we already have. Skype has group chats, but scheduling is too much trouble.

On Mon, Apr 15, 2019 at 6:33 PM Todor Arnaudov <notifications@github.com

wrote:

OK. @khanh93vn https://github.com/khanh93vn @boris-kz https://github.com/boris-kz BTW, may I join a live discussion of yours some time? I don't mind being just a reader and listener at first, in order not to interfere your flow if I'm not prepared to join.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/issues/29#issuecomment-483443412, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGbzZ84MqwAE41D- kSgIKqpu1FnHKks5vhP5MgaJpZM4cttV_ .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/issues/29#issuecomment-483476714, or mute the thread < https://github.com/notifications/unsubscribe-auth/AWSP2KvseYYCuIrAbmb3x2B0kb15a9Cgks5vhSfigaJpZM4cttV_

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/issues/29#issuecomment-483493743, or mute the thread https://github.com/notifications/unsubscribe-auth/AUAXGVLXALXCZQP8eekfXStMNPnT07uMks5vhT5cgaJpZM4cttV_ .

[Twenkid]

OK

[Twenkid]

Regarding the question about ncomp - that there are 4 default comparisons per dx,dy in comp_pixel, not two as in the past, in order to avoid unilateral artifacts. @khanh93vn, @boris-kz

May you give me an example with specific content of the pixels and what is expected as output values after comparison given that input?

Because when I do something like that:

a = np.ones(shape=(5, 5), dtype=int)
>>> a
array([[1, 1, 1, 1, 1],
       [1, 1, 1, 1, 1],
       [1, 1, 1, 1, 1],
       [1, 1, 1, 1, 1],
       [1, 1, 1, 1, 1]])
>>> c = a[2:, 1:-1] - a[:-2, 1:-1]

The output seems to me as if doing unilateral subtraction, current - previous from the raw input, as far as I expect what it should do:

>>> c
array([[0, 0, 0],
       [0, 0, 0],
       [0, 0, 0]])

or

 c = a[1:-1, 2:] - a[1:-1, :-2]
>>> c
array([[0, 0, 0],
       [0, 0, 0],
       [0, 0, 0]])

I suspect that maybe the expected or desirable behavior were that the calculations scroll down or something, to compute the difference in place for the line, they continue with the next line and use the updated previous one as current etc., thus to have double subtraction?

However the results I see above suggest me that one comparison is done, which is not taking into account the result of the previous one stored in place - if that was the goal at all...

If it was scrolling and using values in-place, I assume it would be:

1-1 = 0 and storing it on that line, in place.

Then the following line/row would be: 1-0 = 1

Then the following again: 1-1 = 0

Etc. Which is not "pretty" so may be it shouldn't do.

If the direction is taken into account and for the next lines it's +1, then 1-1+1 = 1 for all.

EDIT: I got it in the next comment. It's a sum of the differences, thus -(1-1) + +(1-1) = (-0) + (+0) = 0 :)

What is the expected output? I'm still misunderstanding something...

EDIT: I think I found out.

[Twenkid]

Well, I see I was using a bad example with equal values (dx,dy should be 0)

Another example makes more sense so I guess it's "correct".

a[1] = [2,2,2,2,2]
>>> a[2] = [4,4,4,4,4]
>>> a
array([[1, 1, 1, 1, 1],
       [2, 2, 2, 2, 2],
       [4, 4, 4, 4, 4],
       [1, 1, 1, 1, 1],
       [1, 1, 1, 1, 1]])
>>> dx = a[1:-1, 2:] - a[1:-1, :-2]
>>> dx
array([[0, 0, 0],
       [0, 0, 0],
       [0, 0, 0]])
>>> dy = a[2:, 1:-1] - a[:-2, 1:-1]
>>> dy
array([[ 3,  3,  3],
       [-1, -1, -1],
       [-3, -3, -3]])

[khanh93vn]

The purpose, I guess, is to build a little "model" at every pixel to predict (project) values of nearby pixels.

Consider p as the function of x and y: p = F(x, y). It's graph is 3D, 3 axes: x, y, p, this apparently isn't linear. Now, consider a region R of the graph. If R is small enough, it could approximately be considered linear (being a plane) inside R. Given that, p could be computed using a linear model:

                                             p = f(x, y) = p0 + a * x + b * y          with x, y ∈ R

This is what Boris called "fine grained search", well, at least that's what I think it is.

Now what we have are pixel values, with comparison, we could produce 2D partial derivatives, so projection of nearby pixel values: p = f(x, y) = _p + y dy + x dy with: p is the projected/predicted pixel _p is the current pixel x, y: difference in coordinate of p vs _p. Can be considered coordinate of p, with (0, 0) being at where _p is dx, dy: results from comparison

The notation here is a little different from normal notation, where derivatives are dp/dx, dp/dy. But that's where problem is: dx, dy we computed are not rates of change, so the above model is wrong. For it to be correct, dy, dx must be divided by ncomp, which is always 4 in the original comparison (this is a coincident, distances between compared pixels are always 1 there, so comparison with increasing range should be reconsidered too). We don't do that immediately because dividing with integer will cause increasing error over time, so ncomp is saved for future operations.

You could try this, using parameters generated from frame_blobs to try to predict further pixels (for example, the nearby diagonal pixels that haven't been compared yet) to see how well it performs. The further projected pixel is, the less accuracy it could produce.

[boris-kz]

Thanks Khanh,

The purpose, I guess, is to build a little "model" at every pixel to

predict (project) values of nearby pixels.

The model here is pixel itself: it predicts adjacent pixels to the extent that the environment is non-random. What we compute is derivatives: accuracy of that prediction, which becomes a higher-order and more tentative prediction. Because it is more tentative, its comparison (verification) is conditional on its magnitude. These secondary comparisons of incremental distance and derivation create deeper blob layers.

Now what we have are pixel values, with comparison, we could produce 2D partial derivatives, so projection of nearby pixel values: p = f(x, y) = _p + y dy + x dy with: p is the projected/predicted pixel _p is the current pixel

Almost, but you actually have here two _ps: _py and _px. And this is prediction from derivatives, which is different from prediction from the pixels. A combined or average prediction will be: p = ((_py + dy / 2) + (_px + dx / 2)) / 2 # for x=1 and y=1 As the search extends, so does the number of alternative predictions, to be combined.

[Twenkid]

Thanks for the clarificaton! I already had an idea about that (from a Khan explanations earlier) and before that (derivatives are about that in general), but that "4" had to be clarified.

p = ((_py + dy / 2) + (_px + dx / 2)) / 2

OK, and there the "4 goes", /2 (for y) and /2 for x.

For recovering (if direct) at longer distances it has to be adjusted by the number of pixels and the accumulated ncomps. When there is a discontinuity, though, it switches to next-level/higher span with a more complex expression/another representation.

Consider p as the function of x and y: p = F(x, y). It's graph is 3D, 3 axes: x, y, p, this apparently isn't linear. Now, consider a region R of the graph. If R is small enough, it could approximately be considered linear (being a plane) inside R. Given that, p could be computed using a linear model:

That reminds me of a discussion two years ago, about similarities between CogAlg to Clustering and the so called "Spectral Clustering" and pixel-grid-graph representations.

One good point there was about principles in clustering:

• similarity
• proximity
• good continuation

CogAlg was supposed to "define them at initial / maximal resolution, AKA accuracy, which matters".

In the hierarchical representation, there are "nodes" of different generality (depth), the tree of patterns that grow.

K: You could try this, using parameters generated from frame_blobs to try to predict further pixels (for example, the nearby diagonal pixels that haven't been compared yet) to see how well it performs. The further projected pixel is, the less accuracy it could produce.

B: Because it is more tentative, its comparison (verification) is conditional on its magnitude. These secondary comparisons of incremental distance and derivation create deeper blob layers.

And the expression G, Ga ... *cost etc. are to adjust for the "distance", in that context? However before a real run through a hierarchy with data, the cost of discontinuity/levels/"syntax" seem not really justified to me. I see multiplies by a number of derivatives, division by spans, but it is to be checked...

In intra_blob:

  for ablob in blob.sub_blob_:  # ablob is defined by ga sign
                Ga_rdn = 0
                G = ablob.Derts[-2][-1]  # I, Dx, Dy, G; Derts: current + higher-layers params, no lower layers yet
                Ga = ablob.Derts[-1][-1]  # I converted per layer, not redundant to higher layers I

Is that mean that Derts is supposed to be loaded with altering type derts (last one -1 has gradient Ga, while the previous -2 is G)? Thus [-2] is the "higher layer"?

Is that ave_blob in: if Ga > ave_blob: the mentioned ave_n_sub_blob (a length) from the FB discussion, or it's a regular magnitude? (Because G and Ga are not counts, but accumulated magnitudes from the derivatives?)

[boris-kz]

Thanks for the clarificaton! I already had an idea about that (from a Khan explanations earlier) and before that (derivatives are about that in general), but that "4" had to be clarified.

p = ((_py + dy / 2) + (_px + dx / 2)) / 2

OK, and there the "4 goes", /2 (for y) and /2 for x.

No, this combined prediction of different layers is for forming feedback, which is not actually done per pixel. It will be done on higher levels, which is not in code yet. ncomp is summed only within a layer.

For recovering (if direct) at longer distances it has to be adjusted by the number of pixels and the accumulated ncomps. When there is a discontinuity, though, it switches to next-level/higher span with a more complex expression/another representation.

At this point we are talking about deeper layers, with sub-blobs of a narrower span.

In the hierarchical representation, there are "nodes" of different

generality (depth), the tree of patterns that grow.

K: You could try this, using parameters generated from frame_blobs to try to predict further pixels (for example, the nearby diagonal pixels that haven't been compared yet) to see how well it performs. The further projected pixel is, the less accuracy it could produce.

B: Because it is more tentative, its comparison (verification) is conditional on its magnitude. These secondary comparisons of incremental distance and derivation create deeper blob layers.

And the expression G, Ga ... *cost etc. are to adjust for the "distance", in that context?

I don't like term "distance" as used in clustering, it's ambiguous and doesn't add anything here.

However before a real run through a hierarchy with data, the cost of discontinuity/levels/"syntax" seem not really justified to me. I see multiplies by a number of derivatives, division by spans, but it is to be checked...

In intra_blob:

for ablob in blob.subblob: # ablob is defined by ga sign Ga_rdn = 0 G = ablob.Derts[-2][-1] # I, Dx, Dy, G; Derts: current + higher-layers params, no lower layers yet Ga = ablob.Derts[-1][-1] # I converted per layer, not redundant to higher layers I

Is that mean that Derts is supposed to be loaded with altering type derts (last one -1 has gradient Ga, while the previous -2 is G)? Thus [-2] is the "higher layer"?

yes, and deeper layers of g type don't have i, instead they use i or g of higher-layer derts as i

Is that ave_blob in: if Ga > ave_blob: the mentioned ave_n_sub_blob (a length) from the FB discussion, or it's a regular magnitude? (Because G and Ga are not counts, but accumulated magnitudes from the derivatives?)

I will post a new edit soon

[Twenkid]

In the hierarchical representation, there are "nodes" of different

generality (depth), the tree of patterns that grow.

K: You could try this, using parameters generated from frame_blobs to try to predict further pixels (for example, the nearby diagonal pixels that haven't been compared yet) to see how well it performs. The further projected pixel is, the less accuracy it could produce.

B: Because it is more tentative, its comparison (verification) is conditional on its magnitude. These secondary comparisons of incremental distance and derivation create deeper blob layers.

And the expression G, Ga ... *cost etc. are to adjust for the "distance", in that context?

I don't like term "distance" as used in clustering, it's ambiguous and doesn't add anything here.

I used it in the context above - bigger span and diminishing effect of more distant pixels (more other pixels affect current), if they are taken individually/their small span.

However before a real run through a hierarchy with data, the cost of discontinuity/levels/"syntax" seem not really justified to me. I see multiplies by a number of derivatives, division by spans, but it is to be checked...

In intra_blob:

for ablob in blob.subblob: # ablob is defined by ga sign Ga_rdn = 0 G = ablob.Derts[-2][-1] # I, Dx, Dy, G; Derts: current + higher-layers params, no lower layers yet Ga = ablob.Derts[-1][-1] # I converted per layer, not redundant to higher layers I

Is that mean that Derts is supposed to be loaded with altering type derts (last one -1 has gradient Ga, while the previous -2 is G)? Thus [-2] is the "higher layer"?

yes, and deeper layers of g type don't have i, instead they use i or g of higher-layer derts as i

OK - they are within the higher-layer span, sub-space of it and I is considered already computed, just new derivatives are computed?

Is that ave_blob in: if Ga > ave_blob: the mentioned ave_n_sub_blob (a length) from the FB discussion, or it's a regular magnitude? (Because G and Ga are not counts, but accumulated magnitudes from the derivatives?)

I will post a new edit soon

OK

—

You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/issues/29#issuecomment-484692687, or mute the thread https://github.com/notifications/unsubscribe-auth/AFSI7WHFRL2TH7LVEK6XGULPRDQLNANCNFSM4HFW2V7Q .

[boris-kz]

And the expression G, Ga ... *cost etc. are to adjust for the "distance", in that context?

I don't like term "distance" as used in clustering, it's ambiguous and doesn't add anything here.

I used it in the context above - bigger span and diminishing effect of more distant pixels (more other pixels affect current), if they are taken individually/their small span.

ok. The cost is that of redundant higher-layer representations and extended syntax. G and Ga directly project corresponding variation over incremented distance, and must also compensate for increasing cost.

However before a real run through a hierarchy with data, the cost of discontinuity/levels/"syntax" seem not really justified to me. I see multiplies by a number of derivatives, division by spans, but it is to be checked...

In intra_blob:

for ablob in blob.subblob: # ablob is defined by ga sign Ga_rdn = 0 G = ablob.Derts[-2][-1] # I, Dx, Dy, G; Derts: current + higher-layers params, no lower layers yet Ga = ablob.Derts[-1][-1] # I converted per layer, not redundant to higher layers I

Is that mean that Derts is supposed to be loaded with altering type derts (last one -1 has gradient Ga, while the previous -2 is G)? Thus [-2] is the "higher layer"?

yes, and deeper layers of g type don't have i, instead they use i or g of higher-layer derts as i

OK - they are within the higher-layer span, sub-space of it and I is considered already computed, just new derivatives are computed?

right, but it's i in derts, not I in Derts. derts is an array of dert, per pixel. each dert represents derivatives per layer, each pixel has derts, with dert per each higher layer

[Twenkid]

On Fri, Apr 19, 2019 at 2:08 AM Boris Kazachenko notifications@github.com wrote:

And the expression G, Ga ... *cost etc. are to adjust for the "distance", in that context?

I don't like term "distance" as used in clustering, it's ambiguous and doesn't add anything here.

I used it in the context above - bigger span and diminishing effect of more distant pixels (more other pixels affect current), if they are taken individually/their small span.

ok. The cost is that of redundant higher-layer representations and extended syntax. G and Ga directly project corresponding variation over incremented distance, and must also compensate for increasing cost.

So I guess I may have combined two concepts: 1) weight of corresponding match (inverse or whatever) - predictive value as accumulated magnitude of the impact in the derivatives (a recent discussion) 2) (I meant that here, as the name "cost" is used) - the cost of evaluation which is a criteria for selection of patterns for additional evaluation and elevation?

Type 2) appears effectively in intra_blob these conditions >G >Ga etc., frame_blobs is more straightforward.

Both concepts are possibly intertwined also, but it could be confusing in general as "predictive value" (as magnitude of the variables) and "cost of prediction" - as computing power and memory to do an actual prediction, given the "predictive values" in the patterns, "higher power operations".

Higher expected predictive value allows the application of higher cost operations, including search - iteration over many possibilities, not just direct computation. ...

Could you express that "corresponding variation" with other words or more specific examples?

Also, the selection conditions with just bigger-than start to "feel" unclear.

That goes also for the general > ave ...

It's a main principle, but there's not an upper limit.

At the lowest level > ave is doing something like thresholding in CV and the upper bound is by default the pixel-values' up (say 255, down is 0). For summed spans though there's no a default upper bound (although there is a base-line filter) and just bigger than seems like a semi comparison.

I understand that the finer selection is supposed to come after many levels, but I "feel" an eventual filter is supposed to have not just one sided bigger-than > but a range of selected values, which for higher levels would have an array of selected ranges of match over selected coordinates (in given pattern-spaces).

Also, is the following guess right: are the sub-blobs and layers aimed at finding and representing gradients in the regular sense - ladders of magnitudes changing in the same direction over adjacent coordinates? Or that is for higher levels?

(The same-sign spans in frame_blobs only capture > than a base line, however that span could have many local minima and maxima of the inverse-match or the compared magnitude along the way)

However before a real run through a hierarchy with data, the cost of discontinuity/levels/"syntax" seem not really justified to me. I see multiplies by a number of derivatives, division by spans, but it is to be checked...

In intra_blob:

for ablob in blob.subblob: # ablob is defined by ga sign Ga_rdn = 0 G = ablob.Derts[-2][-1] # I, Dx, Dy, G; Derts: current + higher-layers params, no lower layers yet Ga = ablob.Derts[-1][-1] # I converted per layer, not redundant to higher layers I

Is that mean that Derts is supposed to be loaded with altering type derts (last one -1 has gradient Ga, while the previous -2 is G)? Thus [-2] is the "higher layer"?

yes, and deeper layers of g type don't have i, instead they use i or g of higher-layer derts as i

OK - they are within the higher-layer span, sub-space of it and I is considered already computed, just new derivatives are computed?

right, but it's i in derts, not I in Derts. derts is an array of dert, per pixel. each dert represents derivatives per layer, each pixel has derts, with dert per each higher layer

OK, I guess that's supposed to reflect:

[ P_params, derts = [ dert = p|a, dx, dy, ncomp, g]]], # one dert per current and higher layers

alternating sub g|a layers: p is replaced by angle

in odd derts and is absent in even derts

and the 3-tuples in the updated version: derts[-1] = ncomp, dy, dx # assign back into dert

But these indices and assignments don't seem right to me, maybe I don't understand it:

for _derts, derts in zip(derts[_start:end], derts[start, end]): i = derts[-1][-rng][0] ncomp, dy, dx = derts[-1] _i = _derts[-rng][0] _ncomp, _dy, _dx = _derts[-1] and then the assignment:

derts[-1] = ncomp, dy, dx # assign back into dert _derts[-1] = _ncomp, _dy, _dx # assign back into _dert

[boris-kz]

So I guess I may have combined two concepts: 1) weight of corresponding match (inverse or whatever) - predictive value as accumulated magnitude of the impact in the derivatives (a recent discussion) 2) (I meant that here, as the name "cost" is used) - the cost of evaluation which is a criteria for selection of patterns for additional evaluation and elevation?

Type 2) appear effectively in intra_blob these conditions >G >Ga etc., frame_blobs is more straightforward.

Both concepts are possibly intertwined also, but it could be confusing in general as "predictive value" (as magnitude of the variables) and "cost of prediction" - as computing power and memory to do an actual prediction, given the "predictive values" in the patterns, "higher power operations".

Well, one of the differences is that redundancy only applies to next step, not previous steps. The latter are already computed

Higher expected predictive value allows the application of higher cost operations, including search - iteration over many possibilities, not just direct computation.

Any given step is "direct", indirect means multiple steps, with evaluation at each.

Could you express that "corresponding variation" with other words or more specific examples?

Match & difference are subsets of comparands' magnitude In this case the comparands will be i | g | ga, and their magnitude is predictive of derivatives' magnitude

Also, the selection conditions with just bigger-than start to "feel" unclear. That goes also for the general > ave ... It's a main principle, but there's not an upper limit. At the lowest level > ave is doing something like thresholding in CV and the upper bound is by default the pixel-values' up (say 255, down is 0). For summed spans though there's no a default upper bound (although there is a base-line filter) and just bigger than seems like a semi comparison. I understand that the finer selection is supposed to come after many levels, but I "feel" an eventual filter is supposed to have not just one sided bigger-than > but a range of selected values, which for higher levels would have an array of selected ranges of match over selected coordinates

You feel that way because you don't think incrementally. At any given point, ultimate evaluation must be binary: input is either included in pattern or initiates a new one. Higher filters only apply to deeper layers of each pattern.

Also, is the following guess right: are the sub-blobs and layers aimed at finding and representing gradients in the regular sense - ladders of magnitudes changing in the same direction over adjacent coordinates? Or that is for higher levels?

(The same-sign spans in frame_blobs only capture > than a base line, however that span could have many local minima and maxima of the inverse-match or the compared magnitude along the way)

right, but it's i in derts, not I in Derts.

derts is an array of dert, per pixel. each dert represents derivatives per layer, each pixel has derts, with dert per each higher layer

OK, I guess that's supposed to reflect:

[ P_params, derts = [ dert = p|a, dx, dy, ncomp, g]]], # one dert per current and higher layers

alternating sub g|a layers: p is replaced by angle

in odd derts and is absent in even derts

and the 3-tuples in the updated version: derts[-1] = ncomp, dy, dx # assign back into dert

I think this is temporary, final tuple will be: ncomp, dy, dx, g

But these indices and assignments don't seem right to me, maybe I don't

understand it:

for _derts, derts in zip(derts[_start:end], derts[start, end]): i = derts[-1][-rng][0] ncomp, dy, dx = derts[-1] _i = _derts[-rng][0] _ncomp, _dy, _dx = _derts[-1] and then the assignment:

derts[-1] = ncomp, dy, dx # assign back into dert _derts[-1] = _ncomp, _dy, _dx # assign back into _dert

Yes, I don't think it's finished. I think it should be something like this:

i = derts[-rng][0] ncomp, dy, dx, g = derts[-1] _i = _derts[-rng][0] _ncomp, _dy, _dx, _g = _derts[-1]

derts[-1] = new_ncomp, new_dy, new_dx, new_g # assign back into dert _derts[-1] = _new_ncomp, _new_dy, _new_dx, _new_g # assign back into _dert

Khanh?

[boris-kz]

Also, is the following guess right: are the sub-blobs and layers aimed at finding and representing gradients in the regular sense - ladders of magnitudes changing in the same direction over adjacent coordinates? Or that is for higher levels?

(The same-sign spans in frame_blobs only capture > than a base line, however that span could have many local minima and maxima of the inverse-match or the compared magnitude along the way)

Yes, G and Ga must have greater magnitude to justify extra cost of deeper layers.

[khanh93vn]

understand it: for _derts, derts in zip(derts[_start:end], derts[start, end]): i = derts[-1][-rng][0] ncomp, dy, dx = derts[-1] _i = _derts[-rng][0] _ncomp, _dy, _dx = _derts[-1] and then the assignment: derts[-1] = ncomp, dy, dx # assign back into dert _derts[-1] = _ncomp, _dy, _dx # assign back into _dert

That's a mistake, should be i = dert[-rng][0].

[Twenkid]

On Friday, April 19, 2019, Boris Kazachenko notifications@github.com wrote:

So I guess I may have combined two concepts: 1) weight of corresponding match (inverse or whatever) - predictive value as accumulated magnitude of the impact in the derivatives (a recent discussion) 2) (I meant that here, as the name "cost" is used) - the cost of evaluation which is a criteria for selection of patterns for additional evaluation and elevation?

Type 2) appear effectively in intra_blob these conditions >G >Ga etc., frame_blobs is more straightforward.

Both concepts are possibly intertwined also, but it could be confusing in general as "predictive value" (as magnitude of the variables) and "cost of prediction" - as computing power and memory to do an actual prediction, given the "predictive values" in the patterns, "higher power operations".

Well, one of the differences is that redundancy only applies to next step, not previous steps. The latter are already computed

"Cost" vars correspond to rdn?

Higher expected predictive value allows the application of higher cost operations, including search - iteration over many possibilities, not just direct computation.

Any given step is "direct", indirect means multiple steps, with evaluation at each.

Could you express that "corresponding variation" with other words or more specific examples?

Match & difference are subsets of comparands' magnitude In this case the comparands will be i | g | ga, and their magnitude is predictive of derivatives' magnitude

Thus is the "variation" you talk about the selection of one of the possible elements mentioned, maybe after finding which provides best lredictive value? Instead of computing just one?

Aren't g and ga also derivatives?

Also, the selection conditions with just bigger-than start to "feel" unclear. That goes also for the general > ave ... It's a main principle, but there's not an upper limit. At the lowest level > ave is doing something like thresholding in CV and the upper bound is by default the pixel-values' up (say 255, down is 0). For summed spans though there's no a default upper bound (although there is a base-line filter) and just bigger than seems like a semi comparison. I understand that the finer selection is supposed to come after many levels, but I "feel" an eventual filter is supposed to have not just one sided bigger-than > but a range of selected values, which for higher levels would have an array of selected ranges of match over selected coordinates

You feel that way because you don't think incrementally. At any given point, ultimate evaluation must be binary: input is either included in pattern or initiates a new one.

And in some point initiating a new level? After evaluating all collected patterns, some buffer is filled-up?

Higher filters only apply to deeper layers of each pattern.

Higher filters - ranges with an upper bound as well or lists of ranges?

Yes, I don't think it's finished. I think it should be something like this:

i = derts[-rng][0] ncomp, dy, dx, g = derts[-1] _i = _derts[-rng][0] _ncomp, _dy, _dx, _g = _derts[-1]

derts[-1] = new_ncomp, new_dy, new_dx, new_g # assign back into dert _derts[-1] = _new_ncomp, _new_dy, _new_dx, _new_g # assign back into _dert

OK

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/issues/29#issuecomment-484754153, or mute the thread https://github.com/notifications/unsubscribe-auth/AFSI7WHC3KXMQEKMNRPE3I3PREZ4ZANCNFSM4HFW2V7Q .

[boris-kz]

Well, one of the differences is that redundancy only applies to next step, not previous steps. The latter are already computed

"Cost" vars correspond to rdn?

Yes, cost per dert = ave rdn, and cost per blob = ave_blob rdn, see last update of intra_blob

Match & difference are subsets of comparands' magnitude In this case the comparands will be i | g | ga, and their magnitude is predictive of derivatives' magnitude

Thus is the "variation" you talk about the selection of one of the possible elements mentioned, maybe after finding which provides best lredictive value? Instead of computing just one?

Aren't g and ga also derivatives?

Yes, but they can be inputs to form deeper derivatives

You feel that way because you don't think incrementally. At any given point, ultimate evaluation must be binary: input is either included in pattern or initiates a new one.

And in some point initiating a new level? After evaluating all collected

patterns, some buffer is filled-up?

Well, the "buffer" is either frame, or single top-layer pattern for "frameless" input. They are forwarded to | initiate higher level after intra_blob is finished exploring nested sub_patterns

Higher filters only apply to deeper layers of each pattern.

Higher filters - ranges with an upper bound as well or lists of ranges?

There are no upper bounds, only incrementally higher filters: ave * incremented rdn. These filters don't define higher-layer root patterns, only sub_patterns nested within them.

[Twenkid]

@khanh93vn Thanks, the now complete source suggests also.

I noticed you've marked that possible usage of a look-up table that we mentioned earlier:

    hyp = hypot(xd, yd)
            y_coef = yd / hyp       # to decompose d into dy. Whole computation could be replaced with a look-up table instead?
            x_coef = xd / hyp       # to decompose d into dx. Whole computation could be replaced with a look-up table instead?

@boris-kz

Higher filters only apply to deeper layers of each pattern. Higher filters - ranges with an upper bound as well or lists of ranges? There are no upper bounds, only incrementally higher filters: ave * incremented rdn.

So, bigger and bigger and bigger... The ultimate patterns will have filters of 99999999999999999999999999999E+9999, matching the Universe. :)

What makes sense to me is that it's assumed that if the comparison is at certain level, that implies that there is a minimum cumulative match between derivatives, so the current one should be bigger than that.

Since each level works within given limited coordinated borders (by the nature of the lower level limited input magnitudes and coordinate ranges), the following possible upper bounds could be also eventually estimated, based on the "magic number" and costs calculations, number of variables, spans of pixels with accumulated magnitudes etc.

These filters don't define higher-layer root patterns, only sub_patterns nested within them.

Here I referred to "higher filters" as to expected provisional "higher-order filters" which could have an upper bound and also be expressed as lists of ranges, instead of just one filter-value and the default bigger-than comparison. :)

And in some point initiating a new level? After evaluating all collected patterns, some buffer is filled-up? Well, the "buffer" is either frame, or single top-layer pattern for "frameless" input. They are forwarded to | initiate higher level after intra_blob is finished exploring nested sub_patterns

My topic was for the moments/criteria for the conceptual generation of a new level.

I realize that there is one process that either includes the input into current pattern or creates a new one - it is similar to parsing code, the parser either begins recognizing and building some more complex syntax structure, say a loop, a function, an expression, recognized syntactic pattern, or it is including the required elements into the recently opened structure that reflects the loop etc.

However there is also a "higher" "supervisor process" that generates the code (you, us) and at certain moments it finishes the search for patterns the way it does for the current level (or stage), packs it and sends it to something else.

The initial limits are the bounds of the input space, the coordinates. The following, traversing all patterns given the defined laws for defining a pattern and enumerating them within the whole previous coordinate space etc.

I see that here you don't refer to these current stages as levels, but "levels of encoding Le", it would be a complete level after the yet non-existing inter-blobs which encompass the whole frame.

Also, in a future setting where there is time or a series of frames, I think naturally there is a recent "buffer" that limits the depth of the comparison.

One path is possibly comparing two-adjacent, producing a higher level (or stage) pattern, then comparing two such patterns, then comparing two of the higher etc....

In time there is also a flow which is like scanning patterns in a frame, just the coordinates will be in "t", thus an iteration like current would be like:

for t, frame in enumerate(frames_, start=window):
   frame =frames_[-t] #searching within a window of frames back

Then "same sign g" would be searched over sequences of frames, which may produce "Ps", "Segs", "Blobs" etc.

An initial meaningful "buffer" would be one that is big enough to capture basic gestures and movement, like a reaching and grasping movement, closing or opening a door, turning a head, a smile from a neutral position or another expression, clapping hands, one, two or three steps of a walk (capturing repetition) etc. Sure it has to adjust the length and could start with two frames, then extending it.

Cross-compare

Indeed, I remember you used to use the expression "cross compare".

Is that meaning all-to-all comparison of all patterns within a range of coordinates, producing derivatives for all? (Which technically is a buffer of patterns or inputs at the low/lower level, producing newderts__, maps of derts__ etc.)

Currently comparison is of adjacent (or rng-distant-coordinates-corresponding) patterns - is the latter corresponding to the "cross comparison"?

...

[boris-kz]

So, bigger and bigger and bigger... The ultimate patterns will have filters of 99999999999999999999999999999E+9999, matching the Universe. :)

You seem to make an effort to misunderstand me. I guess that's a habit from your political and philosophical debates.

What makes sense to me is that it's assumed that if the comparison is at

certain level, that implies that there is a minimum cumulative match between derivatives, so the current one should be bigger than it.

Which means that all we need to define is an increment in that cumulative match

Since each level works within given limited coordinated borders (by the nature of the lower level limited input magnitudes and coordinate ranges), the following possible upper bounds could be also eventually estimated, based on the "magic number" and costs calculations, number of variables, spans of pixels with accumulated magnitudes etc.

These filters don't define higher-layer root patterns, only sub_patterns nested within them.

Here I referred to "higher filters" as to expected provisional "higher-order filters" which could have an upper bound and also be expressed as lists of ranges, instead of just one filter-value and the default bigger-than comparison. :)

There is no need to generate ranges that will never be used. They will be generated as needed from a single increment.

And in some point initiating a new level? After evaluating all collected patterns, some buffer is filled-up? Well, the "buffer" is either frame, or single top-layer pattern for "frameless" input. They are forwarded to | initiate higher level after intra_blob is finished exploring nested sub_patterns

My topic was for the moments/criteria for the conceptual generation of a new level. I realize that there is one process that either includes the input into current pattern or creates a new one - it is similar to parsing code, the parser either begins recognizing and building some more complex syntax structure, say a loop, a function, an expression, recognized syntactic pattern, or it is including the required elements into the recently opened structure that reflects the loop etc.

However there is also a "higher" "supervisor process" that generates the code (you, us) and at certain moments it finishes the search for patterns the way it does for the current level (or stage), packs it and sends it to something else. The initial limits are the bounds of the input space, the coordinates. The following, traversing all patterns given the defined laws for defining a pattern and enumerating them within the whole previous coordinate space etc.

I see that here you don't refer to these current stages as levels, but "levels of encoding Le", it would be a complete level after the yet non-existing inter-blobs which encompass the whole frame.

Also, in a future setting where there is time or a series of frames, I think naturally there is a recent "buffer" that limits the depth of the comparison.

One path is possibly comparing two-adjacent, producing a higher level (or stage) pattern, then comparing two such patterns, then comparing two of the higher etc....

In time there is also a flow which is like scanning patterns in a frame, just the coordinates will be in "t", thus an iteration like current would be like:

for t, frame in enumerate(frames, start=window): frame =frames[-t] #searching within a window of frames back

Then "same sign g" would be searched over sequences of frames, which may produce "Ps", "Segs", "Blobs" etc.

An initial meaningful "buffer" would be one that is big enough to capture basic gestures and movement, like a reaching and grasping movement, closing or opening a door, turning a head, a smile from a neutral position or another expression, clapping hands, one, two or three steps of a walk (capturing repetition) etc. Sure it has to adjust the length and could start with two frames, then extending it.

Again, you are talking about rng top-layer patterns, initialized at 1. That's your buffer, frame is just an initial artifact.

Cross-compare

Indeed, I remember you used to use the expression "cross compare".

Is that meaning all-to-all comparison of all patterns within a range of coordinates, producing derivatives for all? (Which technically is a buffer of patterns or inputs at the low/lower level, producing new derts, maps of derts etc.)

Currently comparison is of adjacent (or rng-distant-coordinates-corresponding) patterns - is the latter corresponding to the "cross comparison"?

Yes

[Twenkid]

На сб, 20.04.2019 г., 14:27 Boris Kazachenko notifications@github.com написа:

So, bigger and bigger and bigger... The ultimate patterns will have filters of 99999999999999999999999999999E+9999, matching the Universe. :)

You seem to make an effort to misunderstand me. I guess that's a habit from your political and philosophical debates.

Well, these are questions due to uncertainty and here - humour.

What makes sense to me is that it's assumed that if the comparison is at

certain level, that implies that there is a minimum cumulative match between derivatives, so the current one should be bigger than it.

Which means that all we need to define is an increment in that cumulative match

OK

Since each level works within given limited coordinated borders (by the nature of the lower level limited input magnitudes and coordinate ranges), the following possible upper bounds could be also eventually estimated, based on the "magic number" and costs calculations, number of variables, spans of pixels with accumulated magnitudes etc.

These filters don't define higher-layer root patterns, only sub_patterns nested within them.

Here I referred to "higher filters" as to expected provisional "higher-order filters" which could have an upper bound and also be expressed as lists of ranges, instead of just one filter-value and the default bigger-than comparison. :)

There is no need to generate ranges that will never be used. They will be generated as needed from a single increment.

OK, so they will be never used and wont be generated. I asked because it is not always clear by default what is meant by "higher", is it just a "higher level" but structurally it is again just scalar to which there is IF BGT or it is supposed to evolve structurally (a higher order, another syntax, another processing) as well as a thread in the algorithm. You are supposed to know best and I am more curious about what comes next than current.

Again, you are talking about rng top-layer patterns, initialized at 1. That's your buffer, frame is just an initial artifact.

Right, technically frame itself is not a pattern, an inter-pattern that covers the input space represented by the frame boundaries is a pattern. But it maps the frame space anyway.

Cross-compare

Indeed, I remember you used to use the expression "cross compare".

Is that meaning all-to-all comparison of all patterns within a range of coordinates, producing derivatives for all? (Which technically is a buffer of patterns or inputs at the low/lower level, producing new derts, maps of derts etc.)

Currently comparison is of adjacent (or rng-distant-coordinates-corresponding) patterns - is the latter corresponding to the "cross comparison"?

Yes

OK

—

You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/issues/29#issuecomment-485097342, or mute the thread https://github.com/notifications/unsubscribe-auth/AFSI7WBPVCAEOCV6F7R6JL3PRL4ZRANCNFSM4HFW2V7Q .

[khanh93vn]

@khanh93vn Thanks, the now complete source suggests also.

I noticed you've marked that possible usage of a look-up table that we mentioned earlier:

    hyp = hypot(xd, yd)
            y_coef = yd / hyp       # to decompose d into dy. Whole computation could be replaced with a look-up table instead?
            x_coef = xd / hyp       # to decompose d into dx. Whole computation could be replaced with a look-up table instead?

Do you mean the fast-clean macros you mentioned before? I only though of using pre-generated value, but macro should make better choice. I'm not used to using macros yet, just happened to know it is used in C. Is it possible in Python?

[Twenkid]

I meant also only the pre-generated only, like traversing a range of xd, yd and storing hypot ... And then reading hyp from the table, for optimization. (I didnt think about including / hyp also, maybe there might be too much values if we want precision (I dont know what precision is needed). You're right that it could go into a macro for shortening the code but AFAIK Boris wouldnt like it. :)

It seems there is a library called MacroPy https://pypi.org/project/MacroPy/

My idea was for custom tool, I dont know if the available macros would fit my ideas for general code synthesis (and it has its compatibility requirements), mine which would take into account CogAlg concepts and code patterns (try to recognize and generalize them) and may include GUI and visualisation without complete compilation.

EDIT (fix): As of the code synthesis - e.g. for routines like the ones you have debugged lately, if the limits and the traversed containers are defined, an already debugged automated traverse-select-... synthesizable iterator.

On Saturday, April 20, 2019, Khanh Nguyen notifications@github.com wrote:

@khanh93vn https://github.com/khanh93vn Thanks, the now complete source suggests also.

I noticed you've marked that possible usage of a look-up table that we mentioned earlier:

hyp = hypot(xd, yd)
        y_coef = yd / hyp       # to decompose d into dy. Whole computation could be replaced with a look-up table instead?
        x_coef = xd / hyp       # to decompose d into dx. Whole computation could be replaced with a look-up table instead?

Do you mean the fast-clean macros you mentioned before? I only though of using pre-generated value, but macro should make better choice. I'm not used to using macros yet, just happened to know it is used in C. Is it possible in Python?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/issues/29#issuecomment-485153467, or mute the thread https://github.com/notifications/unsubscribe-auth/AFSI7WEVOXO5VKUCYFVZ36DPRNT3NANCNFSM4HFW2V7Q .

[Twenkid]

In intra_comp's form_blob: https://github.com/khanh93vn/CogAlg/blob/master/frame_2D_alg/intra_comp.py

For positive blobs, why only Ly and L are summed? (While other params are also summed for negatives as well) Something like "dimensions in pixel-space of same-sign positive sub_blobs"?

(Possibly some indentation issue, in github's view the [par1+...] part is at the level of if ..., not enclosed)

Also shouldn't the [par1 ... part be params[2:-1] (not [2:]): root_blob.sub_Derts[2:-1] = [par1 + par2 for par1, par2 in zip(params[2:-1], root_blob.sub_Derts[2:-1])]

(Since [-1] is the blob etc.)

if s:  # for positive sub_blobs only
            root_blob.sub_Derts[0] += Ly
            root_blob.sub_Derts[1] += L

        root_blob.sub_Derts[2:] = [par1 + par2 for par1, par2 in zip(params[2:], root_blob.sub_Derts[2:])]

        root_blob.sub_Derts[-1] \
            .append(nt_blob( Derts=[(0, s, params, [])],  # typ=0, sign: positive for sub_blob_ > 0?
                            # last term is sub_blob_ of nesting depth = Derts[index]
          (...)

recursive intra_blob' eval_layer' branch() calls add new dert to derts, Dert to Derts, layer of subblobs to blob, where Dert params are summed params of all positive blobs per structure, with optional layer:

[khanh93vn]

For positive blobs, why only Ly and L are summed

not sure but I think it's because otherwise they would be identical to Ly and L in super-blob params. Other params are freshly generated so they'll be summed regardless.

Also shouldn't the [par1 ... part be params[2:-1] (not [2:])

Ah, you're right. I'll update it next time.

I meant also only the pre-generated only, like traversing a range of xd, yd and storing hypot ... And then reading hyp from the table, for optimization. (I didnt think about including / hyp also, maybe there might be too much values if we want precision (I dont know what precision is needed). You're right that it could go into a macro for shortening the code but AFAIK Boris wouldnt like it. :) It seems there is a library called MacroPy https://pypi.org/project/MacroPy/ My idea was for custom tool, I dont know if the available macros would fit my ideas for general code synthesis (and it has its compatibility requirements), mine which would take into account CogAlg concepts and code patterns (try to recognize and generalize them) and may include GUI and visualisation without complete compilation. EDIT (fix): As of the code synthesis - e.g. for routines like the ones you have debugged lately, if the limits and the traversed containers are defined, an already debugged automated traverse-select-... synthesizable iterator.

I see. Thanks for those infos!

[boris-kz]

For positive blobs, why only Ly and L are summed

not sure but I think it's because otherwise they would be identical to Ly and L in super-blob params. Other params are freshly generated so they'll be summed regardless.

Yes, that's to get positive_L / total_L "fill ratio", which is meaningful. Global (blob-wide) Ly and L are same-sign. I am revising blob structure right now, should post soon.

Basic structure is derivation tree represented by Derts [(params, sub)], where sub is subblob nested to the depth = Derts [index] But top Dert represents single blob, while lower Dert' params are merged from corresponding-layer sub_blobs. So, only top Dert has:

• typ, rng, sign: mixed for lower-Dert blobs,
• map, box: same as top Dert, and fill ratio doesn't seem to be as meaningful as for Ly and L
• seg_: to be unfolded via sub_blobs vs. directly

[Twenkid]

In def intra_comp, one code style proposal:

I don't like how the index "i" and y are declared too far from their first usage, while "i" is used only in LOOP2. When possible, the vars are best declared close to their usage, especially counters.

Instead of:

    y0, yn, x0, xn = blob.box
    y = y0  # current y, from seg y0 -> yn - 1
    i = 0   # segment index
    blob.seg_.sort(key=lambda seg: seg[0])  # sort by y0 coordinate
    buff___ = deque(maxlen=rng)
    seg_ = []       # buffer of segments containing line y
    sseg_ = deque() # buffer of sub-segments

I'd do:

    blob.seg_.sort(key=lambda seg: seg[0])  # sort by y0 coordinate
    buff___ = deque(maxlen=rng)
    seg_ = []       # buffer of segments containing line y
    sseg_ = deque() # buffer of sub-segments
    y0, yn, x0, xn = blob.box
    y = y0  # current y, from seg y0 -> yn - 1
    i = 0   # segment index

    while y < yn and i < len(blob.seg_): 
     ...

It also emphasizes the sort which is an important suggestion.

Or:

    buff___ = deque(maxlen=rng)
    seg_ = []       # buffer of segments containing line y
    sseg_ = deque() # buffer of sub-segments
    blob.seg_.sort(key=lambda seg: seg[0])  # sort by y0 coordinate
    y0, yn, x0, xn = blob.box
    y = y0  # current y, from seg y0 -> yn - 1
    i = 0   # segment index

    while y < yn and i < len(blob.seg_): 
     ...

[khanh93vn]

I see. Sorry for the mess, and thank you!

[Twenkid]

NP. It was not a mess, just that order is a bit more logical.

[Twenkid]

Khan, could you add comments with the structures of the derts (all kinds) in comp_range, for all functions? Are they completed?

The indices look confusing to me, suggesting different structures (or I'm missing something...)

For lateral_comp and comp_slice:

for x, derts in enumerate(derts_, start=x0):
            i = derts[-rng+1][0]        # +1 for future derts.append(new_dert)
            ncomp, dy, dx = derts[-1][-4:-1]

I think that suggests derts is a list of [i, ncomp, dy, dx]

     for _derts, derts in zip(_derts_[_start:_end], derts_[start:end]):

        i = derts[comparand_index][0]
        ncomp, dy, dx = derts[-1] 

The first line implies that the 0-th element is i, like [i,ncomp,dy,dx], while the second line suggests that derts consists of a list of 3-tuples, now reading the last one: (ncomp, dy, dx), and i is somehow separate.

In the end as well:

   # return:
        derts[-1] = ncomp, dy, dx
        _derts[-1] = _ncomp, _dy, _dx

(Supposing the containers are accessed by reference and changed in place.)

[Twenkid]

Is it something about the alternating derts format?

seg_ =  # seg_s of lower Derts are packed in their sub_blobs
            [ seg_params,  
              Py_ = # vertical buffer of Ps per segment
                  [ P_params,

                    derts = [ dert = p|a, ncomp, dx, dy, g]]],  # one dert per current and higher layers 
                    # alternating p|a type of dert in derts: 
                    # derts [even][0] = p if top dert, else none or ncomp 
                    # derts [odd] [0] = angle

[boris-kz]

The indices look confusing to me, suggesting different structures (or I'm missing something...)

For lateral_comp and comp_slice:

for x, derts in enumerate(derts_, start=x0): i = derts[-rng+1][0] # +1 for future derts.append(new_dert) ncomp, dy, dx = derts[-1][-4:-1]

I think that suggests derts is a list of [i, ncomp, dy, dx]

Yes, backward indexing is used because alternating derts may have different length: i may or may not be there

 for _derts, derts in zip(_derts_[_start:_end], derts_[start:end]):

    i = derts[comparand_index][0]
    ncomp, dy, dx = derts[-1]

The first line implies that the 0-th element is i, like [i,ncomp,dy,dx], while the second line suggests that derts' consist of a list of 3-tuples, now reading the last one: (ncomp, dy, dx), and i is somehow separate.

I think this is mistake. Khanh?

In the end as well:

return:

  derts[-1] = ncomp, dy, dx
  _derts[-1] = _ncomp, _dy, _dx

this is current-layer dert, it will be appended with g in compute_g, intra_comp.

[boris-kz]

 for _derts, derts in zip(_derts_[_start:_end], derts_[start:end]):

    i = derts[comparand_index][0]
    ncomp, dy, dx = derts[-1]

The first line implies that the 0-th element is i, like [i,ncomp,dy,dx], while the second line suggests that derts' consist of a list of 3-tuples, now reading the last one: (ncomp, dy, dx), and i is somehow separate.

Oh, that's because i is from higher-layer dert, while ncom, dx, dy is current-layer dert. They have different format, current-layer is 3-tuple.

[Twenkid]

Thanks. Whatever is the case though, IMO such format transformations and different formats in the same container and similar identifiers are fragile and they should be reminded thoroughly with expected structures' content near the usage (top comment is too far away).

[boris-kz]

Sure, it's a work in progress. Add comments and make a pull request. For comp_branches, it would be to Khanh's repo.

On Tue, Apr 23, 2019 at 6:29 AM Todor Arnaudov notifications@github.com wrote:

Thanks. Whatever is the case though, IMO such format transformations and different formats in the same container and similar identifiers are fragile and they should be reminded thoroughly with expected structures' content near the usage (top comment is too far away).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/issues/29#issuecomment-485747402, or mute the thread https://github.com/notifications/unsubscribe-auth/AFABOGMABVSYTEFFVQRTMMTPR3QHFANCNFSM4HFW2V7Q .

[Twenkid]

OK, but better for something more significant in content than adding pull-merge burden for two lines of comments.

На ср, 24.04.2019 г., 1:08 Boris Kazachenko notifications@github.com написа:

Sure, it's a work in progress. Add comments and make a pull request. For comp_branches, it would be to Khanh's repo.

On Tue, Apr 23, 2019 at 6:29 AM Todor Arnaudov notifications@github.com wrote:

Thanks. Whatever is the case though, IMO such format transformations and different formats in the same container and similar identifiers are fragile and they should be reminded thoroughly with expected structures' content near the usage (top comment is too far away).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/issues/29#issuecomment-485747402, or mute the thread < https://github.com/notifications/unsubscribe-auth/AFABOGMABVSYTEFFVQRTMMTPR3QHFANCNFSM4HFW2V7Q

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/boris-kz/CogAlg/issues/29#issuecomment-485993005, or mute the thread https://github.com/notifications/unsubscribe-auth/AFSI7WC4AKA3ZPTYIW5ZPN3PR6CFBANCNFSM4HFW2V7Q .