predict reset_index stack trace

[apiszcz]

1 Stack trace 2 Sample code

I am trying to follow the example from: http://soft-matter.github.io/trackpy/tutorial/prediction.html

Any thoughts appreciated, not sure what reset_index thinks f is a string.

== 1 ========================== Traceback (most recent call last): File "lib\testpre\trackpylearn.py", line 21, in tr=pd.concat(pred.link_df_iter(f, 1.5)) File "c:\p\testpre\lib\pandas\tools\merge.py", line 724, in concat copy=copy) File "c:\p\testpre\lib\pandas\tools\merge.py", line 755, in init objs = [obj for obj in objs if obj is not None ] File "c:\p\testpre\lib\trackpy\predict.py", line 52, in link_df_iter for frame in linking.link_df_iter(_args, *_kw): File "c:\p\testpre\lib\trackpy\linking.py", line 696, in link_df_iter labeled_levels, features_forpost, index_iter): File "c:\p\testpre\lib\trackpy\linking.py", line 919, in link prev_level = next(level_iter) File "c:\p\testpre\lib\trackpy\linking.py", line 683, in levels = (_build_level(frame, pos_columns, t_column, diagnostics=diagnostics) File "c:\p\testpre\lib\trackpy\linking.py", line 681, in (frame.reset_index(drop=True) for frame in features_for_reset)) AttributeError: 'str' object has no attribute 'reset_index'

== 2 ========================== import os,pdb,pprint,sys sys.path.insert(0,os.path.abspath('lib')) import numpy as np import pandas as pd from pdb import set_trace as dd import trackpy as tp import trackpy.predict

f=pd.DataFrame(columns=['x','y','frame']) f=f.append(({'x':1,'y':1,'frame':0}),ignore_index=True) f=f.append(({'x':2,'y':2,'frame':1}),ignore_index=True) f=f.append(({'x':3,'y':3,'frame':2}),ignore_index=True) f=f.append(({'x':4,'y':4,'frame':3}),ignore_index=True) f=f.append(({'x':10,'y':10,'frame':0}),ignore_index=True) f=f.append(({'x':20,'y':20,'frame':1}),ignore_index=True) f=f.append(({'x':30,'y':30,'frame':2}),ignore_index=True) f=f.append(({'x':40,'y':40,'frame':3}),ignore_index=True)

pred=trackpy.predict.NearestVelocityPredict() tr=pd.concat(pred.link_df_iter(f, 1.5))

print f print tr

dd()

[danielballan]

I can reproduce.

[nkeim]

Thanks for the excellent bug report! I'm looking into it now.

[apiszcz]

Great, i was hoping my simple sample would help, Python 2.7.9 64 bit BTW (Windows 7 64 bit)

On 3/5/15, Dan Allan notifications@github.com wrote:

I can reproduce.

---

Reply to this email directly or view it on GitHub: https://github.com/soft-matter/trackpy/issues/220#issuecomment-77438945

[apiszcz]

Thank you for trackpy, and this is the DEV version. 0.3somedaysoon

On Thu, Mar 5, 2015 at 3:02 PM, Nathan Keim notifications@github.com wrote:

Thanks for the excellent bug report! I'm looking into it now.

— Reply to this email directly or view it on GitHub https://github.com/soft-matter/trackpy/issues/220#issuecomment-77440321.

[nkeim]

The problem is that you are using pred.link_df_iter() like you would use link_df(), when they are not the same. link_df_iter() requires a sequence or iterator of DataFrames, one for each video frame, whereas link_df() takes a single DataFrame for the entire movie.

I think this points to a couple issues with trackpy that I'd like to check out a little more deeply:

1. Your mistake was a completely natural one to make, yet the error message you got was absolutely worthless.
2. I never got around to implementing pred.link_df(), and I wonder if there's an uncomplicated way to do it.

But for now, here's a workaround that should help:

f_iter = (frame for fnum, frame in f.groupby('frame'))
tr=pd.concat(trackpy.link_df_iter(f_iter, 1.5))

[apiszcz]

That works, thank you. So the documentation I'm reviewing was the example I tried. New result follows. I also do not see how this change uses the trackpy.predict.NearestVelocityPredict()

Is there an approach to handling particles with different velocity characteristics as in this example?

particle 0 1,1 2,2 3,3 4,4 particle 1 10,10 20,20 30,30 40,40

x   y  frame  particle

0 1 1 0 0 1 10 10 0 1 0 2 2 1 0 1 20 20 1 2 0 3 3 2 0 1 30 30 2 3 0 4 4 3 0 1 40 40 3 4 --Return--

When I try the following I get the same result, I am wondering if ADAPTIVE might help here.

pred=trackpy.predict.NearestVelocityPredict() f_iter = (frame for fnum, frame in f.groupby('frame')) tr=pd.concat(pred.link_df_iter(f_iter, 1.5))

the result is the same. x y frame particle 0 1 1 0 0 1 10 10 0 1 0 2 2 1 0 1 20 20 1 2 0 3 3 2 0 1 30 30 2 3 0 4 4 3 0 1 40 40 3 4

On Thu, Mar 5, 2015 at 3:16 PM, Nathan Keim notifications@github.com wrote:

The problem is that you are using pred.link_df_iter() like you would use link_df(), when they are not the same. link_df_iter() requires a sequence or iterator of DataFrames, one for each video frame, whereas link_df() takes a single DataFrame for the entire movie.

I think this points to a couple issues with trackpy that I'd like to check out a little more deeply:

1. Your mistake was a completely natural one to make, yet the error message you got was absolutely worthless.
2. I never got around to implementing pred.link_df(), and I wonder if there's an uncomplicated way to do it.

But for now, here's a workaround that should help:

f_iter = (frame for fnum, frame in f.groupby('frame')) tr=pd.concat(trackpy.link_df_iter(f_iter, 1.5))

— Reply to this email directly or view it on GitHub https://github.com/soft-matter/trackpy/issues/220#issuecomment-77442726.

[apiszcz]

This solves the trivial case, however still curious about velocity.

tr=tp.link_df(f,100)

x   y  frame  particle

0 1 1 0 0 1 2 2 1 0 2 3 3 2 0 3 4 4 3 0 4 10 10 0 1 5 20 20 1 1 6 30 30 2 1 7 40 40 3 1

On Thu, Mar 5, 2015 at 3:16 PM, Nathan Keim notifications@github.com wrote:

The problem is that you are using pred.link_df_iter() like you would use link_df(), when they are not the same. link_df_iter() requires a sequence or iterator of DataFrames, one for each video frame, whereas link_df() takes a single DataFrame for the entire movie.

I think this points to a couple issues with trackpy that I'd like to check out a little more deeply:

1. Your mistake was a completely natural one to make, yet the error message you got was absolutely worthless.
2. I never got around to implementing pred.link_df(), and I wonder if there's an uncomplicated way to do it.

But for now, here's a workaround that should help:

f_iter = (frame for fnum, frame in f.groupby('frame')) tr=pd.concat(trackpy.link_df_iter(f_iter, 1.5))

— Reply to this email directly or view it on GitHub https://github.com/soft-matter/trackpy/issues/220#issuecomment-77442726.

[nkeim]

As you say, this is a trivial example, so you can just use a large search_range with no "tricks."

NearestVelocityPredict "learns" the velocities of your particles, but it does not know this in the first frame. So if link_df_iter() cannot link the first and second frames together, you get bad velocity data and NearestVelocityPredict will be useless. I think that is what is happening here. You can feed an initial velocity field to NearestVelocityPredict, to give it a hint. See the documentation.

Adaptive search may also help here. But in principle prediction is better because it is actually using the velocity information that is already in your data to improve tracking; otherwise that information would be ignored.

Also, I just introduced #221 which might help with the issue you encountered. I had treated prediction as an "advanced" feature for someone who was already comfortable with trackpy, but did not consider that someone might decide to try trackpy in order to use prediction.

[apiszcz]

Great, I will try priming the pump with a velocity 'guess'. I spent some time with a Kalman filter, similar startup issue until things settle, I really need the first position of the item, so far trackpy is giving me. I'll try your suggestion, thank you.

On Thu, Mar 5, 2015 at 5:58 PM, Nathan Keim notifications@github.com wrote:

As you say, this is a trivial example, so you can just use a large search_range with no "tricks."

NearestVelocityPredict "learns" the velocities of your particles, but it does not know this in the first frame. So if link_df_iter() cannot link the first and second frames together, you get bad velocity data and NearestVelocityPredict will be useless. I think that is what is happening here. You can feed an initial velocity field to NearestVelocityPredict, to give it a hint. See the documentation http://soft-matter.github.io/trackpy/generated/trackpy.predict.NearestVelocityPredict.html#trackpy.predict.NearestVelocityPredict .

Adaptive search may also help here. But in principle prediction is better because it is actually using the velocity information that is already in your data to improve tracking; otherwise that information would be ignored.

Also, I just introduced #221 https://github.com/soft-matter/trackpy/pull/221 which might help with the issue you encountered. I had treated prediction as an "advanced" feature for someone who was already comfortable with trackpy, but did not consider that someone might decide to try trackpy in order to use prediction.

— Reply to this email directly or view it on GitHub https://github.com/soft-matter/trackpy/issues/220#issuecomment-77470932.

[apiszcz]

YEARS AGO MEI micro, take the ones you want in weight room.

On Thu, Mar 5, 2015 at 5:58 PM, Nathan Keim notifications@github.com wrote:

As you say, this is a trivial example, so you can just use a large search_range with no "tricks."

NearestVelocityPredict "learns" the velocities of your particles, but it does not know this in the first frame. So if link_df_iter() cannot link the first and second frames together, you get bad velocity data and NearestVelocityPredict will be useless. I think that is what is happening here. You can feed an initial velocity field to NearestVelocityPredict, to give it a hint. See the documentation http://soft-matter.github.io/trackpy/generated/trackpy.predict.NearestVelocityPredict.html#trackpy.predict.NearestVelocityPredict .

Adaptive search may also help here. But in principle prediction is better because it is actually using the velocity information that is already in your data to improve tracking; otherwise that information would be ignored.

Also, I just introduced #221 https://github.com/soft-matter/trackpy/pull/221 which might help with the issue you encountered. I had treated prediction as an "advanced" feature for someone who was already comfortable with trackpy, but did not consider that someone might decide to try trackpy in order to use prediction.

— Reply to this email directly or view it on GitHub https://github.com/soft-matter/trackpy/issues/220#issuecomment-77470932.

[nkeim]

Addressed by #221, pending merge.