Unnecessary duplications

[GoogleCodeExporter]

How is it currently implemented?
Right now, when the CacheExpander returns a CacheUpdate with latestTime_ 
not equal to the latestTime_ in the current safe region, the lesser of the 
two latestTime_'s is accepted as the latestTime_ of the new safe region.  
This means that on the next Periodic Full Update, any wrecks that happened 
after the earlier latestTime_ but before the later latestTime_ will be 
duplicated.

Recommended changes:
When the two latestTime_'s are not equal, instead send another query to 
the server for the region with the earlier latestTime_.  Then, you can 
know that you have all wrecks in both regions that occured before the 
later latestTime_, and can therefore accept that one as the latestTime_ of 
the new region.  There will be no duplication on the next Periodic Full 
Update.

Pros / Cons of current method?
The current method is simpler and easier to implement.  It also involves 
less requests sent to the server.

Pros / Cons of recommended method?
It would eliminate more point duplication, thereby saving memory on the 
device.

Please use labels and text to provide additional information.

Note:  As I was typing this, I realize that it may be easier to instead 
switch latestTime_ to be the time right before the request was sent to the 
server instead of the time of the last returned wreck.  This would greatly 
reduce (but not quite eliminate) the duplication problem, without causing 
the extra requests of the method detailed above.

Original issue reported on code.google.com by shcampbe...@gmail.com on 1 Jul 2009 at 10:43

[GoogleCodeExporter]

Scott, 

Great catch! To clear up what you are saying, here are examples. My suggestions 
are below

An Update coming in with a latest time of 1500, when the cache has a latest 
time of
1000. 
Currently, the min time will be used as the caches latest time, so the cache 
will
expand by adding the update region, and the latest cache time will be 1000. 
When a Full Update occurs, the cache will duplicate any points in the updated 
region
that exist between time 1000 and 1500. 

An Update coming in with a latest time of 1000, when the cache has a latest 
time of
1500. 
Currently, the min time will be used as the caches latest time, so the cache 
will
expand by adding the update region, and the latest cache time will be 1000. 
When a Full Update occurs, the cache will duplicate any points in the original 
cache
area that exist between time 1000 and 1500. 

Therefore, we have point duplication with every single expansion. 

Recommendations: 

I think changing the meaning of the latestTime_ variable will not work. The 
variable
is passed to the server, and if we changed the meaning to represent something 
that is
client-specific, the server will no longer be able to understand that, and I 
don't
think we want it to. 

This exposes the design problem - a Full Update can come in while an expansion 
is
happening. I need to fix this, but it is too big of a change to make right now 
(it
will definitely happen though). 

There are fundamentally two ways to fix this - your idea of more posts, or by
updating the Route and the Waypoints to be able to check for equality. Rather 
than
always adding all points to a given list, you do something like this:

list.removeAll(new_points)
list.addAll(new_points)

This is a poor mans way of merging two lists, and the isEquals() method needs 
to be
correct for this to work. 

I would prefer this method over the more POSTs method. It will result in some
duplicated points coming from the server, so any battery gain in fewer POSTs is 
lost
because we duplicate some points, so POSTs can be slightly larger. 

However, there is an added advantage of doing it client side - it protects 
against
edge cases that are very rare. For example, sometimes the Projection encodes 
exactly
the same point differently, because of round off error. This means that our 
cache
squares, which we think of as being hard lines, actually have 'fuzzy' edges. The
equality checking would protect against this. 

Original comment by hamiltont on 2 Jul 2009 at 4:15

• Added labels: Priority-High, Type-Defect
• Removed labels: Type-Enhancement

[GoogleCodeExporter]

Hamilton,

That's a great point about using the isEquals() method to remove duplications.  
You're right that that would be a better solution than the more POSTs method I 
was 
talking about.  However, I do think that changing the meaning of latestTime_ 
would 
work.  As I understand it, the server just compares the maxtime against the 
time of 
its wrecks, and returns ones with a later time.  So, if the maxtime is the time 
of 
the post instead of the time of the last wreck, the server would still return 
any 
new wrecks, since they would have had to occur after the last post or they 
would 
have been returned in the last one.  Does that make sense?  Am I 
mis-understanding 
how the server does things?

Original comment by shcampbe...@gmail.com on 2 Jul 2009 at 4:40

[GoogleCodeExporter]

Scott, 

That logic makes perfect sense, and would be totally correct if we could rely 
on the
phone time to always 1)be accurate and 2)be synced. 

For example, if a user randomly changes their time to 1970, we don't want that 
time
showing up on the server as when the last post was sent, causing us to resend 
all
points. 

Original comment by hamiltont on 2 Jul 2009 at 4:43

[GoogleCodeExporter]

To elaborate, it is a matter of trust. Currently, we are only trusting the data 
we
receive from the server as 'verified' data. We don't currently have any way to 
verify
the users time. If they are in a different locale, then all bets are off and 
they may
be receiving wrecks an hour old! 

Original comment by hamiltont on 2 Jul 2009 at 4:51

[GoogleCodeExporter]

I understand, that makes perfect sense!

Thanks,
Scott

Original comment by shcampbe...@gmail.com on 2 Jul 2009 at 5:01