Only update affected markers, make UpdateMarkers smarter

[Yeetii]

I think this is the largest performance drain when moving around the map a lot