Orientation of panorama

[nikste]

Is there a way to find out the orientation of the panorama? As in degrees from north for example ? maybe in the rest response? thanks!

[msm1089]

The panoramas cover 360 degrees (of course!), so there is no orientation as such. What is it you are trying to do? I might be able to help as I have used this package and other methods to download partial panoramas with specified direction and field of view.

[nikste]

thanks @msm1089 ! I'm not talking about the field of view (which is indeed 360° always). Some background: I'd like to triangulate stuff between two consecutive panorama images. it seems that the driving direction is always in the center of the panorama image. This is sort of ok for me as long as the car drives in a straight line. As soon as the car turns, however, I don't know which angle from one panorama image corresponds to which angle of the other panorama, because they are all relative to themselves. Does this make any sense? So, I'd be looking for some absolute angle of the pose of the car, something like "the car (and therefore the center of the panorama) is pointing west".

[msm1089]

I don't fully understand your use case sorry, but does the vehicle heading matter? Could be it is not needed for what you want to do. You can get a given angle/field of view (fov) section of a specific panorama by passing the parameters in a querystring, like:

https://maps.googleapis.com/maps/api/streetview?pano=H0ah4v1FiuEYCuBtxrjh9w&fov=90&heading=-179.18078114315978&size=480x360&source=outdoor&key=YOUR_API_KEY

Since the value 0 for the parameter heading always corresponds to North (True North and not magnetic, I believe, but you may want to check), you can perhaps use this absolute compass bearing to arrive at your desired one.

If it helps I have a Jupyter notebook showing the concept. It was part of project at uni. The idea was to be able to take a photo (with given GPS coords) and then use the Google StreetView API and Computer Vision to estimate the heading of the camera that took it. Was a nice idea, but the results were not impressive. Turns out it is pretty damn hard to use CV to match an arbitrary random photo to a nearby panorama node (who'd have thunked it?!). But there may be some stuff in the notebook you can use or that sparks further insight for your particular case.

I'll put it up on in a repo soon and link to it here for you.

[msm1089]

@nikste I have uploaded the notebook I mentioned to https://github.com/msm1089/view-direction-estimation. The notebook is a bit messy, apologies, but I hope it helps you. Feel free to ask for clarification on anything.

[nikste]

thanks @msm1089, thanks a lot for uploading the code! I see what you mean with the heading parameter in the URL, indeed i was looking for something like this. It seems this repository uses 2 different apis to download images: https://github.com/robolyst/streetview/blob/master/streetview/__init__.py#L143 (panoramas without fixed heading) https://github.com/robolyst/streetview/blob/master/streetview/__init__.py#L237 (official google api, only part of the panorama with limited resolution and some reprojection is going on there i think)

In essence I'd like to have the images from the first link, but with some fixed world coordinate system heading like in your second link. Maybe i can use some of the code you provided to align them. This seems like a much simpler problem than what you were trying to do with the semantic segmentation aligning images from 2 sources

[Jameswlb]

thanks @msm1089, thanks a lot for uploading the code! I see what you mean with the heading parameter in the URL, indeed i was looking for something like this. It seems this repository uses 2 different apis to download images: https://github.com/robolyst/streetview/blob/master/streetview/__init__.py#L143 (panoramas without fixed heading) https://github.com/robolyst/streetview/blob/master/streetview/__init__.py#L237 (official google api, only part of the panorama with limited resolution and some reprojection is going on there i think)

In essence I'd like to have the images from the first link, but with some fixed world coordinate system heading like in your second link. Maybe i can use some of the code you provided to align them. This seems like a much simpler problem than what you were trying to do with the semantic segmentation aligning images from 2 sources

May I ask whether you have solve this problem? I also want the orientation information of the GSV....How can I get this orientation information?

[nikste]

@Jameswlb unfortunately i did not solve it, as far as i can see there are two options:

1.use the official api, then you can get the orientation, but can only download rectified images with a certain resolution

2. Use the unofficial api with full panoramic image resolution, but no orientation

[Jameswlb]

Thank you. Maybe the official api is the best solution for my task, but it costs a lot......Maybe I will find other solutions.

2021-07-06 19:37:13"Nikolaas Steenbergen" @.***>写道：

@Jameswlb unfortunately i did not solve it, as far as i can see there are two options:

1.use the official api, then you can get the orientation, but can only download rectified images with a certain resolution

2. Use the unofficial api with full panoramic image resolution, but no orientation

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[msm1089]

Just an idea, but it may be possible to get several portions of the whole panorama, that fully cover it. E.g. 4 sections covering 90 degrees each. Then if you can find the reverse of the transformation used by Google to produce these, maybe can stitch them together to form the full panorama. Bit of a long shot and not sure how complicated this would be.

[nikste]

@msm1089 Yes! I think that should be possible. I was thinking about something similar, but did not have the time to implement something like this. Also the images are restricted to something like 640x640 pxl, so you'd need a lot more images to get a high resolution panorama. @Jameswlb not sure what you want to achieve and how many images you need for that, but they are pretty lenient for pricing for smaller amounts (you get thousands of images for free per month iirc).

[B0RJA]

The function streetview.panoids returns the closest panorama to a position and also its neighboring panoramas. Assuming the current panorama and its two nearest neighbors are in a straight line, the current bearing can be approximated.

import numpy
import math
from math import cos, asin, sqrt
import statistics
import streetview

def distance(lat1, lon1, lat2, lon2):
    p = 0.017453292519943295
    hav = 0.5 - cos((lat2-lat1)*p)/2.0 + cos(lat1*p)*cos(lat2*p) * (1.0-cos((lon2-lon1)*p)) / 2.0
    return 12742.0 * asin(sqrt(hav))

def closest(data, v):
    return sorted([p for p in points], key=lambda p: distance(center['lat'],center['lon'],p['lat'],p['lon']))

def get_bearing(lat1, long1, lat2, long2):
    dLon = (long2 - long1)
    x = math.cos(math.radians(lat2)) * math.sin(math.radians(dLon))
    y = math.cos(math.radians(lat1)) * math.sin(math.radians(lat2)) - math.sin(math.radians(lat1)) * math.cos(math.radians(lat2)) * math.cos(math.radians(dLon))
    brng = numpy.arctan2(x,y)
    brng = numpy.degrees(brng)
    return brng

panoids = streetview.panoids(lat=40.420153, lon=-3.703631)

# current pano from latest year
center = sorted([p for p in panoids if 'year' in p], key=lambda p: p['year'])[-1]

# neighboring points
points = [p for p in panoids if 'year' not in p]
p1 = closest(points, center)[0]
p2 = closest(points, center)[1]

b1 = get_bearing(p1['lat'],p1['lon'],p2['lat'],p2['lon'])
b2 = get_bearing(p1['lat'],p1['lon'],center['lat'],center['lon'])
b3 = get_bearing(center['lat'],center['lon'],p2['lat'],p2['lon'])
diff = max(b1+360, b2+360, b3+360) - min(b1+360, b2+360, b3+360)

# max threshold to avoid curves
if diff < 22.5 :
    print(statistics.median([b1+360,b2+360,b3+360])-360)