Python 3

[y26805]

Since this repo was written in the era of Python 2.7, I have adapted the code to Python 3 in my forked version of the repo. Also added instructions for how to get Street View API key.

https://github.com/y26805/Treepedia_Public

Hope this helps someone.

[y26805]

Found this fork randomly but this Notebook is pretty good at explaining how to use the Treepedia library (also code in this repo is good for Python 3 too).

https://github.com/xiaojianggis/Treepedia_Public/blob/master/Treepedia/GreenViewIndex_computing.ipynb

edit: updated URL https://github.com/xiaojianggis/Treepedia_Public/tree/master/Treepedia

[ubi007]

Thank you for converting the code to Python3. However I'm getting the following error, any leads on how to get around it?

You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE175708>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE1F5088>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE175708>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE10F488>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE175708>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE10F488>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE175708>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE10F488>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE10F488>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE175708>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE10F488>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE175708>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE10F488>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE175708>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE10F488>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE175708>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE10F488>)

[y26805]

Hi, thanks for reporting this issue! Is the file you are using in WGS84? If so, can you share the file with me so I can check to see if I can reproduce this?

Also, will be helpful to know your Python version (3.7, 3.8, or others).

"This is the best of all possible worlds." *— *Gottfried Wilhelm Leibniz

On Mon, 16 Aug 2021 at 20:39, ubi007 @.***> wrote:

Thank you for converting the code to Python3. However I'm getting the following error, any leads on how to get around it?

You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE175708>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE1F5088>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE175708>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE10F488>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE175708>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE10F488>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE175708>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE10F488>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE10F488>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE175708>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE10F488>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE175708>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE10F488>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE175708>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE10F488>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE175708>) You should make sure the input shapefile is WGS84 (<class 'ValueError'>, ValueError('cannot convert float NaN to integer',), <traceback object at 0x0000019EFE10F488>)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mittrees/Treepedia_Public/issues/5#issuecomment-899441760, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADTPUSDKHD5E4QNYSXSRSZTT5D2ILANCNFSM4M34F3SQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

[ubi007]

Thank you for a prompt reply. I've noticed that my shapefile was not in WGS84. I am converting it right now and I'll try it again once it's ready. I'm using Python 3.6.8 One thing I've noticed is that in createPoint.py, MultiLineString is not processed and I'm not sure how this would impact the final result.

[y26805]

Yeah the code that skips MultiLineString was from the original code I believe. Otherwise it gives an error.

The portion of MultiLineString in the shape files I've run the code on is really small, and hence that didn't have any impact on the output.

"This is the best of all possible worlds." *— *Gottfried Wilhelm Leibniz

On Mon, 16 Aug 2021 at 23:07, ubi007 @.***> wrote:

Thank you for a prompt reply. I've noticed that my shapefile was not in WGS84. I am converting it right now and I'll try it again once it's ready. I'm using Python 3.6.8 One thing I've noticed is that in createPoint.py, MultiLineString is not processed and I'm not sure how this would impact the final result.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mittrees/Treepedia_Public/issues/5#issuecomment-899540153, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADTPUSDE4Y4ZBZWHOKLVUXLT5ELRHANCNFSM4M34F3SQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

[minmax10]

it's gone ...

https://github.com/xiaojianggis/Treepedia_Public/blob/master/Treepedia/GreenViewIndex_computing.ipynb

[Polothree]

Hi, I have the following error: urllib.error.HTTPError: HTTP Error 404: Not Found

Is this link in the code style ok ? urlAddress = r'http://maps.google.com/cbk?output=xml&ll=%s,%s'%(lat,lon)

Thanks for your help

[y26805]

it's gone ...

https://github.com/xiaojianggis/Treepedia_Public/blob/master/Treepedia/GreenViewIndex_computing.ipynb

@minmax10 Looks like the notebook got moved, but here's the new location (within the same repo) https://github.com/xiaojianggis/Treepedia_Public/tree/master/Treepedia

[y26805]

urlAddress = r'http://maps.google.com/cbk?output=xml&ll=%s,%s'%(lat,lon)

@Polothree which version of Python are you using? Also, it will be helpful if you could provide a more complete code sample (include your import statements, how you defined your variables, etc..)

[Polothree]

Hi @y26805 , I am using python 3.10.6 version. I am testing the Cambridge shapefile example.

It seems that on the code version you release here: https://github.com/xiaojianggis/Treepedia_Public/tree/master/Treepedia

This part of the code has changed: urlAddress = r'https://maps.googleapis.com/maps/api/streetview/metadata?size=600x300&location=%s,%s&heading=-45&pitch=42&fov=110&key=%s'%(lon, lat, key)

I need a API key. Is that the problem ?

Thanks for your help !

Here is the code I am using:

                   from datetime import datetime

          def GSVpanoMetadataCollector(samplesFeatureClass, ouputTextFolder, batchNum, greenmonth, year=""):
              '''
              This function is used to call the Google API url to collect the metadata of
              Google Street View Panoramas. The input of the function is the shpfile of the create sample site, the output
              is the generate panoinfo matrics stored in the text file

              Parameters: 
                  samplesFeatureClass: the shapefile of the create sample sites
                  batchNum: the number of sites proced every time. If batch size is 1000, the code will save metadata of every 1000 point to a txt file.
                  ouputTextFolder: the output folder for the panoinfo
                  greenmonth: a list of the green season, for example in Boston, greenmonth = ['05','06','07','08','09']
                  year: optional. if specified, only panos dated in that year or older will be returned

              '''

              import urllib
              import xmltodict
              from osgeo import ogr, osr, gdal
              import time
              import os,os.path
              import math
              import streetview
              import pprint

              if not os.path.exists(ouputTextFolder):
                  os.makedirs(ouputTextFolder)

              driver = ogr.GetDriverByName('ESRI Shapefile')
              if driver is None:
                  print('Driver is not available.')

              dataset = driver.Open(samplesFeatureClass)
              if dataset is None:
                  print('Could not open %s' % (samplesFeatureClass))

              layer = dataset.GetLayer()
              sourceProj = layer.GetSpatialRef()
              targetProj = osr.SpatialReference()
              targetProj.ImportFromEPSG(4326) # change the projection of shapefile to the WGS84

              # if GDAL version is 3.0 or above
              if gdal.__version__.startswith('2.') is False:
                  targetProj.SetAxisMappingStrategy(osr.OAMS_TRADITIONAL_GIS_ORDER)

              transform = osr.CoordinateTransformation(sourceProj, targetProj)

              # loop all the features in the featureclass
              feature = layer.GetNextFeature()
              featureNum = layer.GetFeatureCount()
              batch = math.ceil(featureNum/batchNum)

              for b in range(batch):
                  # for each batch process num GSV site
                  start = b*batchNum
                  end = (b+1)*batchNum
                  if end > featureNum:
                      end = featureNum

                  ouputTextFile = 'Pnt_start%s_end%s.txt'%(start,end)
                  ouputGSVinfoFile = os.path.join(ouputTextFolder,ouputTextFile)

                  # skip over those existing txt files
                  if os.path.exists(ouputGSVinfoFile):
                      continue

                  time.sleep(1)

                  with open(ouputGSVinfoFile, 'w') as panoInfoText:
                      # process num feature each time
                      for i in range(start, end):
                          feature = layer.GetFeature(i)        
                          geom = feature.GetGeometryRef()

                          # trasform the current projection of input shapefile to WGS84
                          #WGS84 is Earth centered, earth fixed terrestrial ref system
                          geom.Transform(transform)
                          lon = geom.GetX()
                          lat = geom.GetY()

                          # get the meta data of panoramas 
                          urlAddress = r'http://maps.google.com/cbk?output=xml&ll=%s,%s'%(lat,lon)

                          print(urlAddress)

                          time.sleep(0.05)
                          # the output result of the meta data is a xml object
                          metaDataxml = urllib.request.urlopen(urlAddress)
                          metaData = metaDataxml.read()    

                          data = xmltodict.parse(metaData)

                          # in case there is not panorama in the site, therefore, continue
                          if data['panorama']==None:
                              continue
                          else:
                              panoInfo = data['panorama']['data_properties']   
                              panoDate, panoId, panoLat, panoLon = getPanoItems(panoInfo)

                              if check_pano_month_in_greenmonth(panoDate, greenmonth) is False or year != "":
                                  panoLst = streetview.panoids(lon=lon, lat=lat)
                                  sorted_panoList = sort_pano_list_by_date(panoLst)
                                  panoDate, panoId, panoLat, panoLon = get_next_pano_in_greenmonth(sorted_panoList, greenmonth, year)

                              print('The coordinate (%s,%s), panoId is: %s, panoDate is: %s'%(panoLon,panoLat,panoId, panoDate))
                              lineTxt = 'panoID: %s panoDate: %s longitude: %s latitude: %s\n'%(panoId, panoDate, panoLon, panoLat)
                              panoInfoText.write(lineTxt)

                  panoInfoText.close()

          def getPanoItems(panoInfo):
              # get the meta data of the panorama
              panoDate = panoInfo['@image_date']
              panoId = panoInfo['@pano_id']
              panoLat = panoInfo['@lat']
              panoLon = panoInfo['@lng']
              return panoDate, panoId, panoLat, panoLon

          def check_pano_month_in_greenmonth(panoDate, greenmonth):
              month = panoDate[-2:]
              return month in greenmonth

          def sort_pano_list_by_date(panoLst):
              def func(x):
                  if 'year'in x:
                      return datetime(year=x['year'], month=x['month'], day=1)
                  else:
                      return datetime(year=1, month=1, day=1)
              panoLst.sort(key=func, reverse=True)
              return panoLst

          def get_next_pano_in_greenmonth(panoLst, greenmonth, year):
              greenmonth_int = [int(month) for month in greenmonth]

              for pano in panoLst:
                  if 'month' not in pano.keys():
                      continue
                  month = pano['month']
                  pano_year = pano['year']
                  if month in greenmonth_int and (year == "" or year >= pano_year):
                      return get_pano_items_from_dict(pano)

              print(f"No pano with greenmonth {greenmonth} found. ")
              if year != "":
                  print(f"No pano with year {year} found. ")
              print("Returning info of latest pano")
              return get_pano_items_from_dict(panoLst[0])

          def get_pano_date_str(panoMonth, panoYear):
              return str(panoYear) + '-' + str(panoMonth).zfill(2)

          def get_pano_items_from_dict(pano):
              panoDate = get_pano_date_str(pano['month'], pano['year'])
              panoId = pano['panoid']
              panoLat = pano['lat']
              panoLon = pano['lon']
              return panoDate, panoId, panoLat, panoLon

          # ------------Main Function -------------------    
          if __name__ == "__main__":
              import os, os.path

              os.chdir(r"\\zsfA\SIG\GREENVIEW\Treepedia_Public-master-py3\sample-spatialdata")
              root = os.getcwd()
              inputShp = os.path.join(root,'Cambridge20m.shp')
              outputTxtFolder = os.path.join(root, "metadata")
              batchNum = 1000

              greenmonth = ['01','02','03','04','05','06','07','08','09','10','11','12']
              GSVpanoMetadataCollector(inputShp, outputTxtFolder, batchNum, greenmonth)

              # to get pano dated in 2018 or older (optional)
              # year = 2018 
              # GSVpanoMetadataCollector(inputShp, outputTxtFolder, batchNum, greenmonth, year)

[y26805]

hi @Polothree, yup I believe the endpoint this project was previously using may have been deprecated. Using the official endpoint with an API key is probably the way to go.

I have a section in my fork's README that may help you: https://github.com/y26805/Treepedia_Public#using-the-google-maps-api