osgeonepal / geotile

GeoTile is the python library for tiling the geographic raster data (eg. Tiff etc)
https://geotile.readthedocs.io
MIT License
74 stars 9 forks source link

Need to optimize the generate_tiles function. #55

Open iamtekson opened 8 months ago

iamtekson commented 8 months ago

I tested two different code,

  1. Frist version:
for i, s2 in enumerate(s2_imagery):
    print(i, s2)

    # if i==1:
    #     break

    # get the imagery location name (eg. pokhara, kathmandu, janakpur etc.)
    imagery_location = os.path.basename(s2).split('_')[0] 

    # get year
    year = os.path.basename(s2).split('_')[-2]

    # read imagery
    s2_gt = GeoTile(s2)
    s2_gt.generate_tiles(save_tiles=False)
    s2_gt.convert_nan_to_zero()
    s2_gt.normalize_tiles()

    # # save tiles
    s2_gt.save_tiles(f"{OUTPUT_TILES}_{year}/{imagery_location}", prefix=f'{imagery_location}_s2_')

    print(imagery_location)

    # # # get the mask file based on each locations (eg. rapti, tinau, ratu)
    for j, mask in enumerate(s2_masks):

        if i == j:
            print('Imagery location: ', imagery_location)
            print('Mask location: ', os.path.basename(mask).split('_')[0])

            s2_gt.rasterization(mask, f"{OUTPUT_MASK}/{imagery_location}_{year}_s2_mask.tif")
  1. Second version:
for i, mask in enumerate(mask_files):
    gt = GeoTile(mask)

    mask_location = os.path.basename(mask).split('_')[0]
    year = os.path.basename(mask).split('_')[1]

    # generate the mask file based on each binary output
    gt.generate_tiles(f"../../../data/tiles/mask_rivers_{year}/{mask_location}", prefix=f'{mask_location}_s2_')

The first version tool only 2 min 27 second to run the code however, the second version took 6 min 39 second. My guess is there is something going wrong with parameter save_tiles=True.