Clay-foundation / model

The Clay Foundation Model (in development)
https://clay-foundation.github.io/model/
Apache License 2.0
262 stars 30 forks source link

Embedding factory script looping through each MGRS tile #125

Closed weiji14 closed 1 month ago

weiji14 commented 6 months ago

Jupyter notebook script to generate GeoParquet embedding files on a per MGRS tile basis.

Steps:

  1. The script first generates an mgrs_world.txt file with a list of MGRS code names like 12ABC. Need to run this command first:
    aws s3 ls s3://clay-tiles-02/02/ | tr -s ' ' |  cut -d ' ' -f 3 | cut -d '/' -f 1 > mgrs_world.txt
  2. A for-loop then goes through each MGRS tile, with the model running the prediction to generate GeoParquet files that are uploaded to s3.

Notes:

Closes #120

yellowcap commented 1 month ago

This is out of date I think. Feel free to reopen if appropriate.