Clay-foundation / model

The Clay Foundation Model (in development)
https://clay-foundation.github.io/model/
Apache License 2.0
262 stars 30 forks source link

Make RGB previews at the embedding level #115

Closed brunosan closed 6 months ago

brunosan commented 6 months ago

To ease understanding the model, we should create an RGB of the same extent and file that creates the embeddings. As of now, this means converting the 13 stack datacube into a lossy (no need to be accurate, it's for the human loop) png with RGB. .

PS: As the model evolves with other inputs, it will also be a good way to debug, checking the corresponding input file (Even if it's just RGB, or their equivalents).

The logic could be separate (so we run it on existing embeddings), or part of the datacube or embedding creation.

yellowcap commented 6 months ago

The first test could be to use titiler for visualizing RGB from the geotiff chips. If that does not work well, let's create png data.

brunosan commented 6 months ago

Scope here is the easiest way to "see" how an embedding looks like. For debugging and UI purposes. If that's creating a lossy jpg and saving it, that's fine. If that's a URL call, fine too. Even if the RGB is not exactly the input data at that exact time.

Tagging @MaceGrim for the implications for the app v0

brunosan commented 6 months ago

This is done on #132.

specifically:

def reduce_precision(coordinates, precision=2):
    """
    Reduce the precision of coordinates.
    Needed for Mapbox API requests.
    """
    if isinstance(coordinates[0], (list, tuple)):
        return [reduce_precision(coord, precision) for coord in coordinates]
    else:
        return round(coordinates[0], precision), round(coordinates[1], precision)

def get_mapbox_image(polygon, access_token):
    """
    Get a satellite image from Mapbox API.
    polygon: shapely.geometry.Polygon
    access_token: Mapbox access token
    """
    try:
        rounded_coords = reduce_precision(list(polygon.exterior.coords))
        geojson = {
            "type": "FeatureCollection",
            "features": [
                {
                    "id": "0",
                    "type": "Feature",
                    "properties": {
                        "stroke": "#f00000",
                        "stroke-width": 10,
                        "stroke-opacity": 1,
                        "fill": "#f00000",
                        "fill-opacity": 0,
                    },
                    "geometry": {"type": "Polygon", "coordinates": [rounded_coords]},
                }
            ],
        }
        encoded_geojson = quote(json.dumps(geojson))
        min_lon, min_lat = min(rounded_coords, key=lambda x: (x[0], x[1]))
        max_lon, max_lat = max(rounded_coords, key=lambda x: (x[0], x[1]))
        mapbox_url = (
            "https://api.mapbox.com/styles/v1/mapbox/satellite-streets-v12/static/"
            + f"geojson({encoded_geojson})"
            + f"/[{min_lon},{min_lat},{max_lon},{max_lat}]/"
            + f"512x512?access_token={access_token}"
        )

        response = requests.get(mapbox_url)

        all_ok = 200
        if response.status_code == all_ok:
            return response.content, None
        else:
            return None, f"Error: {response.status_code}, {response.text}"

    except Exception as e:
        return None, f"Error: {str(e)}"