vincentsarago
commented
11 months ago @philvarner I think what you want is just to create a custom write method which accept a ttl
such as
def write(self, overwrite: bool = False, ttl: Optional[int] = None, **kwargs: Any):
if not self._table_exists():
self._create_table(**kwargs)
if self._mosaic_exists():
if not overwrite:
raise MosaicExistsError(
f"Mosaic already exists in {self.table_name}, use `overwrite=True`."
)
self.delete()
items: List[Dict[str, Any]] = []
# Create Metadata item
# Note: `parse_float=Decimal` is required because DynamoDB requires all numbers to be
# in Decimal type (ref: https://blog.ruanbekker.com/blog/2019/02/05/convert-float-to-decimal-data-types-for-boto3-dynamodb-using-python/)
meta = json.loads(
self.mosaic_def.model_dump_json(exclude={"tiles"}), parse_float=Decimal
)
if ttl:
meta["ttl"] = ttl
items.append(
{"quadkey": self._metadata_quadkey, "mosaicId": self.mosaic_name, **meta}
)
# Create Tile items
for quadkey, assets in self.mosaic_def.tiles.items():
item = {"mosaicId": self.mosaic_name, "quadkey": quadkey, "assets": assets}
if ttl:
item["ttl"] = ttl
items.append(item )
self._write_items(items)DynamoDB TTL seems something that could be useful useful to more users (🤷) so we could add it by default in this repo
philvarner
In FilmDrop UI's use of Element84/titiler-mosaicjson, we use the mosaic in an ephemeral way, such that a mosaic is discarded as soon as the user changes the search or moves away. Currently, the DynamoDB backend keeps all of these entries that are never going to be used again.
I'd like to add support in the DynamoDB backend to support passing an optional timeToLive parameter that would allow DynamoDB to delete these entries after the TTL. I can do this work, but wanted buy-in on the approach. The simplest option would be to add kwargs to
BaseBackend#write, and the backends can do whatever they want with those. Another option would be to add an argument likeadditionalValuesof type Dict[str, Any] and these values would be written as additional fields in the DynamoDB entry, so I could configure my Table to use a TTL field oftimeToLive, and then pass a dict like{ "timeToLive": 1696430626}that would get written with each DDB Item.