Cannot delete multiple `functional_aggregation_agg` documents via single API request

[aclum]

I'm having trouble using the API to delete functional_annotation_agg records.

I tried the following

curl -X 'POST' \
  'https://api-dev.microbiomedata.org/queries:run' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer $TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{
  "delete": "functional_annotation_agg",
  "deletes": {"metagenome_annotation_id":"nmdc:73b7ba35a832f4e18f885bcbb88b967d"}
}'

but get a 422

error details

{
  "detail": [
    {
      "type": "missing",
      "loc": [
        "body",
        "CollStatsCommand",
        "collStats"
      ],
      "msg": "Field required",
      "input": {
        "delete": "functional_annotation_agg",
        "deletes": {
          "metagenome_annotation_id": "nmdc:73b7ba35a832f4e18f885bcbb88b967d"
        }
      },
      "url": "https://errors.pydantic.dev/2.6/v/missing"
    },

[aclum]

Not sure it is practical to do this w/the API b/c there are 4057 records but it would be nice to be able to do that.

[aclum]

cc @mbthornton-lbl

[eecavanna]

I think the "deletes" value does not have the shape the API expects. I recommend looking at the blurb for that endpoint in the Swagger UI. I'll post a screenshot/example later (when back at computer), in 5 mins or so.

[eecavanna]

[eecavanna]

@aclum, thanks for including the offending curl command.

I think if you change the curl command like this, the API will accept the request:

curl -X 'POST' \
  'https://api-dev.microbiomedata.org/queries:run' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer $TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{
  "delete": "functional_annotation_agg",
+ "deletes": [{"q": {"metagenome_annotation_id":"nmdc:73b7ba35a832f4e18f885bcbb88b967d"}, "limit": 1}]
- "deletes": {"metagenome_annotation_id":"nmdc:73b7ba35a832f4e18f885bcbb88b967d"}
}'

That change is based on the example shown in the screenshot attached to my previous comment.

[eecavanna]

Unassigning @PeopleMakeCulture as I don't think her attention is necessary on this issue.

[aclum]

That only deletes one record I need 4057 of them deleted. When I try to delete (on runtime dev) with the the mongo id I get an internal server error ie

{
    "delete": "functional_annotation_agg",
    "deletes": [
        {
            "q": {
                "_id": {
                    "$oid": "61e3411de80c1caf55176cee"
                }
            },
            "limit": 1
        }
    ]
}

[eecavanna]

That only deletes one record I need 4057 of them deleted.

Ah, gotcha! I'll look into whether the "limit" property is necessary and whether there is a special value for that property that means "no limit." I will report back.

When I try to delete (on runtime dev) with the the mongo id I get an internal server error

Interesting. While I'm looking into the "limit" property, I'll keep an eye out for code that would get tripped up on a request like the one in your latest comment. This is an area where I think I will want @PeopleMakeCulture's attention after all.

[eecavanna]

Regarding the "limit" property, I think the only options the API allows are 0 and 1. That's based on this code:

class DeleteStatement(BaseModel):
    q: Document
    limit: OneOrZero
    hint: Optional[Dict[str, OneOrMinusOne]] = None

Source: nmdc_runtime/api/models/query.py

I assume a value of 0 means "no limit" (a.k.a. "infinity is the limit"). Will you try with a "limit" value of 0?

[eecavanna]

@aclum told me that when she specified the limit as 0, she was able to delete multiple documents via a single API request.

Re-adding @PeopleMakeCulture. Hi @PeopleMakeCulture, will you (a) confirm that that is the general case; i.e. when someone specifies a limit of 0, the Runtime API interprets that as "no limit" (i.e. the limit is "infinity"); and (b) if that is the general case, add a note about that behavior to the /queries:run endpoint's documentation for future reference?