observationunits returning incorrect results

[dlebauer]

I was just trying to download our traits via brapi … the observationsunits endpoint, e.g. https://terraref.org/brapi/v1/observationunits?studyDbId=6000000010, is very different from the example in the api documentation https://terraref.org/brapi/v1/ui/#/ , e.g. it doesnt have the observation elements etc. I think these used to be there … do you know what happened or am I hitting the wrong endpoint?

e.g. this is the example from the documentation:

"result": {
    "data": [
      {
        "X": "1",
        "Y": "1",
        "blockNumber": "1",
        "entryNumber": "1",
        "entryType": "TEST",
        "germplasmDbId": "1",
        "germplasmName": "Name001",
        "observationLevel": "plot",
        "observationLevels": "block:1;plot:1",
        "observationUnitDbId": "1",
        "observationUnitName": "Plot 1",
        "observationUnitXref": [
          {
            "id": "SAMEA179865230",
            "source": "ebi.biosample"
          },
          {
            "id": "INRA:CoeSt6 _SMH03",
            "source": "gnpis.lot"
          },
          {
            "id": "239865",
            "source": "kernelDB"
          }
        ],
        "observations": [
          {
            "collector": "A. Technician",
            "observationDbId": "1",
            "observationTimeStamp": "2013-06-15T02:03:51Z",
            "observationVariableDbId": "MO_123:100002",
            "observationVariableName": "Plant height",
            "season": "fall 2011",
            "value": "1.2"
          },
          {

and this is the result that I am getting:

 "result": {
    "data": [
      {
        "observationUnitName": "Earth",
        "seasonDbId": "a8317fa6b59bfa8cfe29004423f5613b",
        "studyDbId": "6000000010",
        "treatmentDbId": 6000000020,
        "treatments": [
          {
            "factor": "MAC Sorghum Season 4",
            "modality": "Pre-plant Nitrogen fertilizer incorporated in the field"
          }
        ]
      },
      {
        "observationUnitName": "MAC Field Scanner Season 4 Range 51 Column 7 E",
        "seasonDbId": "a8317fa6b59bfa8cfe29004423f5613b",
        "studyDbId": "6000000010",
        "treatmentDbId": 6000000020,
        "treatments": [
          {
            "factor": "MAC Sorghum Season 4",
            "modality": "Pre-plant Nitrogen fertilizer incorporated in the field"
          }
        ]
      },

cc @KristinaRiemer @robkooper @Chris-Schnaufer @max-zilla

[tcnichol]

Let me look into this along with Max. I haven't done much with any deployed instances in a while so I am not sure if and when anything was changed.

[max-zilla]

this was related to a discussion we had about the timing out of large queries because of the massive number of observations. observations are only included when you specify a specific observationUnitDbId (i.e. plotname).

https://brapi.workbench.terraref.org/brapi/v1/observationunits?studyDbId=6000000010&observationUnitDbId=MAC%20Field%20Scanner%20Season%204%20Range%2051%20Column%208%20E try that one, for example

[max-zilla]

for readability, I just added to your URL:

&observationUnitDbId=MAC Field Scanner Season 4 Range 51 Column 8 E

[dlebauer]

Is there a better way to handle this? It seems like this choice not to return data because it is too big may cause confusion down the line, as it has for @KristinaRiemer and I when trying to use this endpoint ...

For example, could we reduce the issue w/ large results if we return jsonb?

[dlebauer]

also:

• in the result you provided it looks like each observationDbId is repeated so results 1&2 are duplicates, 3&4, and so on
• should return germplasmDbId, germplasmName, and observationLevel = plot where sites.geometry is a polygon

[max-zilla]

will take a look, should be able to quickly address duplication at least, should have time tomorrow.