CSV downloaded with synchronous "download" gives JSON-structured output

[Pratichhya]

When downloading a csv file using a synchronous approach. It returns data in JSON format.

Performed: cube.download("my_result.csv")

Resulted output sample in a csv file:

{"2020-06-01T00:00:00Z":[[0.748832450333613],[0.4964776563741113],[0.6570071710833252],[0.4878170884663786]],"2020-06-04T00:00:00Z":[[0.0230362131024838],[0.0210558454592638],[0.020667860950573],[0.0214809115294978]],"2020-06-06T00:00:00Z":[[0.7757732911277135],[0.5010382193731348],[0.719923551958639],[0.3880461017364886]],"2020-06-09T00:00:00Z":[[0.0369665531354938],[0.0385640003326478],[0.031821943594066],[0.019730403069427]]}

[soxofaan]

can you also include the preceding steps how to build the cube? or the output of cube.print_json()

[Pratichhya]

Hi @soxofaan ,

the process was:

Also, its process garph is:

{
  "process_graph": {
    "loadcollection1": {
      "process_id": "load_collection",
      "arguments": {
        "bands": [
          "B04",
          "B08",
          "SCL"
        ],
        "id": "SENTINEL2_L2A",
        "spatial_extent": null,
        "temporal_extent": [
          "2020-06-01",
          "2020-10-01"
        ]
      }
    },
    "reducedimension1": {
      "process_id": "reduce_dimension",
      "arguments": {
        "data": {
          "from_node": "loadcollection1"
        },
        "dimension": "bands",
        "reducer": {
          "process_graph": {
            "arrayelement1": {
              "process_id": "array_element",
              "arguments": {
                "data": {
                  "from_parameter": "data"
                },
                "index": 1
              }
            },
            "arrayelement2": {
              "process_id": "array_element",
              "arguments": {
                "data": {
                  "from_parameter": "data"
                },
                "index": 0
              }
            },
            "subtract1": {
              "process_id": "subtract",
              "arguments": {
                "x": {
                  "from_node": "arrayelement1"
                },
                "y": {
                  "from_node": "arrayelement2"
                }
              }
            },
            "add1": {
              "process_id": "add",
              "arguments": {
                "x": {
                  "from_node": "arrayelement1"
                },
                "y": {
                  "from_node": "arrayelement2"
                }
              }
            },
            "divide1": {
              "process_id": "divide",
              "arguments": {
                "x": {
                  "from_node": "subtract1"
                },
                "y": {
                  "from_node": "add1"
                }
              },
              "result": true
            }
          }
        }
      }
    },
    "aggregatespatial1": {
      "process_id": "aggregate_spatial",
      "arguments": {
        "data": {
          "from_node": "reducedimension1"
        },
        "geometries": {
          "type": "GeometryCollection",
          "geometries": [
            {
              "type": "Polygon",
              "coordinates": [
                [
                  [
                    5.055945487931457,
                    51.222709834076504
                  ],
                  [
                    5.064972484168688,
                    51.221122565090525
                  ],
                  [
                    5.064972484168688,
                    51.221122565090525
                  ],
                  [
                    5.067474954083448,
                    51.218249806779134
                  ],
                  [
                    5.064827929485983,
                    51.21689628072789
                  ],
                  [
                    5.05917785594747,
                    51.217191909908095
                  ],
                  [
                    5.053553857094518,
                    51.21807492332223
                  ],
                  [
                    5.055945487931457,
                    51.222709834076504
                  ]
                ]
              ]
            },
            {
              "type": "Polygon",
              "coordinates": [
                [
                  [
                    5.063345886679116,
                    51.23087606640057
                  ],
                  [
                    5.06604742694687,
                    51.22886710731809
                  ],
                  [
                    5.070627820472246,
                    51.22874440121892
                  ],
                  [
                    5.068403609708207,
                    51.22657208381529
                  ],
                  [
                    5.064823257492447,
                    51.22676051738515
                  ],
                  [
                    5.064892324615199,
                    51.2283032878514
                  ],
                  [
                    5.063641745941974,
                    51.2285757299238
                  ],
                  [
                    5.062340811262595,
                    51.227722351687945
                  ],
                  [
                    5.06076005158084,
                    51.228042312276536
                  ],
                  [
                    5.063345886679116,
                    51.23087606640057
                  ]
                ]
              ]
            },
            {
              "type": "Polygon",
              "coordinates": [
                [
                  [
                    5.07163184674986,
                    51.23481147556147
                  ],
                  [
                    5.076706025697324,
                    51.23317590781036
                  ],
                  [
                    5.077828303041866,
                    51.233226237184724
                  ],
                  [
                    5.078024733866917,
                    51.23263978271262
                  ],
                  [
                    5.080771081607657,
                    51.23259097170763
                  ],
                  [
                    5.083734842574312,
                    51.23530464074437
                  ],
                  [
                    5.080957826735458,
                    51.23646091560258
                  ],
                  [
                    5.079752631651647,
                    51.23519531038643
                  ],
                  [
                    5.077238400183506,
                    51.23490534677628
                  ],
                  [
                    5.072856439300575,
                    51.23593546777778
                  ],
                  [
                    5.07163184674986,
                    51.23481147556147
                  ]
                ]
              ]
            },
            {
              "type": "Polygon",
              "coordinates": [
                [
                  [
                    5.083897244679042,
                    51.23510639883143
                  ],
                  [
                    5.081302408741335,
                    51.232922477780846
                  ],
                  [
                    5.082963802194108,
                    51.233146058575876
                  ],
                  [
                    5.084497702305552,
                    51.232672717580655
                  ],
                  [
                    5.085732850338428,
                    51.2340852086282
                  ],
                  [
                    5.083897244679042,
                    51.23510639883143
                  ]
                ]
              ]
            }
          ]
        },
        "reducer": {
          "process_graph": {
            "mean1": {
              "process_id": "mean",
              "arguments": {
                "data": {
                  "from_parameter": "data"
                }
              },
              "result": true
            }
          }
        }
      },
      "result": true
    }
  }
}

[soxofaan]

Turns out this is an issue in python client: the output of .aggregate_spatial() is a VectorCube, which does not support automatic format detection like DataCube.

If you add an explicit save_result with "CSV", I get a CSV output file :

aggregates = cube.aggregate_spatial(geometries=geometries, reducer="mean")
result = aggregates.save_result(format="CSV")
result.download("tmp.csv")

[soxofaan]

already fixed this for VectorCube.download.

Still to do for VectorCube.execute_batch

[soxofaan]

addressing #402 will probably simplify and help with the unification of this file format (autodetection) feature

[soxofaan]

closing this ticket: format guessing is supported in VectorCube now (since v0.21.0, released today)