RaaLabs / TSIClient

A python package to extract data from Azure Time Series Insights (TSI).
https://pypi.org/project/TSIClient/
MIT License
6 stars 25 forks source link

Create ability to retrieve data only from selected hierarchies #30

Open rafaelschlatter opened 4 years ago

rafaelschlatter commented 4 years ago

It would be nice to have the functionality to only retrieve timeseries that are part of certain hierarchies. This is useful because hierarchies can be used to differentiate between timeseries that are "issue-free" and ready for data analysis and timeseries that are not yet ready for analysis.

One way to implement this is to pass a list of hierarchies as an optional argument to the getInstances() function, and have the filtering logic in that function.

This logic might already exist in Microsofts TSI apis, maybe it is possible to use a POST request with this url https://{environmentFqdn}/timeseries/instances?api-version=2018-11-01-preview and only retrieve the instances that have a certain hierarchy. I havent completely looked through that, it is described here: https://docs.microsoft.com/en-us/rest/api/time-series-insights/dataaccess(preview)/timeseriesinstances/search.

Another option would be to use the existing GET request in getInstances(), implement the filtering logic ourselves, and initially retrieve all instances, and then filter the results before returning on line 310 in TSIClient.py.

rafaelschlatter commented 4 years ago

If we do the logic ourselves, we might want to do this generally, such that we can filter on any instance attribute and not just the hierarchy id. Meaning allow filtering on any of these instance attributes:

{
  "instances": [
    {
      "typeId": "9b84e946-7b36-4aa0-9d26-71bf48cb2aff",
      "name": "F1W7.GS1",
      "timeSeriesId": [
        "006dfc2d-0324-4937-998c-d16f3b4f1952",
        "T1"
      ],
      "description": "ContosoFarm1W7_GenSpeed1",
      "hierarchyIds": [
        "33d72529-dd73-4c31-93d8-ae4e6cb5605d"
      ],
      "instanceFields": {
        "Name": "GeneratorSpeed",
        "Plant": "Contoso Plant 1",
        "Unit": "W7",
        "System": "Generator System"
      }
    }
  ],
  "continuationToken": "aXsic2tpcCI6MTAwMCwidGFrZSI6MTAwMH0="
}