ODD: Sufficient ReflowOS integration for initial use cases

[adam-burns]

Assess and if necessary implement, debug, test & verify ReflowOS GraphQL endpoints for ODD Valueflows visualisation to support initial use cases.

[adam-burns]

ODD developers thought of some data visualization ideas regarding the ODD. One of those ideas is a map, where the resource-specific supply and demand of the different locations are being displayed. Another one is to visualize processes as graphs.

Do the following make sense:

1. Query all resources of all locations in one list (with ID, name, quantity, unit, location ID)
2. Query all locations (with ID, name, lat, long)
3. Query needs or offers (with ID, name, quantity, unit, due date, location ID): 3.1 Proposals can be requested but is there is a way to track back from proposals to needs or offers?!
4. Query all steps of a process (with or without the economic events)
5. Query all economic events for a location for a specified period of time (with ID, name, resourceClassification, providerID, receiverID)

[densizengin]

ODD developers thought of some data visualization ideas regarding the ODD. One of those ideas is a map, where the resource-specific supply and demand of the different locations are being displayed. Another one is to visualize processes as graphs.

Do the following make sense:

1. Query all resources of all locations in one list (with ID, name, quantity, unit, location ID)

I think you can do this by querying Economic Resources and SpatialThings separately. What I couldn't understand is that do you mean querying by ID, name, quantity, unit, etc.? Because that's a different thing.

2. Query all locations (with ID, name, lat, long)

You can query SpatialThings. But are you referring to filtering here too?

3. Query needs or offers (with ID, name, quantity, unit, due date, location ID):

Can be done by querying Intents. But are you referring to filtering here too?

   3.1  Proposals can be requested but is there is a way to track back from proposals to needs or offers?!

Yeah, this needs to be implemented.

4. Query all steps of a process (with or without the economic events)

You can use outputs and inputs of a Process to query related EconomicEvents.

5. Query all economic events for a location for a specified period of time (with ID, name, resourceClassification, providerID, receiverID)

Is this also a filtering question?

[jonashet]

From our perspective it is desirable for each visualization process to get as much information out of a single query as possible, because each request inherits an own harvesting-process. For the "demand-supply"use-case we need a table, which contains every resource with its respective ID, current location, quantity and unit. In order to achieve this we currently have to send a request to the same "economicResourcesFiltered(locationID)"-Endpoint for every locationID. This constitues a process, which is divided into as many harvesting-steps as there are locationIDs. Instead of filling our DB piece by piece we would prefer having access to a query, which outputs a list of all locations and all of their respective resources (name, unit, id, quantity). At the same time a query, which just outputs all locations (ID, name and coordinates) would come in quite handy as well. I will answer your questions regarding the process-visualization after having consultation with Milko.

[densizengin]

From our perspective it is desirable for each visualization process to get as much information out of a single query as possible, because each request inherits an own harvesting-process. For the "demand-supply"use-case we need a table, which contains every resource with its respective ID, current location, quantity and unit. In order to achieve this we currently have to send a request to the same "economicResourcesFiltered(locationID)"-Endpoint for every locationID. This constitues a process, which is divided into as many harvesting-steps as there are locationIDs. Instead of filling our DB piece by piece we would prefer having access to a query, which outputs a list of all locations and all of their respective resources (name, unit, id, quantity). At the same time a query, which just outputs all locations (ID, name and coordinates) would come in quite handy as well. I will answer your questions regarding the process-visualization after having consultation with Milko.

Hi, Jonas.

GraphQL works somewhat differently than the well-known RESTful JSON APIs over HTTP: You can do multiple things in one go.

You can query for multiple economicResourcesFiltered in one request by giving each a unique alias (can be generated inside a for loop easily), such as:

query {
    _01FRZJ9FN5858X760WC1XJQCV5: economicResourcesFiltered(currentLocation: "01FRZJ9FN5858X760WC1XJQCV5") { ... }
    _01FRZJ9NJB2GN9H6J358ZZNFSM: economicResourcesFiltered(currentLocation: "01FRZJ9NJB2GN9H6J358ZZNFSM") { ... }
    _01FS0M5HK2YXG63ZFZBHFXPN6Y: economicResourcesFiltered(currentLocation: "01FS0M5HK2YXG63ZFZBHFXPN6Y") { ... }
}

and parse the result easily. I used the IDs of the locations (called SpatialThing in the VF vocab), prefixed with an underscore due to the requirements of aliases in GraphQL (they work just like variable names in common programming languages).

You can also query locations with spatialThings and spatialThingsPages (SpatialThing is what you might call location in the VF vocab). The former returns everything in hand, the latter returns the results with pagination.

I hope it helps and please comment on my other questions. ^^

Cheers, srfsh

[MiMonecke]

Hi all, ok some clearifications by reference to the first query: 1. Query all resources of all locations in one list (with ID, name, quantity, unit, location ID) We would like to request all resources of a given location. The response should provide the following properties for each resource: ID, name, quantity, unit, location ID - so brackets here does not indicate parameters. At the moment resources can be requested as it follows: economicResourcesFiltered(agent: [ID]currentLocation: [ID]geolocation: GeolocationFiltersinScopeOf: [ID]state: [ID]tagIds: [ID]): [EconomicResource] For our created use case / idea this query is not sufficient. In this state of the project we are evaluating which visualization ideas provide a benefit for the pilots. I hope we can find a solution or may adapt the use case to obtain data for the creation of meaningful visualizations. Important for us is the availability of data.

I suggest to start with a description of the idea and would like to ask you to provide support in the sense of how desired data can be requested. We would like to receive all resources for all locations to show the potentials that exist in several places. For this case, the data should be stored in two db tables in the ODD: resources with columns: ID, name, quantity, unit, location ID ; locations with columns: ID, name, lat, long. How can we efficiently request data for this use case? I've thought about concerns mentioned by @adam-burns today and I agree that querying the whole resource inventory periodically is not really efficient. Therefore the question: is there a workaround available to indicate changes?

[densizengin]

Hi all, ok some clearifications by reference to the first query: 1. Query all resources of all locations in one list (with ID, name, quantity, unit, location ID) We would like to request all resources of a given location. The response should provide the following properties for each resource: ID, name, quantity, unit, location ID - so brackets here does not indicate parameters. At the moment resources can be requested as it follows: economicResourcesFiltered(agent: [ID]currentLocation: [ID]geolocation: GeolocationFiltersinScopeOf: [ID]state: [ID]tagIds: [ID]): [EconomicResource] For our use case / idea this query is not sufficient. In this state of the project we are evaluating which visualization ideas provide a benefit for the pilots. I hope we can find a solution or may adapt the use case to obtain data for the creation of meaningful visualizations. Important for us is the availability of data.

Do a query like this satisfy your needs:

query {
  _01FS0M606JSHX3W6VW9Y6N0B1Y: economicResourcesFiltered(currentLocation: "01FS0M606JSHX3W6VW9Y6N0B1Y") {
    id # id
    name # name
    onhandQuantity { # quantity
      hasUnit {id label} # unit's label
      hasNumericalValue # unit's value
    }
    accountingQuantity { # quantity
      hasUnit {id label} # unit's label
      hasNumericalValue # unit's value
    }
    currentLocation { # location
      id
      name
      lat alt long
    } 
  }
  _01FS0M5HK2YXG63ZFZBHFXPN6Y: economicResourcesFiltered(currentLocation: "01FS0M5HK2YXG63ZFZBHFXPN6Y") {
    id # id
    name # name
    onhandQuantity { # quantity
      hasUnit {id label} # unit's label
      hasNumericalValue # unit's value
    }
    accountingQuantity { # quantity
      hasUnit {id label} # unit's label
      hasNumericalValue # unit's value
    }
    currentLocation { # location
      id
      name
      lat alt long
    } 
  }
}

You can also simplify it hugely by using fragments:

fragment economicResource on EconomicResource {
  id # id
  name # name
  onhandQuantity { # quantity
    hasUnit {id label} # unit's label
    hasNumericalValue # unit's value
  }
  accountingQuantity { # quantity
    hasUnit {id label} # unit's label
    hasNumericalValue # unit's value
  }
  currentLocation { # location
    id
    name
    lat alt long
  } 
}

query {
  _01FS0M606JSHX3W6VW9Y6N0B1Y: economicResourcesFiltered(currentLocation: "01FS0M606JSHX3W6VW9Y6N0B1Y") {...economicResource}
  _01FS0M5HK2YXG63ZFZBHFXPN6Y: economicResourcesFiltered(currentLocation: "01FS0M5HK2YXG63ZFZBHFXPN6Y") {...economicResource}
  _01FS0MMZPFYKW12XQ0RGENDHMZ: economicResourcesFiltered(currentLocation: "01FS0MMZPFYKW12XQ0RGENDHMZ") {...economicResource}
  _01FS0MT5VH6S4W49VCRMK9KW3Q: economicResourcesFiltered(currentLocation: "01FS0MT5VH6S4W49VCRMK9KW3Q") {...economicResource}
  _01FS0M81ZDPHNAAPN1KGQRKJA2: economicResourcesFiltered(currentLocation: "01FS0M81ZDPHNAAPN1KGQRKJA2") {...economicResource}
}

[densizengin]

Hi all, ok some clearifications by reference to the first query: 1. Query all resources of all locations in one list (with ID, name, quantity, unit, location ID) We would like to request all resources of a given location. The response should provide the following properties for each resource: ID, name, quantity, unit, location ID - so brackets here does not indicate parameters. At the moment resources can be requested as it follows: economicResourcesFiltered(agent: [ID]currentLocation: [ID]geolocation: GeolocationFiltersinScopeOf: [ID]state: [ID]tagIds: [ID]): [EconomicResource] For our use case / idea this query is not sufficient. In this state of the project we are evaluating which visualization ideas provide a benefit for the pilots. I hope we can find a solution or may adapt the use case to obtain data for the creation of meaningful visualizations. Important for us is the availability of data.

Do a query like this satisfy your needs:

query {
  _01FS0M606JSHX3W6VW9Y6N0B1Y: economicResourcesFiltered(currentLocation: "01FS0M606JSHX3W6VW9Y6N0B1Y") {
    id # id
    name # name
    onhandQuantity { # quantity
      hasUnit {id label} # unit's label
      hasNumericalValue # unit's value
    }
    accountingQuantity { # quantity
      hasUnit {id label} # unit's label
      hasNumericalValue # unit's value
    }
    currentLocation { # location
      id
      name
      lat alt long
    } 
  }
  _01FS0M5HK2YXG63ZFZBHFXPN6Y: economicResourcesFiltered(currentLocation: "01FS0M5HK2YXG63ZFZBHFXPN6Y") {
    id # id
    name # name
    onhandQuantity { # quantity
      hasUnit {id label} # unit's label
      hasNumericalValue # unit's value
    }
    accountingQuantity { # quantity
      hasUnit {id label} # unit's label
      hasNumericalValue # unit's value
    }
    currentLocation { # location
      id
      name
      lat alt long
    } 
  }
}

You can also simplify it hugely by using fragments:

fragment economicResource on EconomicResource {
  id # id
  name # name
  onhandQuantity { # quantity
    hasUnit {id label} # unit's label
    hasNumericalValue # unit's value
  }
  accountingQuantity { # quantity
    hasUnit {id label} # unit's label
    hasNumericalValue # unit's value
  }
  currentLocation { # location
    id
    name
    lat alt long
  } 
}

query {
  _01FS0M606JSHX3W6VW9Y6N0B1Y: economicResourcesFiltered(currentLocation: "01FS0M606JSHX3W6VW9Y6N0B1Y") {...economicResource}
  _01FS0M5HK2YXG63ZFZBHFXPN6Y: economicResourcesFiltered(currentLocation: "01FS0M5HK2YXG63ZFZBHFXPN6Y") {...economicResource}
  _01FS0MMZPFYKW12XQ0RGENDHMZ: economicResourcesFiltered(currentLocation: "01FS0MMZPFYKW12XQ0RGENDHMZ") {...economicResource}
  _01FS0MT5VH6S4W49VCRMK9KW3Q: economicResourcesFiltered(currentLocation: "01FS0MT5VH6S4W49VCRMK9KW3Q") {...economicResource}
  _01FS0M81ZDPHNAAPN1KGQRKJA2: economicResourcesFiltered(currentLocation: "01FS0M81ZDPHNAAPN1KGQRKJA2") {...economicResource}
}

@MiMonecke I've got a thumbs up on this. Does that mean I can close the issue now?

[MiMonecke]

Hi @srfsh, may I was a bit slow in my thinking, but I edited/added a question while you were answering the first part of my message. Can you please have a look on that?

Furthermore when we request all resources like suggested, it would be nice to request all available locations with their IDs, name, lat and long. I mean requesting a list of all locations without stating their IDs as parameters. Is that possible? Thanks in advance.

[adam-burns]

I've thought about concerns mentioned by @adam-burns today and I agree that querying the whole resource inventory periodically is not really efficient. Therefore the question: is there a workaround available to indicate changes?

Just to clarify my comment, I mentioned that harvesting as much data over GraphQL as possible for full ingestion raises a tension against its design priniciple of data minimisation.

With respect to querying updates over time, GraphQL does offer subscriptions for updates over time are server side heavy, need to be server side implemented for each use case and not currently implemented for locations.

Given that, I would suggest we review what may be needed or nice for updates over time over the queries you need once we have bedded down the queries that are suitable for your requirements for initial imports.

Once we are there, I would suggest raising a separate issue to track that.

[fosterlynn]

I hope I'm not responding to the wrong part of this issue, I'm not totally clear on the question of how to get all the data you need in one call, vs how to get updates to that data pushed from the server. If I'm not properly connected to this discussion, please just ignore.

Just another option. This particular reverse query probably doesn't exist, but it could. And I notice I haven't defined it on my filtering / query app guidance (in process), I'll add it. It drives from location instead of resources. I don't know how many locations will not have resources.

Not tested, may not even compile, but this is the idea:

 query {
  spatialThings { # locations
      id
      name
      lat alt long
      resources {   # probably doesn't exist yet
        id
        name
        onhandQuantity { # quantity
          hasUnit {id label} # unit's label
          hasNumericalValue # unit's value
        }
        accountingQuantity { # quantity
          hasUnit {id label} # unit's label
          hasNumericalValue # unit's value
        }        
      }
    }

Another thought to consider. Do you want to show all resources at a specific location, irrespective of it they are already planned to be used by the primary accountable itself? Or do you want to show resources being offered by a primary responsible located at a specific location? If the latter, then you will want to use offers (Proposal, Intent, etc.) rather than just resources at a location.