search

fhir-fuel / fhir-fuel.github.io

Place to prepare proposal to FHIR about JSON, JSON-Schema, Swagger/OpenAPI, JSON native databases and other JSON-frendly formats (yaml, edn, avro, protobuf etc) and technologies
Other
21 stars 0 forks source link

accessors by key properties Identifier.system, Telecom.system, Telecom.use #8

Open yogthos opened 7 years ago

yogthos commented 7 years ago

Problem

Currently, telecom is structured using unordered collections:

{"telecom":
  [{"system": "phone",
     "value": "(416) 946-2223",
     "use":"work"}]}

This makes it difficult to address elements within the structure. If we needed to select the work phone, we would need to do one of the following:

  1. transform the structure into one that can be navigated by path
  2. search through the list of elements each time we need to address a particular element

Solution

The solution is to use nested maps to represent the structure:

{"telecom":
  {"work":
   {"phone": [{"value": "(416) 946-2223"}]}
  }
}

It's now possible to select specific types of elements without having to transform it.

niquola commented 7 years ago

Almost, everybody who developed UI for FHIR server wants this! The concern, I see, is that such transformation is very application specific. Approach could be some declarative transformation, which could be configured for the specific application:


PUT /Transformatin/easy-telecom
{
  id: 'easy-telecom',
  resourceType: 'Transformation',
  path: ['telecom'] , 
  transformation: {indexBy: ['use']}
}

GET /Patient?_transformation=easy-telecom
lmckenzi commented 7 years ago

A few considerations:

yogthos commented 7 years ago

I think it would make sense to have a catch all key for use such as unspecified with this approach. Then, any contact information that does not have a specific use could be grouped there.

I don't know if makes sense to try and keep JSON and XML representation in sync. They both have different limitations and very different idiomatic usage. As long as both map to the same underlying semantic model, I think it makes sense to leverage these formats to their best potential as opposed to trying to find the lowest common denominator.

lmckenzi commented 7 years ago

Customizing specific elements in any syntax has a significant cost. All of the code generators, reference implementations, tools that do conversion, etc. would need special logic. We'd need to weigh that cost against any perceived benefit for the user community.

Excluding all of the elements of type "any", these are the elements that would be impacted: CareTeam.telecom, Device.contact, Endpoint.contact, HealthcareService.telecom, Location.telecom, MessageHeader.source.contact, Organization.telecom, Organization.contact.telecom, Parameters.parameter.value[x], Patient.telecom, Patient.contact.telecom, Person.telecom, Practitioner.telecom, PractitoinerRole.telecom, RelatedPerson.telecom, StructureMap.group.rule.source.defaultValue[x], Subscription.contact, Task.input.value[x], Task.output.value[x].

I'd say the change would make sense for less than half of these.

Lloyd McKenzie, P.Eng. Senior Consultant, Information Technology Services Gevity Consulting Inc.

E: lmckenzie@gevityinc.com M: +1 587-334-1110 <1-587-334-1110> W: gevityinc.com

*GEVITY*Enabling a healthier world

CONFIDENTIALITY – This communication is confidential and for the exclusive use of its intended recipients. If you have received this communication through error, please notify the sender and delete the message without copying or disclosing.

REPRESENTATION – Unless explicitly stated otherwise, the opinions and positions expressed in this e-mail do not necessarily reflect those of my employer, my clients nor the organizations with whom I hold governance positions

On Tue, Jun 20, 2017 at 12:21 PM, Dmitri Sotnikov notifications@github.com wrote:

I think it would make sense to have a catch all key for use such as unspecified with this approach. Then, any contact information that does not have a specific use could be grouped there.

I don't know if makes sense to try and keep JSON and XML representation in sync. They both have different limitations and very different idiomatic usage. As long as both map to the same underlying semantic model, I think it makes sense to leverage these formats to their best potential as opposed to trying to find the lowest common denominator.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/fhir-fuel/fhir-fuel.github.io/issues/8#issuecomment-309845082, or mute the thread https://github.com/notifications/unsubscribe-auth/AErxgXQ2gAAcb-bCUPTFPYxhiTEEdgRNks5sGA26gaJpZM4N_z95 .

grahamegrieve commented 7 years ago

this is a use case specific view of the data that would make things better for some and not for others. I think it's a net negative idea in this case.

I think that graphQL is a good contribution here. You can have it looking like this if you want.

On Wed, Jun 21, 2017 at 4:32 AM, Lloyd McKenzie notifications@github.com wrote:

Customizing specific elements in any syntax has a significant cost. All of the code generators, reference implementations, tools that do conversion, etc. would need special logic. We'd need to weigh that cost against any perceived benefit for the user community.

Excluding all of the elements of type "any", these are the elements that would be impacted: CareTeam.telecom, Device.contact, Endpoint.contact, HealthcareService.telecom, Location.telecom, MessageHeader.source.contact, Organization.telecom, Organization.contact.telecom, Parameters.parameter.value[x], Patient.telecom, Patient.contact.telecom, Person.telecom, Practitioner.telecom, PractitoinerRole.telecom, RelatedPerson.telecom, StructureMap.group.rule.source.defaultValue[x], Subscription.contact, Task.input.value[x], Task.output.value[x].

I'd say the change would make sense for less than half of these.

Lloyd McKenzie, P.Eng. Senior Consultant, Information Technology Services Gevity Consulting Inc.

E: lmckenzie@gevityinc.com M: +1 587-334-1110 <(587)%20334-1110> <1-587-334-1110 <(587)%20334-1110>> W: gevityinc.com

*GEVITY*Enabling a healthier world

CONFIDENTIALITY – This communication is confidential and for the exclusive use of its intended recipients. If you have received this communication through error, please notify the sender and delete the message without copying or disclosing.

REPRESENTATION – Unless explicitly stated otherwise, the opinions and positions expressed in this e-mail do not necessarily reflect those of my employer, my clients nor the organizations with whom I hold governance positions

On Tue, Jun 20, 2017 at 12:21 PM, Dmitri Sotnikov < notifications@github.com> wrote:

I think it would make sense to have a catch all key for use such as unspecified with this approach. Then, any contact information that does not have a specific use could be grouped there.

I don't know if makes sense to try and keep JSON and XML representation in sync. They both have different limitations and very different idiomatic usage. As long as both map to the same underlying semantic model, I think it makes sense to leverage these formats to their best potential as opposed to trying to find the lowest common denominator.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/fhir-fuel/fhir-fuel.github.io/issues/8# issuecomment-309845082, or mute the thread https://github.com/notifications/unsubscribe-auth/AErxgXQ2gAAcb- bCUPTFPYxhiTEEdgRNks5sGA26gaJpZM4N_z95

.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/fhir-fuel/fhir-fuel.github.io/issues/8#issuecomment-309848005, or mute the thread https://github.com/notifications/unsubscribe-auth/AFllFc_gR2rNZCd_2f5FG_MQLBiA4f4Rks5sGBAhgaJpZM4N_z95 .

--

http://www.healthintersections.com.au / grahame@healthintersections.com.au / +61 411 867 065

yogthos commented 7 years ago

@lmckenzi

On the other hand, designing the format in a way that makes it difficult to access the data pushes the cost of using it down to all the users of this format. I also agree that these kinds of decisions should be community driven. Have any surveys been conducted to see what challenges the user community is facing, and what are common pain points are?

As far as I'm aware there's no transparent way to track proposed changes to the spec, or what percentage of users would be in favor of any particular change. Zulip is not very discoverable, and it doesn't provide any tools to effectively track and rank issues.

@grahamegrieve

I see a general pattern in FHIR that it uses collections of maps to represent data elements. This is idiomatic when dealing with XML nodes, however it's not the way JSON data is typically structured. Any view of the data will have a trade-off. However, I fail to see how representing data idiomatically, and in a way that it can be easily addressed, is a net negative.

Pretty much in any use case where you actually have to use the data in the payload, you will need to be able to address it efficiently. I think that it's reasonable to expect that the format should be designed with this use case in mind.

Currently, the first thing you have to do with the data received from FHIR service is to transform it into a more usable form before you can work with it. This means that every user of FHIR services has their own ad hoc parser for the format, and this is extremely error prone. I think this has a direct impact on data quality and ultimately patient safety.

grahamegrieve commented 7 years ago

you are assuming that the only way to access the telecom numbers is by use, then other things. Most systems I've ever dealt with do it by type, then use. That's the reason I don't like this proposal: it artificially picks one way of viewing the data and makes it the way the data is structured. use graphQL for that.

On Wed, Jun 21, 2017 at 7:39 AM, Dmitri Sotnikov notifications@github.com wrote:

@lmckenzi https://github.com/lmckenzi

On the other hand, designing the format in a way that makes it difficult to access the data pushes the cost of using it down to all the users of this format. I also agree that these kinds of decisions should be community driven. Have any surveys been conducted to see what challenges the user community is facing, and what are common pain points are?

As far as I'm aware there's no transparent way to track proposed changes to the spec, or what percentage of users would be in favor of any particular change. Zulip is not very discoverable, and it doesn't provide any tools to effectively track and rank issues.

@grahamegrieve https://github.com/grahamegrieve

I see a general pattern in FHIR that it uses collections of maps to represent data elements. This is idiomatic when dealing with XML nodes, however it's not the way JSON data is typically structured. Any view of the data will have a trade-off. However, I fail to see how representing data idiomatically, and in a way that it can be easily addressed, is a net negative.

Pretty much in any use case where you actually have to use the data in the payload, you will need to be able to address it efficiently. I think that it's reasonable to expect that the format should be designed with this use case in mind.

Currently, the first thing you have to do with the data received from FHIR service is to transform it into a more usable form before you can work with it. This means that every user of FHIR services has their own ad hoc parser for the format, and this is extremely error prone. I think this has a direct impact on data quality and ultimately patient safety.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/fhir-fuel/fhir-fuel.github.io/issues/8#issuecomment-309899705, or mute the thread https://github.com/notifications/unsubscribe-auth/AFllFcST8_jrzk4c8ACBvT7nz04f756Fks5sGDwcgaJpZM4N_z95 .

--

http://www.healthintersections.com.au / grahame@healthintersections.com.au / +61 411 867 065

lmckenzi commented 7 years ago

gForge is our tool for tracking requests for change. It's transparent and searchable and allows commenting, but it doesn't support voting. We will hopefully soon be moving to a different tool that does support voting.

Adding in a layer to the object hierarchy makes things easier for those who want to be able to disambugate based on ContactPoint.use, but makes it harder for those who don't care about use - as they now have to search a bunch of paths where previously they only had to look at one.

Lloyd McKenzie, P.Eng. Senior Consultant, Information Technology Services Gevity Consulting Inc.

E: lmckenzie@gevityinc.com M: +1 587-334-1110 <1-587-334-1110> W: gevityinc.com

*GEVITY*Enabling a healthier world

CONFIDENTIALITY – This communication is confidential and for the exclusive use of its intended recipients. If you have received this communication through error, please notify the sender and delete the message without copying or disclosing.

REPRESENTATION – Unless explicitly stated otherwise, the opinions and positions expressed in this e-mail do not necessarily reflect those of my employer, my clients nor the organizations with whom I hold governance positions

On Tue, Jun 20, 2017 at 3:39 PM, Dmitri Sotnikov notifications@github.com wrote:

@lmckenzi https://github.com/lmckenzi

On the other hand, designing the format in a way that makes it difficult to access the data pushes the cost of using it down to all the users of this format. I also agree that these kinds of decisions should be community driven. Have any surveys been conducted to see what challenges the user community is facing, and what are common pain points are?

As far as I'm aware there's no transparent way to track proposed changes to the spec, or what percentage of users would be in favor of any particular change. Zulip is not very discoverable, and it doesn't provide any tools to effectively track and rank issues.

@grahamegrieve https://github.com/grahamegrieve

I see a general pattern in FHIR that it uses collections of maps to represent data elements. This is idiomatic when dealing with XML nodes, however it's not the way JSON data is typically structured. Any view of the data will have a trade-off. However, I fail to see how representing data idiomatically, and in a way that it can be easily addressed, is a net negative.

Pretty much in any use case where you actually have to use the data in the payload, you will need to be able to address it efficiently. I think that it's reasonable to expect that the format should be designed with this use case in mind.

Currently, the first thing you have to do with the data received from FHIR service is to transform it into a more usable form before you can work with it. This means that every user of FHIR services has their own ad hoc parser for the format, and this is extremely error prone. I think this has a direct impact on data quality and ultimately patient safety.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/fhir-fuel/fhir-fuel.github.io/issues/8#issuecomment-309899705, or mute the thread https://github.com/notifications/unsubscribe-auth/AErxgfvBRoPGWAkYnng-wGX-2GITTKS9ks5sGDwcgaJpZM4N_z95 .

yogthos commented 7 years ago

I guess the question here is which use case is the more common one. In either scenario a data transformation would be required for one group of users. This goes back to my question of how it's decided which use case is the more common one.

I do agree that graphQL can help here. If that was part of the FHIR server specification, that would provide the clients with the flexibility to select data in a view that's appropriate to them.

I have looked at gForge, and seems like it is more appropriate as tracker for existing issues as opposed to a forum for feature discussions. Looking forward to trying out the new tool for this once the move happens though.

lmckenzi commented 7 years ago

Well, transformation isn't technically required if you're interested in filtering by use (or system or priority or whatever else). You just iterate and find the repetitions you care about. That's how the data structures work throughout all of FHIR - because different use-cases will require filtering on different things. If you want to transform rather than iterate, that's fine. But we don't want to increase the complexity for those who will need to iterate.

Lloyd McKenzie, P.Eng. Senior Consultant, Information Technology Services Gevity Consulting Inc.

E: lmckenzie@gevityinc.com M: +1 587-334-1110 <1-587-334-1110> W: gevityinc.com

*GEVITY*Enabling a healthier world

CONFIDENTIALITY – This communication is confidential and for the exclusive use of its intended recipients. If you have received this communication through error, please notify the sender and delete the message without copying or disclosing.

REPRESENTATION – Unless explicitly stated otherwise, the opinions and positions expressed in this e-mail do not necessarily reflect those of my employer, my clients nor the organizations with whom I hold governance positions

On Tue, Jun 20, 2017 at 3:58 PM, Dmitri Sotnikov notifications@github.com wrote:

I guess the question here is which use case is the more common one. In either scenario a data transformation would be required for one group of users. This goes back to my question of how it's decided which use case is the more common one.

I do agree that graphQL can help here. If that was part of the FHIR server specification, that would provide the clients with the flexibility to select data in a view that's appropriate to them.

I have looked at gForge, and seems like it is more appropriate as tracker for existing issues as opposed to a forum for feature discussions. Looking forward to trying out the new tool for this once the move happens though.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/fhir-fuel/fhir-fuel.github.io/issues/8#issuecomment-309903724, or mute the thread https://github.com/notifications/unsubscribe-auth/AErxgZ_BmgiE9cxnTMdRXiQXVjbHMj7rks5sGEBogaJpZM4N_z95 .

grahamegrieve commented 7 years ago

I guess the question here is which use case is the more common one. In either scenario a data transformation would be required for one group of users. This goes back to my question of how it's decided which use case is the more common one.

there's other use cases again. The current structure is a compromise between users for interoperability, not an optimal structure for a single EHR that can control it's own choices.

I do agree that graphQL can help here. If that was part of the FHIR server specification, that would provide the clients with the flexibility to select data in a view that's appropriate to them.

Added the spec yesterday: http://build.fhir.org/graphql.html.

I have looked at gForge, and seems like it is more appropriate as tracker for existing issues as opposed to a forum for feature discussions. Looking forward to trying out the new tool for this once the move happens though.

we're all looking forward to that

Grahame

yogthos commented 7 years ago

The reason I proposed the change is due to the fact how UIs will represent the data. The most common scenario is that you would group your contact information by work/home/etc in the UI. Ultimately, the data has to be shown to the users at some point, so UI representation needs be consideration when designing the data model in my opinion.

As I mentioned earlier, you're necessarily increasing complexity for a particular group of users with either approach. It sounds like you're advocating for the lowest common denominator that's least opinionated. However, if one use case is much more common than the other, it would make sense to favor it. It's difficult to make a call on this without having the numbers available.

Note that I'm not advocating optimizing for a specific EHR, but rather for making the general use case easier.

@grahamegrieve thanks, I'll take a look at the GraphQL docs.

lmckenzi commented 7 years ago

What we're challenging is the assertion that that's the general use-case. It's a use-case for organizing contact information for "person" ContactPoints, but it's certainly not the only one, nor necessarily the most common. And it's not a use-case at all for non-person ContactPoints - which makes up over half the uses of the data type.

Lloyd McKenzie, P.Eng. Senior Consultant, Information Technology Services Gevity Consulting Inc.

E: lmckenzie@gevityinc.com M: +1 587-334-1110 <1-587-334-1110> W: gevityinc.com

*GEVITY*Enabling a healthier world

CONFIDENTIALITY – This communication is confidential and for the exclusive use of its intended recipients. If you have received this communication through error, please notify the sender and delete the message without copying or disclosing.

REPRESENTATION – Unless explicitly stated otherwise, the opinions and positions expressed in this e-mail do not necessarily reflect those of my employer, my clients nor the organizations with whom I hold governance positions

On Wed, Jun 21, 2017 at 10:45 AM, Dmitri Sotnikov notifications@github.com wrote:

The reason I proposed the change is due to the fact how UIs will represent the data. The most common scenario is that you would group your contact information by work/home/etc in the UI. Ultimately, the data has to be shown to the users at some point, so UI representation needs be consideration when designing the data model in my opinion.

As I mentioned earlier, you're necessarily increasing complexity for a particular group of users with either approach. It sounds like you're advocating for the lowest common denominator that's least opinionated. However, if one use case is much more common than the other, it would make sense to favor it. It's difficult to make a call on this without having the numbers available.

Note that I'm not advocating optimizing for a specific EHR, but rather for making the general use case easier.

@grahamegrieve https://github.com/grahamegrieve thanks, I'll take a look at the GraphQL docs.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/fhir-fuel/fhir-fuel.github.io/issues/8#issuecomment-310137956, or mute the thread https://github.com/notifications/unsubscribe-auth/AErxgQWI-puj30lZapnVEGNHQnWYXNbpks5sGUifgaJpZM4N_z95 .

niquola commented 7 years ago

@lmckenzi - we started this repo aka incubator for proposals, some discussed and matured ideas definitely will be posted to gForce.

The idea of simplified accessors is a sugar and could be implemented as a library with an informal specification, without moving to FHIR standard directly.

grahamegrieve commented 7 years ago

we can define accessors, sure - umm, where we do we define them? the reference implementations already define a few, but they define different ones each. In languages that support helper classes, I have some extensive helper classes... but you're not using the reference implementations...?

There are a very extensions that we use to hint things to code generators, for various decorations they might want to add. Is that something that you are thinking of?

Grahame

On Thu, Jun 22, 2017 at 7:49 AM, niquola notifications@github.com wrote:

@lmckenzi https://github.com/lmckenzi - we started this repo aka incubator for proposals, some discussed and matured ideas definitely will be posted to gForce.

The idea of simplified accessors is a sugar and could be implemented as a library with an informal specification, without moving to FHIR standard directly.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/fhir-fuel/fhir-fuel.github.io/issues/8#issuecomment-310215220, or mute the thread https://github.com/notifications/unsubscribe-auth/AFllFX20KKeVrGJ1yfPR-LHFqICPQ4PDks5sGZADgaJpZM4N_z95 .

--

http://www.healthintersections.com.au / grahame@healthintersections.com.au / +61 411 867 065

yogthos commented 7 years ago

@lmckenzi I'm not making any assertions regarding what the general use case is. I'm basing the proposal on the fact that it's an issue for me as well as other people I interact with. However, this is clearly anecdotal and only represents my personal understanding of the situation. I have to assume that you're making decisions empirically based on the feedback you collect from the users as opposed to simply using personal judgement.

grahamegrieve commented 7 years ago

we try to. There's always an element of judgement involved. Hopefully that's judgement, not bias.

There was a time when I had a draft of a 'map' data type hanging around my system somewhere, but I eventually deleted it because I never saw any place where it made sense - though I defined 'sense' as n..1 values per key, which was a little narrower. I had a quick scan for places where it might make sense now, and came up with nothing. The only place is Extension... and I think that's too late to fix. GraphQL is the solution, really, now. Why I've been pushing it the last few months.

Grahame

On Thu, Jun 22, 2017 at 8:51 AM, Dmitri Sotnikov notifications@github.com wrote:

@lmckenzi https://github.com/lmckenzi I'm not making any assertions regarding what the general use case is. I'm basing the proposal on the fact that it's an issue for me as well as other people I interact with. However, this is clearly anecdotal and only represents my personal understanding of the situation. I have to assume that you're making decisions empirically based on the feedback you collect from the users as opposed to simply using personal judgement.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/fhir-fuel/fhir-fuel.github.io/issues/8#issuecomment-310227158, or mute the thread https://github.com/notifications/unsubscribe-auth/AFllFRX6z06M3cKtOU6_2SZPDryK7tWyks5sGZ5sgaJpZM4N_z95 .

--

http://www.healthintersections.com.au / grahame@healthintersections.com.au / +61 411 867 065

yogthos commented 7 years ago

I agree, GraphQL adds a lot of flexibility for the clients allowing them to select precisely the parts of the resources that they need.

brianpos commented 7 years ago

With that suggested representation you'd need to make that representation an array also, as I know lots of people that have multiple mobile numbers for work (and the rank was introduced in STU3 to permit supporting the preference of use).

niquola commented 6 years ago

Here is the best description of problem and solution I’ve ever seen - https://json-ld.org/spec/latest/json-ld/#data-indexing