aug0211 commented 6 months ago

[ ] Establish a set of principles for this design
[ ] Create a standard with examples for each potential entry type, always abiding by the principles defined
[ ] Refactor the forming of the message string per the principles

aug0211 commented 6 months ago

Principle 1

Machine readable

To be useful, the format must be parseable in a consistent fashion, so that iOS Shortcut Automations can reliably process the message.

Examples of "good":

Consistent delimiters
Consistent variable usage

Examples of "bad":

Special characters that require atypical/challenging decoding to handle in automations

Principle 2

Human Readable

Machine readable does not mean that the text cannot also be human readable. The format can be consistent, parseable by a machine (Shortcut Automation) and also human friendly by using form and terms familiar to human language.

Examples of "good":

Users must be able to read text messages and immediately, with 0 special understanding, memorization, or interpretation, understand exactly what a message intended to action
Full human words opposed to abbreviations
Familiar text formatting such as commas (,), colons (:), and new lines (carriage returns)
Capitalization of words following typical written grammar rules
Widely recognized abbreviations in the T1D world, including and limited to "carbs" for carbohydrates, "g" for grams, "u" for units, "temp" for "temporary"
Explicitly declare all field with appropriate values, even when 0 (eg, meals with no fat, no protein, etc)

Examples of "bad":

Inconsistent sequencing of events in messages
Obscure/uncommon punctuation such as ^, {, >, \, _, ', ] and so on
Uncommon abbreviations such as "bol" or "ovrrd"
Omitting any optional fields such as fat, carbs, protein

Principle 3

Simplify iOS Shortcut Automation development

iOS Shortcut Automations cannot be programmatically shared to other users. Each user who wishes to use this functionality will be require to hand-craft their own automations to process and decode messages sent from LoopFollow Remote. Avoid introducing unnecessary complexity in this step for users.

Examples of "good":

Simplified/logical naming of fields in messages for more intuitive parsing
Consider whether call outs (eg, to JS or similar) are an option we'd like to use to enable copy/paste of more code
Native/out of the box functionality to handle looping structures if available

Examples of "bad":

Any custom logic/highly unique parsing
Obscure looping structures
Unnecessarily repetitive actions, eg deeply (N=3+) nested structures

Principle 4

Traceability

It can be important for safety, debugging, and overall understanding of T1D caregiver management to understand exactly who did what, when.

We will include tags/user IDs/information to specify who did what, when, how in our messages.

Examples of "good":

Declare that the message is coming from LoopFollow Remote
Declare the ID of the LoopFollow Remote installation (require a new field in LoopFollow Remote to capture sender ID)
Declare the purpose of each message, including Remote Meal, Remote Bolus, Remote Temp Target, Remote Override, Remote Custom Action
Declare a timestamp for when the message was formulated by LoopFollow Remote

Examples of "bad":

Omitting any of the above information

aug0211 commented 6 months ago

Also, we should probably do this:

https://apple.stackexchange.com/questions/442458/is-it-possible-to-share-a-shortcut-automation-with-others

TLDR: create our automations in a shortcut. The shortcut can then be shared.

Now, let the automation call the saved shortcut when the automation fires.

This makes life much easier for users and allows us to supply the majority of a working shell to users to customize - all they need to do is enter their personally saved temp targets and profile overrides.

dsnallfot commented 6 months ago

Great break down of what's important here, and I have no conflicting opinion on any of this.

The only thing I would like to add is that we most likely want the users to be able to select between the two already implemented methods:

the paid Twilio service (for the smoothest experience), or
the free of charge solution with deep links to open personal url shortcuts on the caregivers phone.

The formatting should ideally be the same regardless of method, but there could be some limitations in what information that can be included in a deep link url that opens the relevant shortcut in the shortcuts app on the caregiver phone (we of course need to test all suggested formatting with both methods).

And another slight difference between methods, is when sending sms commands (with shortcuts) over iMessage directly from the caregivers phone to the looping phone, the "who sent this"-part at least get obvious. (Probably the only UX upside with choosing shortcuts over Twilio, besides the free of additional charge part that is :))

dsnallfot commented 6 months ago

And another thought connected to your shared link regarding how we could share a preconfigured shortcut that runs after a trigger (The automation trigger in this case is the only thing the user needs to define self from skratch - with a provided step by step instruction of course).

Ideally we should try to make the message string and formatting in a way that it just needs one automation trigger on the receiving phone: example. The trigger automation logic could be:

When you receive a message from "Twilio, Mom or Dad", and it contains the phrase "Loop Follow Remote" -> run shortcut (our preconfigured shortcut that contains all logic/deciphering) "Loop Follow Remote Action"

And the preconfigured shortcut should be able to handle all various remote actions that we want to send and enact.

Is this in line with your ideas about the final design?

aug0211 commented 6 months ago

And another slight difference between methods, is when sending sms commands (with shortcuts) over iMessage directly from the caregivers phone to the looping phone, the "who sent this"-part at least get obvious. (Probably the only UX upside with choosing shortcuts over Twilio, besides the free of additional charge part that is :))

I was thinking of the situation where there are multiple caregivers (say mom and dad).

If we wanted to one day log who sent the command (whether that be in a Nightscout note, an SMS from the receiving phone back to caregivers, or even just by interrogating the looping phone's message history) it'd be useful to have an explicit name/ID attached to every message to see who sent it.

aug0211 commented 6 months ago

Ideally we should try to make the message string and formatting in a way that it just needs one automation trigger on the receiving phone...

...

...And the preconfigured shortcut should be able to handle all various remote actions that we want to send and enact.

I think this is preferable, though one thing I'm not totally sure of is whether automations do a better job with 1 big automation handling N steps, or N smaller automations handling 1 step. This might be worth us researching or testing at some point, or it could be a total waste of time and a non-issue either way.

The only other consideration I had is whether we want to intentionally separate out automation handlers for safety.

This would allow/force users to make a decision to say, for instance "I'm making a decision to build an automation that calls the iAPS shortcut to add carbs." If the processors are separated, this is one more cognitive checkpoint for users to make sure they want to enable this.

When grouped as one automation, of course the entire processor would be included and all remote actions would be included.

The scenario where this could be valuable is someone who is comfortable setting temp targets or profile overrides but doesn't want to send remote meals or boluses, for instance. I don't know if this extra friction is a good thing or a bad thing.

Either way, I think that's beyond the scope of the message format it self and would agree that the message format should play nicely either way (and also needs to play nicely with local shortcuts or Twilio API).

aug0211 commented 6 months ago

Here are some sample messages that I think could work.

Format

Message Type: Loop Follow Remote Command Sender Name: Secret Code: Command Type: Event Detail 1: Event Detail 2: Event Detail 3: Event Detail 4:

Examples

Note: each block of text below is an individual message. This includes 6 separate example messages.

Message Type: Loop Follow Remote Command Sender Name: Auggie Secret Code: xh8eq-38njp-rls91-in58d Command Type: Bolus Bolus Amount: 1.0

Message Type: Loop Follow Remote Command Sender Name: Auggie Secret Code: xh8eq-38njp-rls91-in58d Command Type: Meal Carbs: 35.0 Fat: 26.5 Protein: 40.0 Note: 🌮 Tacos Bolus: 3.5

Message Type: Loop Follow Remote Command Sender Name: Auggie Secret Code: xh8eq-38njp-rls91-in58d Command Type: Meal Carbs: 3.6 Fat: 0.0 Protein: 0.0 Note: 🍭 Low treatment Bolus: 0.0

Message Type: Loop Follow Remote Command Sender Name: Auggie Sender Code: xh8eq-38njp-rls91-in58d Command Type: Temp Target Target Name: 🏃‍♂️ Exercise

Message Type: Loop Follow Remote Command Sender Name: Auggie Secret Code: xh8eq-38njp-rls91-in58d Command Type: Override Override Name: 🍽️ Mealtime

Message Type: Loop Follow Remote Command Sender Name: Auggie Secret Code: xh8eq-38njp-rls91-in58d Command Type: Custom Action Custom Action: 🍔🍟 McDonald’s

aug0211 commented 6 months ago

Outstanding item:

One design principle was no omission of fields - that is, all fields and values are required. Example: do not omit carbs, fat, protein, bolus amount in the meal entry - send as 0.0 rather than leaving off.

This is done for completeness and for debugging purposes, each message of this type should always have identical structure and will explicitly declare an allowable value of 0.0 for these fields for anyone trying to read message history and understand something that happened.

Issue: how to handle "Notes" sent on a Meal entry, when no value is provided.

Since Notes is a String, the double of 0.0 wouldn't be appropriate. The text null version of a String is, well, "" - the empty string, which is exactly what I'd like to avoid stated in principles.

This leaves us with selecting a default value for this.

One option is to search for the default value in the automation and discard it if found. Another example is to consider a "default" value that we want to always display in Nightscout (current iAPS does something like "remote shortcut" if I recall correctly.

Proposal: we default this to "Sender Name" value. This has potential benefit of being valuable for anyone looking at NS to quickly identify that the entry was done by person XYZ.

Not sure this is great, because it won't be consistent in NS when other notes are included. Open to other ideas too!

dsnallfot commented 6 months ago

Here are some sample messages that I think could work.

Format

Message Type: Loop Follow Remote Command Sender Name: Secret Code: Command Type: Event Detail 1: Event Detail 2: Event Detail 3: Event Detail 4:

Examples

Note: each block of text below is an individual message. This includes 6 separate example messages.

Message Type: Loop Follow Remote Command Sender Name: Auggie Secret Code: xh8eq-38njp-rls91-in58d Command Type: Bolus Bolus Amount: 1.0

Message Type: Loop Follow Remote Command Sender Name: Auggie Secret Code: xh8eq-38njp-rls91-in58d Command Type: Meal Carbs: 35.0 Fat: 26.5 Protein: 40.0 Note: 🌮 Tacos Bolus: 3.5

Message Type: Loop Follow Remote Command Sender Name: Auggie Secret Code: xh8eq-38njp-rls91-in58d Command Type: Meal Carbs: 3.6 Fat: 0.0 Protein: 0.0 Note: 🍭 Low treatment Bolus: 0.0

Message Type: Loop Follow Remote Command Sender Name: Auggie Sender Code: xh8eq-38njp-rls91-in58d Command Type: Temp Target Target Name: 🏃‍♂️ Exercise

Message Type: Loop Follow Remote Command Sender Name: Auggie Secret Code: xh8eq-38njp-rls91-in58d Command Type: Override Override Name: 🍽️ Mealtime

Message Type: Loop Follow Remote Command Sender Name: Auggie Secret Code: xh8eq-38njp-rls91-in58d Command Type: Custom Action Custom Action: 🍔🍟 McDonald’s

I think this looks like a great structure.

There is one thing we however need to check and consider.

To be able to send the commands as a single "low cost" sms (Twilio) there are some character limitations to consider. The sms can be up to 3200 characters (split in segments and reassembled on receiving phone) in total, Twilio recommends max 320 characters. But anything over 160 results in extra cost per message. (Or only 70 characters if containing emojis)

Most of the examples above would exceed 160 characters (especially when including linebreaks \n and such. And well over 70 with emojis.

This should not be an issue for messages sent over iMessage.

https://www.twilio.com/docs/glossary/what-sms-character-limit

dsnallfot commented 6 months ago

This explains message segments well https://www.twilio.com/en-us/blog/what-the-heck-is-a-segment-html

dsnallfot commented 6 months ago

Testing some formatting right now (for the longest string, the meal string) with or without bolus enabled. Like this:

            if UserDefaultsRepository.hideRemoteBolus.value {
                // Construct and return the combinedString without bolus
                return "Remote Meal\nCarbs: \(carbs)g\nFat: \(fats)g\nProtein: \(proteins)g\nNote: \(cleanedMealNotes)\nEntered by: \(name)\nSecret code: \(secret)"
            } else {
                // Construct and return the combinedString with bolus
                return "Remote Meal\nCarbs: \(carbs)g\nFat: \(fats)g\nProtein: \(proteins)g\nNote: \(cleanedMealNotes)\nInsulin \(trimmedBolusValue)\nEntered by: \(name)\nSecret code: \(secret)"
            }

It works with iOS shortcuts and Twilio API, is readable, and as long as it doesn't contain any emojis it fits well within 1 sms segment. Feel free to tweak the text and attributes order. But just a proof of concept example.

Example notification from shortcut	SMS from Twilio	Character count

dsnallfot commented 6 months ago

I think it could be clever to use the headlines: "Remote Meal" "Remote Override" "Remote Bolus" "Remote Temp Target" and "Remote Custom Action". Or some variant of that: "Loop Follow Remote Meal" "Loop Follow Remote Bolus" and so on

That way we can choose/test to just use one message trigger automation shortcut on the receiving phone (Trigger on message from Twilio, X, Y containing "Remote" -> Run shortcut).

Or build one trigger automation per command type. Trigger on messages containing "Remote Meal" for instance.

dsnallfot commented 6 months ago

And one final note on this for now.

These two Remote Commands have now been tested end to end with two separate shortcuts ("Loop Follow Remote Meal" and Loop Follow Remote Bolus") that are triggered to run by incoming messages with the phrases "Remote Meal" and "Remote Bolus". - example screenshots are from my Swedish translated version Måltid=Meal)

The mentioned shortcuts are already possible to share to anyone and install on any iPhone without beeing tech savvy. Added import questions are

To specify which decimal separator that is used in the users instance of iAPS ( . or , )
To enter the secret code exactly as entered in Loop Follow Remote Control Settings

And. If/when we chose to alter text and or attributes that we want to use, it's quite easy to change it in the LF code as well as in the mentioned shortcuts. This testing was done to prove that infrastructure works as expected and it was successful 😊

I keep all changes in my personal branch for now, and PR to dev remote when we agreed upon formatting. No rush

dsnallfot commented 6 months ago

And one more side note. I use integers for Carbs, fat and protein in the LF code (also changed keyboard in Remote Meal View to number pad without decimals in my personal branch and just kept decimal pad for bolus entries).

This is to avoid having to convert string values to decimals/doubles and back and forth since it’s a hazzle. It’s of course possible to do, but I really don’t think we need fractions of grams when entering carbs and fpu:s, or do we? 🤔

aug0211 commented 6 months ago

We do enter decimals 😂 Maybe it’s not necessary or highly impactful. I’m curious, what do you enter for a meal that is say 19.5g carbs, 22.3g fat, 17.7g protein?

For us it’s just easier to not even think through the ambiguity and consider whether rounding up or down is better 🤷‍♂️

dsnallfot commented 6 months ago

Ok got it. I would enter 20, 22 and 18 but that is of course an added manual calculation that's unnecessary if you have some nutrition source that states exact decimals that you could just copy paste.

I don't think the nutrition facts of any food is within 0.x precision so that's not the issue, it's the issue to be able to enter as exact entries as you want to of course. 20.049 g carbs should be possible to handle for instance.

Let's put this on the to do list, to refactor all code to use any amount of decimals.

The biggest added code step is that the shortcut on the receiving phone needs to transform the provided string to the correct localized decimal separator ( . or , ) for iAPS to accept the entry. This transformation could possible be done in the LF code when creating the string before sending the sms. Right now I use US decimal separator (.). In Sweden and other EU countries we use ,

Loop follow uses . In all code but NS accepts both . and ,

example of replace decimal separator logic in shortcut automation (this is for insulin entries in remote meals)

dsnallfot commented 6 months ago

Will look into if there is an easier way to maintain the localized decimals entries from LF->sms string->Receiving phone. As long as the sending and receiving phone has the same locale it maybe could be possible to don't do any transformation of the entries at all. (but to use double instead of decimal values for all calculations in the code, and to present the values as entered when creating the strings).

dsnallfot commented 6 months ago

Code to allow decimal carbs and FPUs now added to current PR #64 awaiting approval. The big reformatting work is not in that PR however. All those changes can go into a new PR when we decided how it should look

aug0211 commented 5 months ago

@dsnallfot do you mind sharing the full schema/format you're using right now? Similar to what's in this comment?

From our direct messages, and from the screen shot you have in this comment above, I think you've uncovered a lot of the "hard limits" we're up against that we did not know about early on (mostly character limits to keep costs low for users).

It seems that the format you're using right now is not only meeting your personal needs, but is in line with the principles we set out with at our start.

I'd like to propose we use what you have - if you don't mind writing up the format to share here for one final review (and for future use in documentation).

dsnallfot commented 5 months ago

I will write down the formatting shortly like in your comment above. I have included my suggested formatting in a PR #70 since it was connected to the code used for disabling send buttons etc (we can of course tweak the formatting anyhow we want, but with this PR "my" latest version is default until else is decided)

dsnallfot commented 5 months ago

Below is the structure I use right now with examples how it looks like in iMessage history on the receiving phone (and what's in PR #70)

General Structure:

Command Name (Useful as trigger phrase in receiving phone automation)
Entries to use in iAPS shortcuts (Amount carbs, amount fat, amount protein, amount bolus, notes, override names, temps target names, and custom action names)
Entered by: Name
Secret Code

Remote Custom Action example: Remote Custom Action 🥪 Standard Lunch Entered by: Dad Secret Code: S3cr3tc0d3

Remote Meal example: Remote Meal Carbs: 20g Fat: 25.5g Protein: 15g Notes: 🍕 Pizza Insulin: 1.65U Entered by: Dad Secret Code: S3cr3tc0d3

Remote Meal example (When remote bolus is toggled off in settings): Remote Meal Carbs: 20g Fat: 25.5g Protein: 15g Notes: 🍕 Pizza Entered by: Dad Secret Code: S3cr3tc0d3

Remote Bolus example: Remote Bolus Insulin: 1.65U Entered by: Dad Secret Code: S3cr3tc0d3

Remote Override example: Remote Override 🏄‍♂️ Surfing profile Entered by: Dad Secret Code: S3cr3tc0d3

Remote Temp Target example: Remote Temp Target 🍬 After low Entered by: Dad Secret Code: S3cr3tc0d3

And the exact code/strings for each command looks like this right now:

Remote Custom Action

"Remote Custom Action\n\(selectedCustomAction)\nEntered by: \(name)\nSecret Code: \(secret)"

Remote Meal

"Remote Meal\nCarbs: \(carbsValue)g\nFat: \(fatsValue)g\nProtein: \(proteinsValue)g\nNotes: \(cleanedMealNotes)\nInsulin: \(trimmedBolusValue)U\nEntered by: \(name)\nSecret Code: \(secret)"

Remote Meal (when Remote Bolus is toggled off in settings)

"Remote Meal\nCarbs: \(carbsValue)g\nFat: \(fatsValue)g\nProtein: \(proteinsValue)g\nNotes: \(cleanedMealNotes)\nEntered by: \(name)\nSecret Code: \(secret)"

Remote Bolus

"Remote Bolus\nInsulin: \(trimmedBolusValue)U\nEntered by: \(name)\nSecret Code: \(secret)"

Remote Override

"Remote Override\n\(selectedOverride)\nEntered by: \(name)\nSecret Code: \(secret)"

Remote Temp Target

"Remote Temp Target\n\(selectedTempTarget)\nEntered by: \(name)\nSecret Code: \(secret)"

dsnallfot commented 5 months ago

This is how a meal message with only carbs entered would look (Notes empty, but all other fields 0.0

And this is a meal message with only carbs (when remote bolus is toggled off in settings)

dsnallfot / LoopFollowiAPS

Refactor message string formatting #47

Principle 1

Machine readable

Principle 2

Human Readable

Principle 3

Simplify iOS Shortcut Automation development

Principle 4

Traceability

Format

Examples

Format

Examples