Pad block errors on initial notifications

[cconstab]

Describe the bug When sending the first notification when using atTalk the first messages fail due to pad block failures.

To Reproduce Steps to reproduce the behavior:

1. First I created two new atSigns
2. Then ran atTalk for them to chat
3. And then on the first message sent it is not decrypyted and the Pad Block error is given.
4. The second and further messages work just fine

Expected behavior The first and any other notifications should be encrypted correctly

Screenshots pair of atSigns talking to each other

cconstab@servalan:~/at_talk$ dart bin/at_talk.dart -a "@22string" -t "@disappointed1" -v 
INFO|2022-07-14 22:05:57.723904|HiveBase|commit_log_7442a3f42488e80740f61b68d7abb1e7059978ae40e5651598a2ea3ffd727f15 initialized successfully 

INFO|2022-07-14 22:05:57.741977|HiveBase|7442a3f42488e80740f61b68d7abb1e7059978ae40e5651598a2ea3ffd727f15 initialized successfully 

INFO|2022-07-14 22:05:59.177485|Monitor|monitor started for @22string with last notification time: null 

INFO|2022-07-14 22:06:00.501848|AtLookup|auth success 

INFO|2022-07-14 22:06:00.528163| atTalk |Waiting for initial sync 

Synching your data............INFO|2022-07-14 22:06:06.439768|AtLookup|auth success 

.INFO|2022-07-14 22:06:06.561561|SyncService|Returning the serverCommitId 3 

INFO|2022-07-14 22:06:06.684874|SyncService|Returning the serverCommitId 3 

INFO|2022-07-14 22:06:06.827320|SyncService|Inside syncComplete  SyncRequestSource.app  : Closure: (dynamic) => void 

INFO|2022-07-14 22:06:06.827395|SyncService|Sending result to onDone callback 

INFO|2022-07-14 22:06:06.827551| atTalk |syncResult.syncStatus: SyncStatus.success 

INFO|2022-07-14 22:06:06.827637| atTalk |syncResult.lastSyncedOn 2022-07-15 05:06:06.827236Z 

INFO|2022-07-14 22:06:06.948857|SyncService|Returning the serverCommitId 7 

.
@22string: hello
@22string: INFO|2022-07-14 22:06:13.656188| atTalk |SUCCESS:id: 23dc96dc-9f17-4edd-a624-be88db006cf4 status: NotificationStatusEnum.delivered 

INFO|2022-07-14 22:06:15.191857|SyncService|Returning the serverCommitId 8 

INFO|2022-07-14 22:06:15.312833|SyncService|Returning the serverCommitId 8 

INFO|2022-07-14 22:06:15.715496|SyncService|Inside syncComplete  SyncRequestSource.app  : null 

INFO|2022-07-14 22:06:15.836817|SyncService|Returning the serverCommitId 11 

hello
@22string: INFO|2022-07-14 22:06:34.599005| atTalk |SUCCESS:id: f1a0645e-a3dd-4ec3-861c-8e5ff05db6c6 status: NotificationStatusEnum.delivered 

SEVERE|2022-07-14 22:06:47.109737|NotificationServiceImpl|unable to decrypt notification value: Exception: Internal server exception : Request to remote secondary @disappointed1 at 914f81f4 

INFO|2022-07-14 22:06:47.110333| atTalk |atTalk update recieved from @disappointed1 notification id : dd06f198-d7e1-48d7-9503-f905b31976e6 

@disappointed1: hzbJtPKqKjUHesQrbt1w2g==
@22string: INFO|2022-07-14 22:06:50.190066|SyncService|Returning the serverCommitId 12 

INFO|2022-07-14 22:06:50.311035|SyncService|Returning the serverCommitId 12 

INFO|2022-07-14 22:06:50.566329|SyncService|Inside syncComplete  SyncRequestSource.app  : null 

SEVERE|2022-07-14 22:06:50.688672|EncryptionUtil|Error while decrypting value: Invalid argument(s): Invalid or corrupted pad block #0      PKCS7Padding.padCount (package:pointycastle/paddings/pkcs7.dart:42:7)
#1      PaddedBlockCipherImpl.doFinal (package:pointycastle/padded_block_cipher/padded_block_cipher_impl.dart:112:30)
#2      PaddedBlockCipherImpl.process (package:pointycastle/padded_block_cipher/padded_block_cipher_impl.dart:74:25)
#3      AES.decrypt (package:encrypt/src/algorithms/aes.dart:63:22)
#4      Encrypter.decryptBytes (package:encrypt/src/encrypter.dart:25:17)
#5      Encrypter.decrypt (package:encrypt/src/encrypter.dart:31:17)
#6      Encrypter.decrypt64 (package:encrypt/src/encrypter.dart:41:12)
#7      EncryptionUtil.decryptValue (package:at_client/src/util/encryption_util.dart:30:24)
#8      SharedKeyDecryption.decrypt (package:at_client/src/decryption_service/shared_key_decryption.dart:71:27)
<asynchronous suspension>
#9      NotificationServiceImpl._getDecryptedNotifications (package:at_client/src/service/notification_service_impl.dart:298:11)
<asynchronous suspension>
#10     NotificationServiceImpl._internalNotificationCallback.<anonymous closure> (package:at_client/src/service/notification_service_impl.dart:142:17)
<asynchronous suspension>

SEVERE|2022-07-14 22:06:50.688902|NotificationServiceImpl|unable to decrypt notification value: Exception: Invalid argument(s): Invalid or corrupted pad block 

INFO|2022-07-14 22:06:50.689059|SyncService|Returning the serverCommitId 15 

INFO|2022-07-14 22:06:55.190859|SyncService|Returning the serverCommitId 15 

INFO|2022-07-14 22:06:55.311887|SyncService|Returning the serverCommitId 15 

SEVERE|2022-07-14 22:06:55.557046|SyncService|update/delete for key cached:@22string:attalk.ai6bh@disappointed1 failed. Error code AT0003 error message Invalid syntax 

INFO|2022-07-14 22:06:55.557863|SyncService|Inside syncComplete  SyncRequestSource.system  : Closure: (SyncResult) => void from Function '_onDone@94025363':. 

INFO|2022-07-14 22:06:55.604154| atTalk |atTalk update recieved from @disappointed1 notification id : 33d41b10-ca6f-4eb6-b183-7e19b4b04764 

@disappointed1: there
@22string: INFO|2022-07-14 22:06:55.669839|SyncService|Returning the serverCommitId 17 

INFO|2022-07-14 22:07:00.202868|SyncService|Returning the serverCommitId 17 

INFO|2022-07-14 22:07:00.334847|SyncService|Returning the serverCommitId 17 

SEVERE|2022-07-14 22:07:00.600923|SyncService|update/delete for key cached:@22string:shared_key@disappointed1 failed. Error code AT0003 error message Invalid syntax 

INFO|2022-07-14 22:07:00.601553|SyncService|Inside syncComplete  SyncRequestSource.app  : null 

INFO|2022-07-14 22:07:00.722832|SyncService|Returning the serverCommitId 18 

@22string: 
@22string: ^C
cconstab@servalan:~/at_talk$

and the second

cconstab@servalan:~/at_talk$ dart bin/at_talk.dart -a "@disappointed1" -t "@22string" -v
INFO|2022-07-14 22:05:51.615216|HiveBase|commit_log_83e2e1f0d8e1418c168ee31d8bb481cf9c8b34c62a6de2b258bafb6588ea08b4 initialized successfully 

INFO|2022-07-14 22:05:51.634042|HiveBase|83e2e1f0d8e1418c168ee31d8bb481cf9c8b34c62a6de2b258bafb6588ea08b4 initialized successfully 

INFO|2022-07-14 22:05:53.144470|Monitor|monitor started for @disappointed1 with last notification time: null 

INFO|2022-07-14 22:05:54.468799|AtLookup|auth success 

INFO|2022-07-14 22:05:54.499208| atTalk |Waiting for initial sync 

Synching your data...............INFO|2022-07-14 22:06:01.602758|AtLookup|auth success 

INFO|2022-07-14 22:06:01.724503|SyncService|Returning the serverCommitId 2 

INFO|2022-07-14 22:06:01.847878|SyncService|Returning the serverCommitId 2 

.INFO|2022-07-14 22:06:02.111885|SyncService|Inside syncComplete  SyncRequestSource.app  : Closure: (dynamic) => void 

INFO|2022-07-14 22:06:02.111967|SyncService|Sending result to onDone callback 

INFO|2022-07-14 22:06:02.112119| atTalk |syncResult.syncStatus: SyncStatus.success 

INFO|2022-07-14 22:06:02.112195| atTalk |syncResult.lastSyncedOn 2022-07-15 05:06:02.111803Z 

INFO|2022-07-14 22:06:02.232997|SyncService|Returning the serverCommitId 3 

.
@disappointed1: SEVERE|2022-07-14 22:06:13.102927|NotificationServiceImpl|unable to decrypt notification value: Exception: Internal server exception : Request to remote secondary @22string at 21768de0 

INFO|2022-07-14 22:06:13.103486| atTalk |atTalk update recieved from @22string notification id : 23dc96dc-9f17-4edd-a624-be88db006cf4 

@22string: ksM2BBOfcd+Y5SPyiIen+Q==
@disappointed1: SEVERE|2022-07-14 22:06:15.849474|EncryptionUtil|Error while decrypting value: Invalid argument(s): Invalid or corrupted pad block #0      PKCS7Padding.padCount (package:pointycastle/paddings/pkcs7.dart:42:7)
#1      PaddedBlockCipherImpl.doFinal (package:pointycastle/padded_block_cipher/padded_block_cipher_impl.dart:112:30)
#2      PaddedBlockCipherImpl.process (package:pointycastle/padded_block_cipher/padded_block_cipher_impl.dart:74:25)
#3      AES.decrypt (package:encrypt/src/algorithms/aes.dart:63:22)
#4      Encrypter.decryptBytes (package:encrypt/src/encrypter.dart:25:17)
#5      Encrypter.decrypt (package:encrypt/src/encrypter.dart:31:17)
#6      Encrypter.decrypt64 (package:encrypt/src/encrypter.dart:41:12)
#7      EncryptionUtil.decryptValue (package:at_client/src/util/encryption_util.dart:30:24)
#8      SharedKeyDecryption.decrypt (package:at_client/src/decryption_service/shared_key_decryption.dart:71:27)
<asynchronous suspension>
#9      NotificationServiceImpl._getDecryptedNotifications (package:at_client/src/service/notification_service_impl.dart:298:11)
<asynchronous suspension>
#10     NotificationServiceImpl._internalNotificationCallback.<anonymous closure> (package:at_client/src/service/notification_service_impl.dart:142:17)
<asynchronous suspension>

SEVERE|2022-07-14 22:06:15.849701|NotificationServiceImpl|unable to decrypt notification value: Exception: Invalid argument(s): Invalid or corrupted pad block 

INFO|2022-07-14 22:06:20.191895|SyncService|Returning the serverCommitId 5 

INFO|2022-07-14 22:06:20.313044|SyncService|Returning the serverCommitId 5 

INFO|2022-07-14 22:06:20.749187|SyncService|Inside syncComplete  SyncRequestSource.app  : null 

INFO|2022-07-14 22:06:21.189041|SyncService|Returning the serverCommitId 10 

INFO|2022-07-14 22:06:32.484607| atTalk |atTalk update recieved from @22string notification id : f1a0645e-a3dd-4ec3-861c-8e5ff05db6c6 

@22string: hello
@disappointed1: INFO|2022-07-14 22:06:40.190066|SyncService|Returning the serverCommitId 11 

INFO|2022-07-14 22:06:40.311054|SyncService|Returning the serverCommitId 11 

SEVERE|2022-07-14 22:06:40.555221|SyncService|update/delete for key cached:@disappointed1:attalk.ai6bh@22string failed. Error code AT0003 error message Invalid syntax 

SEVERE|2022-07-14 22:06:40.555654|SyncService|update/delete for key cached:@disappointed1:shared_key@22string failed. Error code AT0003 error message Invalid syntax 

INFO|2022-07-14 22:06:40.556100|SyncService|Inside syncComplete  SyncRequestSource.app  : null 

INFO|2022-07-14 22:06:40.677056|SyncService|Returning the serverCommitId 12 

there
@disappointed1: INFO|2022-07-14 22:06:47.898381| atTalk |SUCCESS:id: dd06f198-d7e1-48d7-9503-f905b31976e6 status: NotificationStatusEnum.delivered 

INFO|2022-07-14 22:06:50.190048|SyncService|Returning the serverCommitId 13 

INFO|2022-07-14 22:06:50.311036|SyncService|Returning the serverCommitId 13 

SEVERE|2022-07-14 22:06:50.567070|SyncService|update/delete for key cached:@disappointed1:attalk.ai6bh@22string failed. Error code AT0003 error message Invalid syntax 

INFO|2022-07-14 22:06:50.568441|SyncService|Inside syncComplete  SyncRequestSource.app  : null 

INFO|2022-07-14 22:06:50.689064|SyncService|Returning the serverCommitId 16 

there
@disappointed1: INFO|2022-07-14 22:06:57.719935| atTalk |SUCCESS:id: 33d41b10-ca6f-4eb6-b183-7e19b4b04764 status: NotificationStatusEnum.delivered 

@disappointed1: 
@disappointed1: ^C
cconstab@servalan:~/at_talk$ 

Note the encrypted message is sent ksM2BBOfcd+Y5SPyiIen+Q== but the receiver does not know how to decrypt is my assumption ?

Were you using an @‎application when the bug was found? -atTalk

Additional context Add any other context about the problem here.

[sitaram-kalluri]

The initial notifications fail to decrypt because the when notifying with value, the notification reaches first to the receiver's atsign, but the encrypted shared_key (which is needed to decrypt the value on receiver end) gets sync'ed to the sender cloud secondary (from the sender's client) at a later point of time. Hence receiver fails to find the shared_key which leads to pad block errors.

Attaching the client-server logs.(Sender - sitaram and Receiver - murali) pad-block-issue-logs.zip

[gkc]

@cconstab as a partial mitigation workaround we could have the sender start (before sync) by sharing something (anything) with the other atSign, then waiting for sync.

[gkc]

@kalluriramkumar what are your thoughts so far re a fix? We can discuss in tomorrows early sync

[gkc]

Also - we should be able to do better than just "pad block" - can we improve the exception handling here in terms of intent and chaining and explanation? I'd like to see something like, "Failed to decrypt value shared by atSign because we have not yet received the encryption shared key from them"

[cconstab]

As a side note... I see this odd cached key in the public space which I do not think it should ever be public.

cconstab@cally:~$ openssl s_client -tls1_2 -ign_eof -quiet e32bbf28-4350-53eb-a7b2-0a08829d485f.swarm0002.atsign.zone:3644
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = e32bbf28-4350-53eb-a7b2-0a08829d485f.swarm0002.atsign.zone
verify return:1
@scan
data:["cached:publickey@iot_device02","cached:publickey@iot_manager","publickey","publickey@iot_device01","signing_publickey@iot_device01"]
@
error:AT0003-Invalid syntax : invalid command
@read:errno=0
cconstab@cally:~$

I wondered if the key is actually in place but not being found as it is here ?

[sitaram-kalluri]

Also - we should be able to do better than just "pad block" - can we improve the exception handling here in terms of intent and chaining and explanation? I'd like to see something like, "Failed to decrypt value shared by atSign because we have not yet received the encryption shared key from them"

Sure Gary. will implement this.

[sitaram-kalluri]

Also - we should be able to do better than just "pad block" - can we improve the exception handling here in terms of intent and chaining and explanation? I'd like to see something like, "Failed to decrypt value shared by atSign because we have not yet received the encryption shared key from them"

This is implemented in the following PR-621. This specific changes are in encryption_util.dart file and notification_service_impl.dart-> _internalNotificationCallback in the PR.

[sitaram-kalluri]

The changes related to exception chaining are completed and need to merge it trunk branch. Futher @gkc proposed to write the shared_key directly to remote secondary instead of sync to expedite the sync'ing of shared key. Hence carry forwarding the issue to PR-43

[sitaram-kalluri]

• [x] Chain the exceptions
• [ ] Write shared_key to cloud secondary expedite the sync'ing of shared key

[cconstab]

Any update on this issue ?

[sitaram-kalluri]

Any update on this issue ?

@cconstab : The changes related to the exception chaining are merged and published. To expedite the sync'ing of the shared_key, we have approach ready. But it needs a minor change in sync_conflict_resolution code. Hence wanted to check with @gkc before pushing the changes.

[gkc]

@cconstab : But it needs a minor change in sync_conflict_resolution code. Hence wanted to check with @gkc before pushing the changes.

@kalluriramkumar What do I need to review?

[sitaram-kalluri]

@gkc: When sync'ing the shared key to remote secondary directly, the key gets sync'ed back to local secondary (because server is ahead of the local). At this point, failure occurs in sync conflict resolution because it tried to decrypt the data (we do not encrypt the shared_key).

As part of fixing other bug @murali-shris, added logic in sync conflict resolution to check if isEncrypted is set to true before decrypting the data should solve my issue as well. I will add the changes back and run tests. If i still see the issue, will bring up the discussion. At the moment i good to proceed further.

[gkc]

We (@gkc @VJag @murali-shris and @kalluriramkumar) discussed this on Thursday Oct 13, @Vjag will review where we are and how we will fix this properly. @VJag when you are ready, please can you add (into this ticket) your assessment of where we are and recommendations for what we need to do

[gkc]

@gkc and @VJag had follow-up discussion today.

In summary, we need to break down this important lifecycle (delivery of shared encryption key to recipient) into several parts

1. Sender generates symmetric encryption key, and encrypts it with asymmetric public key of recipient ==> Requires public key of recipient to be available.
2. Sender delivers symmetric encryption key to its cloud secondary ==> Need to add collision detection here also, in the cloud secondary, to cover the scenario where a different sender client has already created and shared a symmetric encryption key for this recipient
3. Sender may now send notifications to the recipient which are encrypted using the symmetric encryption key

Steps 1 and 2 can be encapsulated into a client-side 'getSharedEncryptionKey' method, which must have a mutex to prevent race conditions, and which must throw appropriate exceptions if (1) there is no existing key which has been shared and (2) the public key of the recipient is not available so cannot encrypt a new shared key, or the key has not yet been delivered to the sender's cloud secondary (and possible collision detection managed)

As long as encrypting code, whether sending notifications or sharing data, always uses this getSharedEncryptionKey method, then we can be assured that the recipient will always have the symmetric key available to them when they need it, as notification delivery sequence will ensure that the auto-notify of the symmetric key creation is delivered before any other notifications. This will enable us to remove the current workaround which involves including the full shared key in the metadata of data shared via updates, and as a result payloads will see a large decrease in size

The collision management aspect of this is what requires a sender to not use a shared encryption key until it has been delivered to the sender's cloud secondary.

We discussed other aspects as well, which we will return to in the future:

1. atSign resets
2. Allow clients to create multiple shared encryption keys. We will assign them unique identifiers in a reserved part of the namespace, and we will make them immutable after initial creation. We can then include just the identifier of the shared encryption key in metadata, which the recipient can use to retrieve the correct key.
3. Key metadata (which algorithm, key length, etc)

[gkc]

Burned 3SP in PR47. Moving to PR48. Estimate 8SP for implementation including extensive testing to ensure we have a rock-solid solution.

[gkc]

@VJag if you think this is 13SP please update the label

[VJag]

Notes from the meeting that I and Gary had on the potential solutions:

Sol 1: Make the shared encryption key part of the payload Conclusion: It was concluded that making the shared encryption key part of the payload is not the best way to solve the problem as it increases the payload size. When the key is cached and accessible to the recipient sending it repeatedly as part of the payload can be avoided.

Sol 2: Shared key generation process in the client (SDK) should only return a value when it synced to the cloud secondary Conclusion: This solution ensures that the shared encryption is definitely available to the recipient as the one who is sharing is ensuring that the key reaches the cloud secondary at the minimum.

However, there can be a timing issue when a notification sent reaches the recipient and then the client goes offline. At that point, though the shared key is available in the sender or the recipient's cloud server as long as it is not in the recipient's local secondary it will still result in a decryption issue.

A general issue with shared key generation is two clients for an atSign generating it in parallel. At this point, one of them will override the other after a sync. We were discussing ways to perform conflict resolution in the server for this scenario.

8SP is what we can spend in this sprint. The time needed on this ticket is, to find the solution and implement it.

[VJag]

Another solution approach I have in my mind is:

"A sender continues to send the shared AES encryption key as part of the payload till the time he sees a notification/cached key from the recipient to indicate that the shared encryption key is available to the client"

Let's say this confirmation key is "cached:@gary:share_key_accessible@jagan". So @jagan will make the shared key part of the payload as long as he does not see this key. When the recipient sees that the shared key is synced to the client and is available for decryption, a confirmation in the form of a cached key is sent. Now the recipient on seeing this will stop sending the key in the payload.

[gkc]

Continuing to send key in payload fixes one problem but does not address the issue regarding collision when more than one sender client is in play, and both of them create a symmetric encryption key (and we assume there is only one in the data store)

[VJag]

Yes, Gary. Though it is not elegant, it does solve the two-client problem. The explanation would be when the key is part of the payload, the one from the payload should be used for decryption. So it does not matter if the two clients would have generated two keys as the keys are still decryptable. However, we still have a problem where the recipient might get two cached shared keys at two different points in time (The single shared key). I will try to explain this better in the arch call tonight.

[gkc]

True. But we need to agree that either (1) we will have a single shared encryption key per atsign pair (current intended behaviour) or (2) we can have multiple shared encryption keys per atsign pair

(2) is where I believe we need to go anyway. If we stick with (1) then we need to make it right

[cconstab]

I think we should just stick with one aes key as it's tough to manage 1 let alone X.

So we need some where to offer a mutex for aes creation.. The atServer is well placed I think for that.. More discussion to be

A diagram that shows a way to not have atClients create a new shared AES key with a new atSign for tomorrows arch call or for you to improve on over night..

https://mermaid.live/edit#pako:eNrtVu9v0zAQ_VdO_kpX2jRNt4AGAyaYEAhRJCSUL05yaaw5dnAcWJj2v3NO0h-MDlhXJISmfmhr3917z-d7ySVLdIosZBV-rlEl-ELwheFFpEpurEhEyZWFp1yKBN_lWuG2jQ88lmi37cwx0Srlprm2Gev4F1s90AYqHBwfXy8ZgtT6vC7Dso5p4zU2LneZ1nG6Td5bbREkZhZ0tqk4hJPT-Tk28BIVGm4xBZGB0hb4Fy6kw6GKCZeyeRybh8ethPfzk1X5drWuKM9qoDM2TWnbta7uVuSO_t-H_u0Zu4oQnnapVIdbKgsWL2zpkrp4iNgrlFLDMx0PINdfgRuERtdPIgYPYJ3dwT76kybdDGzbrLshr4A68M37GLoDFhl971XrLRHvKHKz_Aqwv849WIFVxRe4N5U7Qd5NZjs4RizybnLWeGeKiFTCIjkKRS9zhVbgxinTBmJt845IBYkuhFpAZnTRX8RIuY-60RI-5tyCqEAhpm4qK-CQ5JicA1cpVI1K3KQ6aIeXc9qXBnnaQIyoYLGcZ4fyz_jc2Zqxk7bNZoBOjg6xWjW6qF0bacH-uO6OISHFpHEANifNdZm2DkZOtA7bv2PuIsKR7YUk3HWduvSzpFaFwa7NtbJCtkiu9yuE_enZ6CNJetOxIyxHK24gkcJdbseceDkibqQcdRfAM4sGpj35aiPdoEROTIb33n_v_f-l9z93zl_tbv1swAo0BRcpvRZfRgqIG41YgREL6WeKGa-ljVikrii0c7XTVFhtWJhxWeGA8drqOT0DWGhNjcug_tV6FUXvu5-0LpZB9JeFl-yChTNvMpzOgsNx4M0m_jQIBqxh4YE39Yb-ke8H_ujwcBIER9OrAfvWVhgP_ZnnTUbB1B-Pvdl4FFx9B_kTKYM

[gkc]

@cconstab once we've solved this for one AES key, the same pattern will work for many. We can start by making this work for what we have now - one. In future we will need to support at least two for when it becomes time to re-encrypt and re-share as we move to a higher key length or different algorithm or whatever.

We can use the server as mutex indeed that's by far the simplest way to solve the "simultaneous creation of shared aes key by multiple clients" problem. It is worth mentioning though that enabling multiple keys to be used will render this a non issue as each client can cut its own & use a guid as its ID

However : I can't see your diagram on my phone right now, but I don't see how we can cut the key on the server in a way that keeps the key invisible to the server??

[cconstab]

We do not cut the key on the server instead we use it to manage a mutex.

[gkc]

Oh ok I thought you were proposing something new. (Can't read diagram on my phone!) Yes that's what Jagan & I were discussing; we have two ways of doing it : (1) server code to handle this specific case (2) introduce concept of variable mutability e.g. in this case we would want write once, no subsequent delete or change (but ttl would hold, for auto deletion)

[VJag]

sequenceDiagram
participant @alicePhone
participant @aliceTablet
participant @aliceSecondary
participant @bobSecondary
participant @bobPhone
@alicePhone ->> @aliceSecondary: lookup:publicKey@bob
@aliceTablet ->> @aliceSecondary: lookup:publicKey@bob
Note left of @alicePhone: AESkey Generated if not available locally<br/> @bobRSApublicKey<br/>used to encrypt<br/>AESkey
Note left of @aliceTablet: AESkey Generated if not available locally<br/> @bobRSApublicKey<br/>used to encrypt<br/>AESkey
@alicePhone ->> @aliceSecondary: @bob :Encrypted atKey textphone@alice "Hello Bob, how are you?" + Encrypted AESkey;
@aliceTablet ->> @aliceSecondary: @bob :Encrypted atKey texttablet@alice "Hello Bob, how are you?" + Encrypted AESkey;
@aliceSecondary ->> @bobSecondary: notify: atKey textphone@alice "Hello Bob, how are you?" + Encrypted AESkey;
@aliceSecondary ->> @bobSecondary: notify: atKey texttablet@alice "Hello Bob, how are you?" + Encrypted AESkey;
@bobSecondary ->> @bobPhone: notify:message atKey textphone@alice "Hello Bob, how are you?" + Encrypted AESkey;
@bobSecondary ->> @bobPhone: notify:message atKey texttablet@alice "Hello Bob, how are you?" + Encrypted AESkey;
Note right of @bobPhone: Inconsitent AES Encryption key for both atKeys coming from @alice

note left of @alicePhone: What is needed is a check and sync if AES key has already been generated

@alicePhone ->> @aliceSecondary: lookup:publicKey@bob
@aliceTablet ->> @aliceSecondary: lookup:publicKey@bob
Note left of @alicePhone: If AES key is not available locally or on secondary mutext set on secondary and created, then updated to secondary <br/> @bobRSApublicKey<br/>used to encrypt<br/>AESkey
Note left of @aliceTablet: If AES key is not available locally or on secondary and mutext cannot be set on secondary then recheck until AES is available <br/> @bobRSApublicKey<br/>used to encrypt<br/>AESkey
Note left of @aliceSecondary: If Mutext is set by client and the AESKey not set after 5 seconds Mutext is released.
@alicePhone ->> @aliceSecondary: @bob :Encrypted atKey textphone@alice "Hello Bob, how are you?" + Encrypted AESkey;
@aliceTablet ->> @aliceSecondary: @bob :Encrypted atKey texttablet@alice "Hello Bob, how are you?" + Encrypted AESkey;
@aliceSecondary ->> @bobSecondary: notify: atKey textphone@alice "Hello Bob, how are you?" + Encrypted AESkey;
@aliceSecondary ->> @bobSecondary: notify: atKey texttablet@alice "Hello Bob, how are you?" + Encrypted AESkey;
@bobSecondary ->> @bobPhone: notify:message atKey textphone@alice "Hello Bob, how are you?" + Encrypted AESkey;
@bobSecondary ->> @bobPhone: notify:message atKey texttablet@alice "Hello Bob, how are you?" + Encrypted AESkey;
Note right of @bobPhone: Consistent AES Encryption key for both atKeys coming from @alice

[VJag]

Colin's diagram is embedded as a mark down. Its the same diagram.

[sitaram-kalluri]

@gkc: Had a discussion with @VJag and the following is the first step in implementation - Ensure the notifications are not triggered unless the encrypted shared is synced to the respective atsign cloud secondary. Can you please confirm if this is fine?

[gkc]

Sounds good, yes

[gkc]

Can we try to finish this off in PR50?

[gkc]

Clearing the estimate; we should re-estimate what it's going to take to complete this. @kalluriramkumar can you lay out all of the major tasks required to complete this work, please?

[sitaram-kalluri]

Clearing the estimate; we should re-estimate what it's going to take to complete this. @kalluriramkumar can you lay out all of the major tasks required to complete this work, please?

@gkc: Based on my discussion with Jagan the contract on the client should be "SDK should prevent a key or a notification to be shared with another atSign till the time the SDK is sure that the shared key is available to the recipient"

For this to be implemented:

On a high level, refactor getSharedKey method follows:

1. Look IF the shared_key is already available in the Keystore (local and cloud secondary)
2. Add a new local variable (local:isSharedKeySynced) which holds a boolean value to indicate the state if shared_key is synced to the cloud secondary - (This is optional, we can enhance this to a better way that gives info if shared_key is synced to cloud secondary or not)
3. If shared_key is synced to cloud secondary, return the shared key.
4. If shared_key is not available from step-1, Generate a new shared_key
5. Trigger "sync" to sync the shared_key to the cloud secondary
6. On successful sync completion, update the local variable
7. Return the new shared key

[sitaram-kalluri]

@gkc: Below is the flow chart of the proposed solution:

flowchart TD
    Y(((Start)))
    Y --> A
    A[Put a key]
    B[Notify a key with value]
    Y --> B
    A --> C[Look for encryptedSharedKey]
    B --> C
    C --> D{is encryptedSharedKey found}
    D --> |Found| E{Is encryptedSharedKey synced}
    E --> |Synced| K[Return encryptedSharedKey] --> Z
    E --> |Not Synced| L[Run Update verb to sync encryptedSharedKey on cloud]
    D --> |Not found| F[Generate AES Key]
    F --> G{look for cached encryptionPublicKey}
    G --> |Found| H[Encrypt the shared Key]
    G --> |Not found| I[Run plookup verb to fetch encryptionPublicKey]
    I --> |Found| H
    I --> |Not found| J[Throw AtPublicKeyNotFoundException] --> Z(((end)))
    H --> L
    L --> M{is sync successful}
    M --> |Yes| K
    M --> |No| N[Rethrow exception received when running update verb] --> Z

[gkc]

@kalluriramkumar Can you please link the relevant branch(es) and/or PR(s) to this ticket, and can you also please provide a brief summary of what's left to do?

[sitaram-kalluri]

@kalluriramkumar Can you please link the relevant branch(es) and/or PR(s) to this ticket, and can you also please provide a brief summary of what's left to do?

Completed changes in at_client SDK and pushed the changes to the following branch : initial_pad_block_issue

Pending work: In the persistence secondary, at the moment, we do not have a way to get the latest commit entry for a given key to check if the key is synced to the cloud secondary. Also, the same at_commit_log is used between the at_client SDK and at_secondary server which leads to bugs because a fix to the client SDK bug might have a side effect on the at_secondary server and vice versa. So, to fix the above problems following refactorings are planned:

1. Make at_commit log abstract and have a separate concrete class for the Client commit log and server commit log
2. Have a new method in the Client Commit log to retrieve the latest commit entry of a key.

Completed the code refactorings, dev testing, PR review and merge to trunk are pending.

[gkc]

Thanks @kalluriramkumar. I don't think we need to do estimation poker for this, can you add your own SP estimate here for the remaining effort please?

[sitaram-kalluri]

Following are the draft PR that fixes the issue: https://github.com/atsign-foundation/at_client_sdk/pull/823

[sitaram-kalluri]

The changes are merged to trunk branch

[sitaram-kalluri]

The changes are published in at_client - v3.0.49

[cconstab]

confirmed working thank you