search

ethereum / portal-network-specs

Official repository for specifications for the Portal Network
316 stars 85 forks source link

add spec for portal_historyTraceRecursiveFindContent #236

Closed mynameisdaniil closed 1 year ago

mynameisdaniil commented 1 year ago

PREFACE: There's a portal_historyTraceRecursiveFindContent call that trin and glados support and use. I've been trying to implement it for fluffy and found few inconsistencies (https://github.com/ethereum/trin/issues/987). This PR is an attempt to fix shortcomings that I've found and make this improved API implementation standardized.

Here's a spec for portal_historyTraceRecursiveFindContent request, that is implemented by trin and is suported by glados. This spc differs from original idea described in https://github.com/ethereum/trin/pull/482 and also differs from current implementation. Main differences being: removed unused fields, removed extra fields (i.e. we dont need port and ip when we have enr), all fields are camelCase. Also, there is support for tracing cancelled requests e.g. we received data from somewhere else, so we cancel all other requests. Here's an example of the correct JSON response:

{
  "jsonrpc": "2.0",
  "id": "1",
  "result": {
    "content": "0x...",
    "utpTransfer": true,
    "trace": {
      "origin": "0xe0df4ef723b2961c6a35478001f84dfbdf4df053bc5ab3e754881b03e2e10d4a",
      "targetId": "0xeb53a191fa5bb22d1b931396442540d41f2180cc17110c9dee1b0e7107ec59ee",
      "receivedFrom": "0xebf07f5e9dc2d34b8fd65cf61f011db854ca64a2d1868235712fd09fdb0ba1cc",
      "responses": {
        "0xe0df4ef723b2961c6a35478001f84dfbdf4df053bc5ab3e754881b03e2e10d4a": {
          "durationMs": 0,
          "respondedWith": [
            "0xe0df4ef723b2961c6a35478001f84dfbdf4df053bc5ab3e754881b03e2e10d4a",
            "0xec9b86d41a6e4aa9be617fe6a4d10b2d45cbc37ba8884ccfafbafbccd20ac6e9",
            "0xced7d0a627c603fb55b965bd50691d23e7f78315105ed8f16c5a3de2b8566951",
            "0xef8239a84cd7fef2b2fa981862a83484415fa6682cd048e2f3c6fdc2b16c67a8",
            "0xc90486fe5d0b05b3bddb19feb0759193eca86f99e479ddf5248f21daf68ae92c",
            "0xea4a138fdb4e3d5da6c8586ef2cbec28fcfe5eb0d844002242870d0b846ea604",
            "0xfb44abc120fbc6457726f1d0f60449d38fa0ba881fc6a67414ba0c39544b16da",
            "0xcaa3e6c89ba36b34e893f8af85278faec44490b31b3fa800d5206624f0ed31b1",
            "0xe2a8e0e1608f53f15d61e8d452e4f11e3212355a9a72ba0dd0ace1f323d05601",
            "0xc06b283ad920f1ec09172993c90e50e22536d21340f07999f1a61429658d7982",
            "0xe2947593a33f83c2ab6d289598452140e5a733d1f6548f95dcde004f9250c283",
            "0xe08dd39a4f1b32609501e50fb8010de8df2cd525a7b21fb5edee65175ea68af8",
            "0xfa2958e0ab6643986b2600d9c1f61ca94ec88fd5435a37b62a84078af08f295f",
            "0xebf07f5e9dc2d34b8fd65cf61f011db854ca64a2d1868235712fd09fdb0ba1cc",
            "0xe60860a17e3ee5c030f201b1adeef4217aacda44dbe638813ef9e9df735072a9",
            "0xe9477304bbea03f485308238251262e3b1c4e843aeec73a47f02950ebc915680",
            "0xef28afd82717fb34a7e434282b44e8222cd54361ea5dc428dea1480b3cf17c77"
          ]
        },
        "0xc06b283ad920f1ec09172993c90e50e22536d21340f07999f1a61429658d7982": {
          "durationMs": 95,
          "respondedWith": [
            "0xea4a138fdb4e3d5da6c8586ef2cbec28fcfe5eb0d844002242870d0b846ea604",
            "0xef8239a84cd7fef2b2fa981862a83484415fa6682cd048e2f3c6fdc2b16c67a8",
            "0xe3ca97b535caa6323d5135aed79c5c4befb76a3f02757c81820b595b6c8701c2",
            "0xe2947593a33f83c2ab6d289598452140e5a733d1f6548f95dcde004f9250c283",
            "0xe2a8e0e1608f53f15d61e8d452e4f11e3212355a9a72ba0dd0ace1f323d05601",
            "0xe08dd39a4f1b32609501e50fb8010de8df2cd525a7b21fb5edee65175ea68af8",
            "0xfa2958e0ab6643986b2600d9c1f61ca94ec88fd5435a37b62a84078af08f295f"
          ]
        },
        "0xebf07f5e9dc2d34b8fd65cf61f011db854ca64a2d1868235712fd09fdb0ba1cc": {
          "durationMs": 22074,
          "respondedWith": []
        },
        "0xcaa3e6c89ba36b34e893f8af85278faec44490b31b3fa800d5206624f0ed31b1": {
          "durationMs": 69,
          "respondedWith": [
            "0xe9477304bbea03f485308238251262e3b1c4e843aeec73a47f02950ebc915680",
            "0xef28afd82717fb34a7e434282b44e8222cd54361ea5dc428dea1480b3cf17c77",
            "0xef8239a84cd7fef2b2fa981862a83484415fa6682cd048e2f3c6fdc2b16c67a8",
            "0xe2a8e0e1608f53f15d61e8d452e4f11e3212355a9a72ba0dd0ace1f323d05601",
            "0xe08dd39a4f1b32609501e50fb8010de8df2cd525a7b21fb5edee65175ea68af8",
            "0xe60860a17e3ee5c030f201b1adeef4217aacda44dbe638813ef9e9df735072a9",
            "0xfb44abc120fbc6457726f1d0f60449d38fa0ba881fc6a67414ba0c39544b16da"
          ]
        }
      },
      "metadata": {
        "0xe60860a17e3ee5c030f201b1adeef4217aacda44dbe638813ef9e9df735072a9": {
          "enr": "enr:-Jy4QJH3aJho62WToTcmDFNCIa7c2pd5iWEYvJ5fV9qznhkBAfkDyegfi5IaMkFIf6UXJHVd6ImSO26fy-5QHScrJ2YEY5Z0IDAuMS4xLWFscGhhLjEtMmZlMTU2gmlkgnY0gmlwhGj4OA6Jc2VjcDI1NmsxoQLvgmU3KbEJsOG-Zq7mEsAX5HTMpag86Tq7Erbztr0iFIN1ZHCCIyg",
          "distance": "0xd5bc130846557ed2b611227e9cbb4f5658d5a88ccf7341cd0e2e7ae74bc2b47"
        },
        "0xe3ca97b535caa6323d5135aed79c5c4befb76a3f02757c81820b595b6c8701c2": {
          "enr": "enr:-Jy4QNPP9u6uDQGhByFzBKmzOUMq2fbOY_SMHr4Mc0nX6lKHJimWNkkDUbBjs5X1v7FTuJPfH8_pIhwGs2E3yMGC6CkEY5Z0IDAuMS4xLWFscGhhLjEtMmZlMTU2gmlkgnY0gmlwhLKAP4SJc2VjcDI1NmsxoQNm2ssWXNdQUHu2W7wsJYr-qEnXBO2Pq7Wv5DKW-Av-goN1ZHCCIyg",
          "distance": "0x8993624cf91141f26c2263893b91c9ff096eaf31564701c6c10572a6b6b582c"
        },
        "0xef28afd82717fb34a7e434282b44e8222cd54361ea5dc428dea1480b3cf17c77": {
          "enr": "enr:-Ia4QOzeIK9lYLw8fhHC4GVczw4ClakKABzHPiR1SX5Kv5E-bVeaw102ilB2xRQUHwnRvR9fKgUkvGAjvbgLBVAUL4EBY2aCaWSCdjSCaXCEwiEo74lzZWNwMjU2azGhA2IvluKzgI4Yc8Dt1QLGWyeRQD-a7bA6J2uXRAjfTNzgg3VkcIIjpQ",
          "distance": "0x47b0e49dd4c4919bc7727be6f61a8f633f4c3adfd4cc8b530ba467a3b1d2599"
        },
        "0xe0df4ef723b2961c6a35478001f84dfbdf4df053bc5ab3e754881b03e2e10d4a": {
          "enr": "enr:-H64QOLgLHJTEhn_FkivuYUUwWrPFuwpnuDjs4E-KCv1E-XrY0zDTejgbnuMgJZ9cyXU5r1Jmm8qinZS_QTOLJCeVbgBY2aCaWSCdjSJc2VjcDI1NmsxoQPU6osBdjbknktmNkIXtsBku4zjBUYfrPV--xjL6JujPoN1ZHCCIzE",
          "distance": "0xb8cef66d9e9243171a6541645dd0d2fc06c709fab4bbf7aba931572e50d54a4"
        },
        "0xc06b283ad920f1ec09172993c90e50e22536d21340f07999f1a61429658d7982": {
          "enr": "enr:-Ia4QO9Zu8zi0jdS3eFuv7_FD3kBllSRgTdufHzUuxBrEFwtGM-3-1TO9OlYVBcCGgciNrKNPxQ0b1-3y_qLdAmecCIBY2aCaWSCdjSCaXCEwiEo7olzZWNwMjU2azGhA2DTiKsYWl8IgICmFvq_5YPOIYxAaJ7xEs1jiwC5xG19g3VkcIIjqQ",
          "distance": "0x2b3889ab237b43c112843a058d2b10363a1752df57e175041fbd1a586261206c"
        },
        "0xe08dd39a4f1b32609501e50fb8010de8df2cd525a7b21fb5edee65175ea68af8": {
          "enr": "enr:-Jy4QPVQn7tP1bW7Xm5RzZx09-JEHeJPQAeBgRovEECW22lZCJaXn1BRfzM9FlcRPRwYXoI9SbbMk42MaYrNK-mdLLYEY5Z0IDAuMS4xLWFscGhhLjEtMmZlMTU2gmlkgnY0gmlwhJ31OTWJc2VjcDI1NmsxoQMo_DLYhV1nqAVC1ayEIwrhoFCcHvWuhC_J-w-n_4aHP4N1ZHCCIyg",
          "distance": "0xbde720bb540804d8e92f699fc244d3cc00d55e9b0a3132803f56b66594ad316"
        },
        "0xebf07f5e9dc2d34b8fd65cf61f011db854ca64a2d1868235712fd09fdb0ba1cc": {
          "enr": "enr:-Ia4QDSzKbddtdW8pGLZyn1Yw9tGZccl5u7VQXWKWBOaEjd3Q49VYCKkV9udPA8I4j3vikjTaGetZ3s_geRhtT-H0MYBY2aCaWSCdjSCaXCEwiEo7olzZWNwMjU2azGhAsFATTHvg4vRtluT6Flx9RkIokKBgr7eJ0UzIgkOzr6-g3VkcIIjoA",
          "distance": "0xa3decf6799616694454f605b245d6c4bebe46ec6978ea89f34deeedce7f822"
        },
        "0xe2a8e0e1608f53f15d61e8d452e4f11e3212355a9a72ba0dd0ace1f323d05601": {
          "enr": "enr:-Ia4QHpbJ84-XiN640RqkptlxAVBQNIzD9jk9reF35E_gh-kagOy3LhVVj6XuQgx6O509zx2aCMhjrLHljRh-HLWsW4BY2aCaWSCdjSCaXCEwiEo74lzZWNwMjU2azGhAypMdN_yTfjaSAB_v8LhaGHFU2BaHmLjcTfIn5eYZiQKg3VkcIIjoQ",
          "distance": "0x9fb41709ad4e1dc46f2fb4216c1b1ca2d33b5968d63b6903eb7ef82243c0fef"
        },
        "0xef8239a84cd7fef2b2fa981862a83484415fa6682cd048e2f3c6fdc2b16c67a8": {
          "enr": "enr:-Jy4QLUwx_IRaQ7yb6QJhjA9PRDwWDGmzeGq91WLwODLzSk_ZbuKA8EINBXYkGzDuG3PoeKKTPqAeDbiP30PWg2PrpoEY5Z0IDAuMS4xLWFscGhhLjEtMmZlMTU2gmlkgnY0gmlwhK6KCkGJc2VjcDI1NmsxoQOLjSawYrDoCh5zDJSsqfIM-AoPVRvsBoXghvcjWluMLYN1ZHCCIyg",
          "distance": "0x4d19839b68c4cdfa9698b8e268d74505e7e26a43bc1447f1dddf3b3b6803e46"
        },
        "0xc90486fe5d0b05b3bddb19feb0759193eca86f99e479ddf5248f21daf68ae92c": {
          "enr": "enr:-Ia4QLzwXPohhSBPzk-2FAVk-PKx2dnGI-FUlol-4NXOcbcLQSq1Ajl_m-ByXtgUfwd9f-HxPpIt6BbtdwSs0BJ4eUcBY2aCaWSCdjSCaXCEwiEo7olzZWNwMjU2azGhAv9xe4DRrJqclwBqBYBp8D-n0W9wbyiI725b2BfvTpdpg3VkcIIjpQ",
          "distance": "0x2257276fa750b79ea6480a68f450d147f389ef55f368d168ca942fabf166b0c2"
        },
        "0xea4a138fdb4e3d5da6c8586ef2cbec28fcfe5eb0d844002242870d0b846ea604": {
          "enr": "enr:-Jy4QOekx02vPpSoVuL-8AQCJwX5sH7XiuV5B6eQRhkBueINHNKqQ-K9XxrG22nj2bs1gF9SdiO3chf6gbXBUL_ILwAEY5Z0IDAuMS4xLWFscGhhLjEtMmZlMTU2gmlkgnY0gmlwhI_0qIWJc2VjcDI1NmsxoQMlRVStHXnLaYvWnsMKKViBUp2XqwtSYy2sjSecFpI-GYN1ZHCCIyg",
          "distance": "0x119b21e21158f70bd5b4bf8b6eeacfce3dfde7ccf550cbfac9c037a8382ffea"
        },
        "0xfa2958e0ab6643986b2600d9c1f61ca94ec88fd5435a37b62a84078af08f295f": {
          "enr": "enr:-Jy4QJwq_0vBEo9QU2IFyUfcqf7tLmNtW-TBOZVgGZPtGxdmb8PhLd7J2LE24sHH2sirklwRxut56Wd9G4S5degZoJEBY5Z0IDAuMS4xLWFscGhhLjEtNDg4MzAwgmlkgnY0gmlwhEFtPv2Jc2VjcDI1NmsxoQJ8uh7r4PaIO39O0TJPk7yhUy-eMlcgsOrGjbSqiObkeYN1ZHCCIzQ",
          "distance": "0x117af971513df1b570b5134f85d35c7d51e90f19544b3b2bc49f09fbf76370b1"
        },
        "0xcaa3e6c89ba36b34e893f8af85278faec44490b31b3fa800d5206624f0ed31b1": {
          "enr": "enr:-Jy4QNLwwyac00IVlAy_3acpHEfHCYd60GeVn7tRr-zDq8UsDHLdLr5sYaPFbh-lClSEEaPsdeOO40zlYe0hrBhpXsIEY5Z0IDAuMS4xLWFscGhhLjEtMmZlMTU2gmlkgnY0gmlwhLKA9suJc2VjcDI1NmsxoQJlES6pw5P2DbCX3qeY5lyoyEyf5e2FnIQUexSlLbJNH4N1ZHCCIyg",
          "distance": "0x21f0475961f8d919f300eb39c102cf7adb65107f0c2ea49d3b3b6855f701685f"
        },
        "0xced7d0a627c603fb55b965bd50691d23e7f78315105ed8f16c5a3de2b8566951": {
          "enr": "enr:-Jy4QNm_fikSgRfbymTfR4h9r16a2HbVBpHr0GdfCZxJMIkabJWhb3lIiKKSiyMWAJDvgtabL6_5w_WTFW3YlS6RzWQEY5Z0IDAuMS4xLWFscGhhLjEtMmZlMTU2gmlkgnY0gmlwhLymbAiJc2VjcDI1NmsxoQINfaprZwxFpPNebhmPGgbfsoMqoDOw0RSugG3ZNBuJQIN1ZHCCIyg",
          "distance": "0x25847137dd9db1d64e2a762b144c5df7f8d603d9074fd46c82413393bfba30bf"
        },
        "0xec9b86d41a6e4aa9be617fe6a4d10b2d45cbc37ba8884ccfafbafbccd20ac6e9": {
          "enr": "enr:-Jy4QNq4an-eLbsT1_YCFNbFhWhSCr_9VPFGx5vu7OCx45ShLNnLl_iyjjqFkSm_aJUdTo_UN0uFNCK2le6VUGN3Vo0EY5Z0IDAuMS4xLWFscGhhLjEtMmZlMTU2gmlkgnY0gmlwhJ_fk76Jc2VjcDI1NmsxoQMm7bz3DNRHJawIBejJ-EoNT7IqBAnfKXlbA-NpV5pf6YN1ZHCCIyg",
          "distance": "0x7c82745e035f884a5f26c70e0f44bf95aea43b7bf99405241a1f5bdd5e69f07"
        },
        "0xfb44abc120fbc6457726f1d0f60449d38fa0ba881fc6a67414ba0c39544b16da": {
          "enr": "enr:-Jy4QHt0rEYK7rJMkvwscneV1Cu_FOxkvhpMxGwFPiYqJokfREOZosDazswRM7UuwlcRocDHpx0KTxrcgXmW7M3M9SkEY5Z0IDAuMS4xLWFscGhhLjEtMmZlMTU2gmlkgnY0gmlwhJ_f0nSJc2VjcDI1NmsxoQIIWB3Z7_hmKaJdb9Swqz9cgC0qtKnHhENlPsi-X9rU0YN1ZHCCIyg",
          "distance": "0x10170a50daa074686cb5e246b221090790813a4408d7aae9faa1024853a74f34"
        },
        "0xe2947593a33f83c2ab6d289598452140e5a733d1f6548f95dcde004f9250c283": {
          "enr": "enr:-Jy4QNm1KTH3LdJGFYn7Fk69rhVCU-Rt2gpD25XiXoywMwIOHBrmaFYz9e6aj-m3_AWE6_NjFgkJwsprK46buKaTL40EY5Z0IDAuMS4xLWFscGhhLjEtMmZlMTU2gmlkgnY0gmlwhJ_f5XyJc2VjcDI1NmsxoQJvm4dI5P4rUq4vJLkTM1de01wxoRpdVR_7nBdav5h4OoN1ZHCCIyg",
          "distance": "0x9c7d402596431efb0fe3b03dc606194fa86b31de145830832c50e3e95bc9b6d"
        },
        "0xe9477304bbea03f485308238251262e3b1c4e843aeec73a47f02950ebc915680": {
          "enr": "enr:-Jy4QIoqrAfisFOq_-JJKUneDCbokXCQZ3yiEYzHihrfs4B_On3UjDqx9hT0r5UxJ8gdj0pFUSsGtwhFDaunhR1aUwQBY5Z0IDAuMS4xLWFscGhhLjEtNDg4MzAwgmlkgnY0gmlwhEFtPv2Jc2VjcDI1NmsxoQLNIITkWLfvhGfD30MpqqI-UFVUmuIZLYivgx1_f2wif4N1ZHCCIyw",
          "distance": "0x214d29541b1b1d99ea391ae61372237aee5688fb9fd7f3991199b7fbb7d0f6e"
        }
      },
      "cancelled": [
        "0xe3ca97b535caa6323d5135aed79c5c4befb76a3f02757c81820b595b6c8701c2",
        "0xe60860a17e3ee5c030f201b1adeef4217aacda44dbe638813ef9e9df735072a9"
      ],
      "startedAtMs": 202755541
    }
  }
}

origin - originator of the request i.e. local node. targetId - target content ID receivedFrom - node the data was received from responses - map from the nodes which don't have content to the nodes that probably have this content or closer to it in the respondedWith field. Except for the node that has the content, this node's respondedWith must be empty list. durationMs is the time from the beginning of the original request up to receiving data from this node. metadata - map from the node id to metadata, enr and distance. metadata must include all the nodes mentioned in any other fields in the response. cancelled - list of requests that were cancelled before receiving data. Optional.

Here's glados update to support trace request in this current form: https://github.com/ethereum/glados/pull/177

UPD: I've updated this post to reflect latest changes

mynameisdaniil commented 1 year ago

@njgheorghita can you also take a look at glados PR? (https://github.com/ethereum/glados/pull/177) I can't add anyone to the list of reviewers there. Also, this change will require some changes in trin. How can we arrange that? I have an issue opened in trin repo for that (https://github.com/ethereum/trin/issues/987).

njgheorghita commented 1 year ago

@mynameisdaniil Well, I'd say it looks like the glados pr is blocked until we get the changes through on trin, I'll pick that issue (987) up and once it gets merged / included in the next release then we'll move on to getting the glados pr through.

njgheorghita commented 1 year ago

@mynameisdaniil In the specs you have the content field name as "contentValue" but in the example list in the pr description you have just "content". Also in the example, it looks like it should be "content": "0x" not "content": "". Just wondering since you say "Here's an example of the correct JSON response:". Personally, I'd prefer to see the spec updated to just "content" rather than "contentValue" - but just wondering what your thoughts about it are? How did you generate the "correct JSON response"?

kdeme commented 1 year ago

Personally, I'd prefer to see the spec updated to just "content" rather than "contentValue" - but just wondering what your thoughts about it are? How did you generate the "correct JSON response"?

I have no preference between contentValue vs content. My only request is that it should be consistent for both portal_historyTraceRecursiveFindContent and portal_historyRecursiveFindContent.

njgheorghita commented 1 year ago

My only request is that it should be consistent for both portal_historyTraceRecursiveFindContent and portal_historyRecursiveFindContent.

Yeah, agreed. I only just now noticed that the spec for portal_historyRecursiveFindContent uses "contentValue". In trin we use "content" for that endpoint, and it looks to me like fluffy does as well? As well as ultralight. In which case I'd recommend changing "contentValue" -> "content" in this pr for both portal_historyRecursiveFindContent & portal_historyTraceRecursiveFindContent

njgheorghita commented 1 year ago

Also worth mentioning here (from https://github.com/ethereum/trin/issues/987#issuecomment-1766838403) that we should leave in the startedAt field inside the metadata since there are plans to use that field in glados eventually. Sorry for all the change requests post-approval :sweat_smile:

mrferris commented 1 year ago

Looks good, only thing is that I agree that startedAt should be added back in because while the caller knows when they sent the jsonrpc command, they don't know that the query started at exactly that moment and that it wasn't queued up to be sent some time later.

mynameisdaniil commented 1 year ago

@mrferris Agree, but I propose we make it just integer in milliseconds, so it will fit into puny 53 bit of js integers. And I don't think we really need nanoseconds precision anyway.

mynameisdaniil commented 1 year ago

@njgheorghita yeah, I was looking into the spec and not into the code, that's why it was 'contentValue'. I made it just content now, and also changed for LocalContentResult (https://github.com/ethereum/portal-network-specs/pull/236/files#diff-c2dedab1970d2849f29aa317f197e32966e0aaa38fab13e02dd24b62cb0f8482L186) which is not implemented according to spec in fluffy (we return just a string https://github.com/status-im/nimbus-eth1/blob/master/fluffy/rpc/rpc_portal_api.nim#L188C37-L188C37) and I guess the situation is the same for other clients. I was going to quickly create issues for other clients to fix this. But I now see that it isn't trivial and I will need some time to figure this out. There is a mess of inconsistencies around returning content.

kdeme commented 1 year ago

I guess the situation is the same for other clients. I was going to quickly create issues for other clients to fix this. But I now see that it isn't trivial and I will need some time to figure this out. There is a mess of inconsistencies around returning content.

In this case, I think it makes more sense to just adjust the specs for portal_*LocalContent to not return an object but just the content in hexstring. IIRC this is already used in Hive and works for the clients there, you might want to verify that though.

The contentValue -> content change must be made consistent for portal_historyRecursiveFindContent though.

mynameisdaniil commented 1 year ago

@njgheorghita

Also in the example, it looks like it should be "content": "0x" not "content": "".

That's because I just di" content in vim. Of course it should be 0x...

How did you generate the "correct JSON response"?

With the current implementation in Fluffy. I did spec and code update at the same time, trying to keep it consistent.

njgheorghita commented 1 year ago

@mynameisdaniil Thanks for making those changes! Everything looks good to me

mynameisdaniil commented 1 year ago

@mrferris what do you think about https://github.com/ethereum/portal-network-specs/pull/236/files#diff-7c7eba9e61ee0bdf859040ec9b558b0d81838316425ec96ee231b6b06bf5c204R11 ? I feel like it's better to keep all time measurements in same unit i.e. milliseconds. If we need nanosecond precision, let's make it seconds + nanoseconds remainder everywhere i.e. for both startedAt and duration.