Don't wait for a full batch before returning DSN mappings

[teor2345]

StreamExt::chunks waits for 10,000 mappings before sending any mappings. It only sends partial chunks when the stream ends. This makes testing tricky, and could result in lost mappings in production.

So we want StreamExt::ready_chunks, which batches into groups of 10,000, but sends the remainder as soon as the stream is pending.

I have tested this PR using the code in fake-map-testing, and it produces the default mapping immediately, as expected:

$ ./subspace-node run --dev --pool-kbytes 200000
$ ./subspace-farmer farm --dev --reward-address ... path=$(mktemp -d),size=100Gi,record-chunks-mode=ConcurrentChunks
$ websocat --jsonrpc ws://127.0.0.1:9944 | tee -a mappings.txt
subspace_subscribeArchivedObjectMappings
{"jsonrpc":"2.0","result":"8t473lH1zriHDOoT","id":1}
{"jsonrpc":"2.0","method":"subspace_archived_object_mappings","params":{"subscription":"8t473lH1zriHDOoT","result":{"v0":{"objects":[{"hash":"0000000000000000000000000000000000000000000000000000000000000000","pieceIndex":0,"offset":0}]}}}}

Code contributor checklist:

• [x] I have read, understood and followed contributing guide

[teor2345]

My testing branch is also producing real mappings with this fix (formatted using jq .):

{
  "jsonrpc": "2.0",
  "method": "subspace_archived_object_mappings",
  "params": {
    "subscription": "FIdyZC3akrmN2dzj",
    "result": {
      "v0": {
        "objects": [
          {
            "hash": "71b278f3dc434447fc620500e47b6a80b0cb0df76a1051119fe19ed4953242df",
            "pieceIndex": 256,
            "offset": 1089
          },
          {
            "hash": "71b278f3dc434447fc620500e47b6a80b0cb0df76a1051119fe19ed4953242df",
            "pieceIndex": 384,
            "offset": 1077
          },
          {
            "hash": "a05e75a8c313f37f4da2a6d4d82d06a55fe2b8c6167c70ad858e9ad8ffc47ccf",
            "pieceIndex": 400,
            "offset": 263241
          },
          {
            "hash": "e476f1b379438de7a1acfd567a94a8c53f08b9714042f7f17e5791645afc3176",
            "pieceIndex": 404,
            "offset": 328782
          },
          {
            "hash": "b810a70051c31bf539766c0c5afbee6d0925d0997b34ca9009c76f910f22bb03",
            "pieceIndex": 408,
            "offset": 394323
          },
          {
            "hash": "5e08206eec0129e54bb78effc3af9bdfa02f4e794eb537f155ec96683bf31d65",
            "pieceIndex": 424,
            "offset": 656487
          },
          {
            "hash": "cd52534deb84680e8f65e19336ea173a9df54ab37f42db928dadaa67e85fa17b",
            "pieceIndex": 428,
            "offset": 722030
          },
          {
            "hash": "cd52534deb84680e8f65e19336ea173a9df54ab37f42db928dadaa67e85fa17b",
            "pieceIndex": 440,
            "offset": 918653
          },
          {
            "hash": "5141aeb0dad82969c8335dff7b6b4ab53d986a26e27dbe58ca7d4a133d5f0512",
            "pieceIndex": 450,
            "offset": 33927
          }
        ]
      }
    }
  }
}
{
  "jsonrpc": "2.0",
  "method": "subspace_archived_object_mappings",
  "params": {
    "subscription": "FIdyZC3akrmN2dzj",
    "result": {
      "v0": {
        "objects": [
          {
            "hash": "a9990109ad6646d0e3695ead1acea059d19e13d11f299266f81abd15f199006c",
            "pieceIndex": 532,
            "offset": 329208
          }
        ]
      }
    }
  }
}

[teor2345]

Failed in the merge queue with:

2024-09-02 05:00:44 [//Charlie (DomainId(0))] 🔖 Producing singleton receipt at slot Slot(35) domain_best_number_onchain=23 head_receipt_number=21
thread 'tests::test_bad_receipt_chain' panicked at domains/client/domain-operator/src/tests.rs:4586:9:
assertion failed: !ferdie.does_receipt_exist(receipt_hash).unwrap()

https://github.com/autonomys/subspace/actions/runs/10659868548/job/29543036746

Seems unrelated so I'm going to re-queue it.

[nazar-pc]

Seems unrelated so I'm going to re-queue it.

Please report such things in engineering channel

[teor2345]

Seems unrelated so I'm going to re-queue it.

Please report such things in engineering channel

I'd already posted it in the engineering channel before you replied 🙂