Support for /block endpoints

[aphelionz]

References:

• https://github.com/ipfs/js-ipfs/tree/master/packages/interface-ipfs-core/src/block
• https://docs-beta.ipfs.io/reference/http/api/
• https://github.com/ipfs/interop (no occurrences)

/block/put

• Unless specified, this command returns dag-pb CIDv0 CIDs.
• Arguments
  • arg [file]: The data to be stored as an IPFS block. Required: yes.
  • format [string]: cid format for blocks to be created with. Required: no. Note: raw is the only format used in tests
  • mhtype [string]: multihash hash function. Default: "sha2-256". Required: no.
    • Setting 'mhtype' to anything other than 'sha2-256' or format to anything other than 'v0' will result in CIDv1.
  • mhlen [int]: multihash hash length. Default: "-1". Required: no.
  • pin [bool]: pin added blocks recursively. Default: "false". Required: no.
• Request Body is Argument "data" is of file type. This endpoint expects a file in the body of the request as 'multipart/form-data'.
• On success, the call to this endpoint will return with 200 and the following body:

{
  "Key": "<string>",
  "Size": "<int>"
}

• curl -F file=@myfile "http://localhost:5001/api/v0/block/put?format=&mhtype=sha2-256&mhlen=-1&pin=false"
• should error with array of blocks

/block/rm

• requires dag.put (could refactor the tests to block.put)
• requires refs.local
• does NOT error with array of cids
• errors when removing nonexistant blocks, except with force
• Arguments
  • arg [string]: Bash58 encoded multihash of block(s) to remove. Required: yes.
  • force [bool]: Ignore nonexistent blocks. Required: no.
  • quiet [bool]: Write minimal output. Required: no.

    Response

• will fail on this test
• On success, the call to this endpoint will return with 200 and the following body:

{
  "Error": "<string>",
  "Hash": "<string>"
}

• cURL Example curl "http://localhost:5001/api/v0/block/rm?arg=&force=&quiet="

/block/get

• requires block.put first
• block.get arguments: cid of block (/api/v0/block/?arg=[cid])
• As discussed in other issues, referencing block by cid requires overhead such as versioning e.g. https://github.com/ipfs/js-ipfs/blob/master/packages/interface-ipfs-core/src/block/get.js#L64
• Blocks added as CIDv1 can be retrieved as CIDv0. Question: does this generalize to any CID version? Probably.
• HTTP call responds with 200 and plain/text response

/block/stat

Print information of a raw IPFS block.

• Requires block.put
• Stats by CID
• Arguments
  • arg [string]: The base58 multihash of an existing block to stat. Required: yes.
• On success, the call to this endpoint will return with 200 and the following body:

{
  "Key": "<string>",
  "Size": "<int>"
}

• cURL Example: curl "http://localhost:5001/api/v0/block/stat?arg="

[koivunej]

The whole example request as made by the conformance tests:

POST /api/v0/block/put HTTP/1.1
accept: application/json
Content-Type: multipart/form-data;boundary=--------------------------850260422399253935824202
Content-Length: 204
User-Agent: node-fetch/1.0 (+https://github.com/bitinn/node-fetch)
Accept-Encoding: gzip,deflate
Connection: close
Host: 127.0.0.1:34523

----------------------------850260422399253935824202
Content-Disposition: form-data; name="file"
Content-Type: application/octet-stream

blorb
----------------------------850260422399253935824202--

And the same interaction through go-ipfs 0.4.23 as in echo -n blorb | ipfs block put:

POST /api/v0/block/put?encoding=json&mhlen=-1&mhtype=sha2-256&pin=false&stream-channels=true HTTP/1.1
Host: 127.0.0.1:5001
User-Agent: go-ipfs-cmds/http
Connection: close
Transfer-Encoding: chunked
Content-Type: multipart/form-data; boundary=bcea56e168b34c6f47fcd98852ccfbc4a207ae23191ddd0d925f69b6e5eb
Accept-Encoding: gzip

--bcea56e168b34c6f47fcd98852ccfbc4a207ae23191ddd0d925f69b6e5eb
Abspath: /dev/stdin
Content-Disposition: form-data; name="file"; filename=""
Content-Type: application/octet-stream

blorb
--bcea56e168b34c6f47fcd98852ccfbc4a207ae23191ddd0d925f69b6e5eb--
HTTP/1.1 200 OK
Access-Control-Allow-Headers: X-Stream-Output, X-Chunked-Output, X-Content-Length
Access-Control-Expose-Headers: X-Stream-Output, X-Chunked-Output, X-Content-Length
Content-Type: application/json
Server: go-ipfs/0.4.23
Trailer: X-Stream-Error
Vary: Origin
Date: Fri, 27 Mar 2020 13:38:06 GMT
Connection: close
Transfer-Encoding: chunked

{"Key":"QmPv52ekjS75L4JmHpXVeuJ5uX2ecSfSZo88NSyxwA3rAQ","Size":5}

It's nice that the js http client sends the Content-Length so we can at least start from that and maybe later run into issues with that requirement. The CID returned from go-ipfs is the same as expected in the test.

[dvc94ch]

wantlist depends on get, rm depends on dag + pin.

[dvc94ch]

pin depends on add which depends on a bunch of other stuff I assume, dag tests use add also

[koivunej]

dvc94ch could you explain the above a bit?

wantlist depends on get, rm depends on dag + pin

Aah, I see the comma now. I spent a lot of time thinking how could wantlist depend on rm + dag + pin.

pin depends on add which depends on a bunch of other stuff I assume, dag tests use add also

I think the pinning under the hood is good enough at the moment with #117. Even if an API is used we don't need to implement it to the fullest by the tests right now.

I drew a confusing graphviz on the dependencies I gathered. It will not make anyone any wiser, but in addition to some details I missed I don't think I found anything conflicting. Still wondering if that /object/put can easily be replaced with /block/put ... Not sure.

[koivunej]

From the http request examples it would seem like the javascript version is ok but the Abspath header from go-ipfs does not seem to be allowed by RFC7578 nor supported by multipart::server::FieldHeaders which is the implementation used in warp::filters::multipart.

Strange that the docs put the file as named arg but the requests show that it is in fact called file and that is the only required field.

[koivunej]

With go-ipfs 0.4.23 the format argument is interesting as well.

For value of v0: the other parameter mhtype will be required to be sha2-256 while the help with arguments --mhtype sha3-256 suggests at sha2-255-32:

$ ./ipfs block put --mhtype sha3-256 --format v0 empty
Error: only sha2-255-32 is allowed with CIDv0
$ ./ipfs block put --mhtype sha2-255-32 --format v0 empty
Error: unrecognized multihash function: sha2-255-32
$ ./ipfs block put --mhtype sha2-256 --format v0 empty
QmdfTbBqBPQ7VNxZEYEj14VmRuZBkqFbiwReogJgS1zR1n

Can't find another accepted value for the parameter:

$ ./ipfs block put --mhtype sha3-256 --format v1 empty
Error: unrecognized format: v1

[koivunej]

131 leaves the following tests failing:

• .block.get "should get a block added as CIDv0 with a CIDv1"
• .block.get "should get a block added as CIDv1 with a CIDv0"
• .block.rm (wasn't included)
• .block.stat (doesn't do anything(?))

[aphelionz]

What is involved in transliterating CIDv0 <---> CIDv1? Is that the right term, transliterating?

Also what do you mean by "doesn't do anything?" it looks like it kinda does something here https://github.com/ipfs/js-ipfs/blob/master/packages/interface-ipfs-core/src/block/stat.js#L28

[koivunej]

Yeah the "doesn't do anything" was badly written. I was just wondering what was I meaning. It was that there is only boilerplate in place: https://github.com/ipfs-rust/rust-ipfs/blob/master/http/src/v0/block.rs#L137-L144 and this isn't even routed yet. Same as wasn't included, also badly written.

CIDv0 <---> CIDv1

I was thinking if this should be "always use cidv1 internally" but there are probably some pitfalls I didn't yet check from the other impls.

[aphelionz]

block.stat was implemented fully in #137