Open matthewhammer opened 3 years ago
One may wonder:
Why does Candid Spaces convert Candid data into an AST for the
put
andgetImage
APIs?If we want to transfer arbitrary data, we can just send blob over the wire, and let the canister decode to its proper type?
These are valid and natural questions.
We could write a Candid serialization/deserialization implementation in Motoko, that converts from Blob into Candid AST and back. This route seems even better (more useful) but also seemed like more work, in the short term, than insisting that the user do the parsing into and from an AST.
Then, I thought, perhaps the local Candid tools could do that extra step, since the canister may want to accept and store the AST directly (like in my case, here).
We could write a Candid serialization/deserialization implementation in Motoko, that converts from Blob into Candid AST and back. This route seems even better (more useful) but also seemed like more work, in the short term, than insisting that the user do the parsing into and from an AST.
This is actually the "shortcut", see https://github.com/dfinity/motoko/blob/master/test/run/idl.mo. The real solution is generics in Candid :)
dynamic
" in our meetings, AFAIK. In particular, for candid spaces to accept any candid value stream but also see its structure (for queries, etc.), I want to inspect each Blob
-based value without knowing its type, a priori, like in a dynamically-typed language. I think that idl primitive stuff is not exposing enough type information, and perhaps there just isn't enough in Candid, since we only include the hashes of fields.for candid spaces to accept any candid value stream but also see its structure (for queries, etc.), I want to inspect each Blob-based value without knowing its type
I don't know if that's possible in a strongly typed language like Motoko. As you mentioned, we need the type information as an input to deserialization. The deserialization expects two types: the wire type (comes from the message) and the expected type. With only the wire type, it's like using the candid value untyped.
Actually, when you receive the message, the sender should know the candid type. You can ask the sender to get the type by calling the candid_tmp_hack
. I think your use case is similar to Paul's qr scanner: https://github.com/ninegua/ic-qr-scanner
I think for now, the easiest thing is to finish the tool that I am creating to send the AST over the wire, with full field names.
See #10 for a draft PR.
I realize that the wire format is inefficient, but that's not the main issue right now. Later, Candid Spaces can offer a more efficient put
operation that accepts the (ordinary candid) Blob
format whose decoding we are debating now. Somehow, we need to send the type with the Blob
, or get it from somewhere. I don't think it's generally the case that every candid value has a service or a type to go along with it, but I see that in many cases, they might. In the short term, #10 plans to reuse the functionality of parsing and type-checking from the candid crate, and then add a minimal layer to send the parsed AST over the wire. Almost there now.
I finished and merged #10
It creates a simple tool caniput
using the existing candid crate. Its chief purpose is to put data into candid spaces, and provide the text/binary parsing that I don't want to implement in Motoko, at least not yet. For now, the tool permits me to reuse that functionality of the candid
crate, and do it "client side" rather than "server side" (for now).
Now, with a bit more polish, caniput
can transmit files and binary data too. (Not yet in #10)
Related: https://github.com/dfinity/candid/issues/245
I still don't quite see the exact requirement here. Does one of the ideas listed above fits your need? Feel free to add more ideas there.
Simple CLI tool for importing and exporting data from a Candid Spaces canister.
The Rust crate may provide almost enough.
But, we need to "re-encode" and "re-decode" the Candid data as a Candid AST, as the API accepts and emits these ASTs, as to be generic and multi-purpose.
That is to say,
for importing into the canister with the tool (from a local machine), we want to read ordinary Candid data as human readable text or in serialized binary form, and then represent and import this data as Candid value AST, sent to the canister (see
put
).for exporting, we want to reverse the process, converting a Candid value AST into either ordinary text or serialized binary form (see
getFullImage
andgetSubImage
, which respond with the same data types accepted byput
).Perhaps this is best done as a feature added to either the Candid tools, or
ic-repl
, since those are the tools that already exist?Thoughts @chenyan-dfinity?