jbenet / transformer

transformer - multiformat data conversion
transform.datadex.io
130 stars 7 forks source link

`package.json` transformer type #9

Closed jbenet closed 10 years ago

jbenet commented 10 years ago

Transformer modules need a signal in package.json. Currently leaning on using either:

Example for types:

{
   ...
   "transformer": {
      "id": "ip-address",
      "type": "Type"
   }
   ...
}

Example for conversion:

{
   ...
   "transformer": {
      "id": "ip-address-to-buffer",
      "type": "Conversion"
   }
   ...
}
jbenet commented 10 years ago

Could also embed the whole transformer description jsonld doc:

Example for types:

{
   ...
   "transformer": {
     "@context": "http://transformer.io/context/transformer.jsonld",
     "type": "Type",
     "id": "ip-address",
     "schema": "ip-address"
   },
   ...
}

Example for conversion:

{
   ...
   "transformer": {
     "@context": "http://transformer.io/context/transformer.jsonld",
     "type": "Conversion",
     "input": "ip-address",
     "output": "buffer",
     "id": "ip-address-to-buffer",
     "description": "IP Address to Buffer conversion."
   },
   ...
}

This gets annoying to type by hand, given the size. This would be easier with our own wrapper of npm init and npm publish that automatically keeps all this information up to date. E.g.

> transformer init ip-address
... prompts ...
> ls
index.js                <--- has code skeleton
package.json            <--- filled out with relevant transformer stuff

The plan is to also run a service (at transformer.io or something) that gives all this info:

> curl transformer.io/api/v0/ip-address?pretty=true
{
  "@context": "http://transformer.io/context/transformer.jsonld",
  "type": "Type",
  "id": "ip-address",
  "schema": "ip-address"
}

Btw, the transformer description src is available on any type in js:

var transformer = require('dat-tranformer');
var ipaddr = transformer('ip-address');
console.log(ipaddr.src);

and cli:

transform --src ip-address
jbenet commented 10 years ago

Hm, another approach is for the description to be generated as a separate file:

> cat transformer.jsonld
{
  "@context": "http://transformer.io/context/transformer.jsonld",
  "type": "Type",
  "id": "ip-address",
  "schema": "ip-address"
}

And the npm package.json references it:

{
   ...
   "transformer": "./transformer.jsonld",
   ...
}

Pros:

Cons:

Thoughts on this @maxogden ? If we had to have the transformer json description either embedded whole into the package.json or as a separate file, what would you prefer? (maybe allow both?)

jbenet commented 10 years ago

For now, using:

// in package.json
{
  "transformer": "transformer.jsonld",
  "keywords": [
    "transformer",        // all must
    "transformer-type" // type must
    "transformer-conversion" // conversion must
  ]
}

And transformer.jsonld has:

{
 "@context": "http://transformer.io/context/transformer.jsonld",
 "type": "type",
 "id": "ascii",
 "description": "ascii strings.",
 "schema": "string"
}

Which will be useful to resolve conversions from npm data directly. (Probably using https://github.com/mcollina/levelgraph or something)