Integration with clib

[stephenmathieson]

I've been working on a similar concept, called clib. Thoughts on working together and possibly allowing ccan modules to be installed with clib(1)?

[rustyrussell]

Sounds like a great idea! What's the best way to do the integration?

[stephenmathieson]

For now, we'd have to split the ccan repo into individual parts. Each lib would need its own repo and to specify its own package.json file (including its source/header files and dependencies).

In time, support could be added for fetching packages listed within a "main" repository. Something like: clib install rustyrussell/ccan/ccan/daemonize, but that feels kinda awkward :/

Thoughts @jwerle @visionmedia

[jb55]

I think a script that imports ccan libs into github and tests them would be ideal here. No need to introduce complexity to support ccan. Support for arbitrary remotes with a ccan proxy is another possibility.

[jwerle]

@jb55 A lot of ccan modules make references to other ccan modules in the code.. :( See https://github.com/rustyrussell/ccan/blob/master/ccan/crc/crc.c#L27

[jb55]

This is kind of the same discussions component/component went through when people wanted to support bower packages. It didn't make sense to support every other ecosystem without turning your own into a mudball. The solution was, if you wanted a package from bower/ccan/etc you could simply fork/clone it and add a package.json, fixing any include refs along the way.

An import script to do this automatically would be handy as well.

[jwerle]

I agree ! I find this to be annoying as I'd like to be able to use packages from anywhere. It would be nice if we could preserve directory structure if the user wanted. cc @stephenmathieson thoughts ?

[stephenmathieson]

I like the import script idea, but it doesn't solve keeping the repos up-to-date with the upstream repository (e.g. this one).

This likely isn't a huge deal, as we're already doing it a few places (linenoise, inih, etc.).

[rustyrussell]

Stephen Mathieson notifications@github.com writes:

I've been working on a similar concept, called clib. Thoughts on working together and possibly allowing ccan modules to be installed with clib(1)?

So, I really like the clib concept.

In many ways, CCAN modules are a subset of what clib allows (eg. we insist on certain filenames), so clib-izing should be quite automatable.

Having a way to pull part of a git tree would help (a-la git subtree?).

Writing a ccantool --clib which spits out the package.json would be pretty straightforward, with caveats: in particular, _info is a full C program so it can detect dependencies based on preprocessor defs, runtime tests, etc. Since most _info don't do this, we can ignore it for the first cut at least.

Here's a horrible shell script which demonstrates the idea (is there a JSON grammar for the package.json file somewhere?):

! /bin/sh -e

eg /home/rusty/devel/cvs/ccan/ccan/foo/bar => /home/rusty/devel/cvs/ccan

find_ccandir() { if [ x"$(basename "$1")" != xccan ]; then if [ $1 = '/' ]; then echo "Could not determine ccan directory" >&2 exit 1 fi find_ccandir "$(dirname "$1")" else dirname "$1" fi }

Simplistic...

json_escape() {

dash's echo respects escape sequences... Boo!

/bin/echo "$@" | sed -e 's/"/\"/g' -e 's/\/\\/g'

}

Eg:

{

"name": "mon",

"version": "1.1.1",

"repo": "visionmedia/mon",

"description": "Simple process monitoring",

"keywords": ["process", "monitoring", "monitor", "availability"],

"license": "MIT",

"install": "make install"

}

if [ $# = 1 ]; then cd $1 elif [ $# != 0 ]; then echo "Usage: $0 [moduledir]">&2 exit 1 fi

MODDIR="$(pwd)" CCANDIR="${CCANDIR:-$(find_ccandir "$MODDIR")}"

To get doc_extract

PATH="$CCANDIR/tools:$PATH" export PATH

if [ ! -f _info ]; then echo "$MODDIR does not have _info: not a ccan module?" >&2 exit 1 fi

CCAN modules can be nested.

FULLNAME=$(json_escape "${MODDIR##*ccan/}") NAME=$(json_escape "$(basename "$FULLNAME")") VERSION=$(json_escape "$(doc_extract version _info)") DESCRIPTION=$(json_escape "$(doc_extract summary _info)") KEYWORDS=$(json_escape "$(doc_extract keywords _info)") LICENSE=$(json_escape "$(doc_extract license _info)")

echo '{' echo ' "name": "'"$NAME"'",' echo ' "repo": "ccan/'"$FULLNAME"'",' echo ' "description": "'"$DESCRIPTION"'",' [ -z "$VERSION" ] || echo ' "version": "'"$VERSION"'",' [ -z "$KEYWORDS" ] || echo ' "keywords": "'"$KEYWORDS"'",' [ -z "$LICENSE" ] || echo ' "license": "'"$LICENSE"'",' echo '}'

[dumblob]

@stephenmathieson any ideas about the solution @rustyrussell proposed?

[willemt]

@dumblob we created a conversion script here, just need to automate the creation of Github repos: https://github.com/clibs/clib/pull/128