NodeBase Block

[mikeal]

Being that talks are only 20 minutes it would be a good idea to get an understanding of what each speaker is talking about so that we can order the talks in such a way that we can limit overlap and let the talks build on each other a bit.

@rvagg @dominictarr @hij1nx what are you thinking?

[mikeal]

@hij1nx your working title is hij1nx talks in italics about database shit :P

[mikeal]

adding @juliangruber

[mikeal]

@hij1nx @juliangruber I think it would be good to figure out what underlying modules you're taking for granted so that @rvagg and @dominictarr might cover them in their talks and lay a good foundation for yours.

[rvagg]

I was thinking about introducing this https://github.com/rvagg/lmdb/tree/lmdb and perhaps talking a bit about storage primitives. Not sure, 20 mins is tight so it'd need to be focused.

@mikeal what are your thoughts on the level of technical detail needed here? Are we getting close to any kind of saturation point in the community where people are starting to get a handle on this level* stuff and how leveldb works or should we be covering some of the basics?

[mikeal]

I think that the block should start with a doc that gives a good overview stack and community. That said, the talk doesn't need to be about that, it should just cover it.

@rvagg if you could construct a talk that was more like a story "I wrote levelup, we broke it out in to these smaller components and then people wrote alternate implementations below and a huge ecosystem on top," similar to what you did at that meetup in SF we were at but spend way less time explaining the ins and outs of each library, just cover where they fit in to the ecosystem. that could be done in about 10 minutes if you really ratchet it down, then you could spend the next 10 minutes talking about lmdb.

it could only work if you really didn't get in to any implementation details in leveldb or any of the leveldb forks in the first 10 minutes, you could then introduce some of that information when explaining lmdb by throwing out some comparative points with the leveldb forks.

do you think that would work? if you don't think it is enough time we could try for a different person handling the introductory content and layer you in after.

[juliangruber]

I think I'll be talking about using leveldb on the client, with things like multilevel and level-reactive and 2 other modules I'm writing right now.

[mikeal]

awe man, i was looking forward to a talk about search :)

real quick, can you give us the gist of the other 2 modules. you don't have to get in to specifics but i just want to make sure any foundational shit is covered before you speak.

[rvagg]

I can trim it down, ultimately there's too much nerdery in the lmdb stuff and I've done the "how leveldb works" stuff to death (as have @hij1nx, @dominictarr, @juliangruber and others). I'm much more interested in how the culture of modularity is shaping what we do so I can focus on the basics of that and then do a look backwards in the stack to the 3 leveldb implementations and lmdb (and more perhaps, there's level.js, leveldown-gap, memdown, locket, mysqldown [the "wut?" factor] and eventually a pure leveljs). Which leaves everyone else to cover the upwards part of the stack. So @dominictarr, @juliangruber, @hij1nx it'd be good if you didn't get too far off into your specific stuff but cover some of the stack from levelup upwards.

I don't know of the format would allow it but there's scope for pulling @substack into this block since he's been doing some very interesting work here too and he's done streams to death.

[juliangruber]

@substack and @dominictarr have been doing most about search, I only have this one module implementing this one idea. Maybe I could cut that quickly to show the spirit of just inventing shit for your needs rather than having to switch to a database that supports the kind of geospatial queries you need.

Oh, and I can totally show using level-places in the client transparently over multilevel.

[juliangruber]

One module will be a local cache, so you can do

var remote = multilevel.client(); // ...
var cache = levelup('cache', { db: MemDown }); // or db: leveljs
var db = new PersistentCache(cache, remote);

Then you keep using db and for every key that you use it will store that locally and if necessary update the remove => Don't worry about caching and subsequent lookups of the same data will be very fast.

[dominictarr]

So, I was gonna do my talk on "how to write a level db module", essentially, about patterns for extending leveldb cleanly, with the aim of being composable with other modules. perhaps I should go first?

<3 italics

[mikeal]

actually, i think it would be best for you to go second. if @rvagg tells the "story" it'll be clear where your content fits in and it also means that @rvagg will have an easier time describing where all the components go since you'll be covering it a bit better after.

[max-mapper]

@hij1nx you're working with open data AND growing a beard??? you'll be hearing from my lawyers

[mikeal]

Paolo is being moved to the "Beard Track."

:P

the order is @rvagg @dominictarr @juliangruber @hij1nx

[davglass]

If there is a beard track, I want on the tattoo track!! Wait, I might be alone on that, never mind :)

[davglass]

Sweet! My left arm sleeve (totally geeky) will be mostly done by the time the event comes around. I should do my talk in a tank top :)

[rvagg]

OK, CampJS is done and my brain has finally been able to move to nodeconfeu. Here's what I'm thinking for my basic structure:

• Some mind-bending around the flexibility that's being enabled in NodeBaseLand: Eugene Ware hacked together a project at CampJS that extends node-mysql2 on the server-side to be able to act as a fake MySQL server, which he then used to decode basic SQL into MongoDB-style queries which could be converted to LevelUP queries. Then there's David Björklund's mysqlDOWN which can back LevelUP with an actual MySQL server. Then in there somewhere I ought to be able to fit level.js to show LevelUP in the browser. The whole point being to build a "whaaaaaaat?" moment which can demonstrate the wins of modularity.
• Rant a bit about modularity: small core, highly dynamic and modular user-land, I need to work on this silly chart to make it more interesting & impressive: http://r.va.gg/presentations/sf.nodebase.meetup/#/10
• Talk briefly about back-ends, the LevelDB forks, LMDB, the crazy AbstractLevelDOWN-extension experiments going on, Level.js for the browser.

I probably won't have time to talk much about above LevelUP which leaves @dominictarr in particular to talk about that stuff.

I'll share my slides with you guys when I have something started.

@mikeal how are the hack-sessions / workshops working? Are they free-form? Is there any NodeBase focus to any of them and do we need to prepare extra stuff for them?

I did this at my CampJS workshop, a descendant of stream-adventure: https://github.com/rvagg/learnyounode/ and I want us to do a "Learn You Some Node Databases" version of the same at some point!

[mikeal]

@rvagg i had assumed that Cian talked to ya'll already, but there is a nodebase session "on the books", you all can structure it however you like, it'll be an hour and have no more than 40 people at once. you may need to run it more than once, depending on the level of signups.

we moved the NodeBase session to Monday so that it is finished up before the sessions.

[rvagg]

ok, I hadn't heard about that.

and yes, I noticed that we'd been moved to Monday which means I'm first; I just hope you have a cool keynote up front or the night before so I don't have to set the tone for the whole event.

[rvagg]

I'm clearly avoiding properly fleshing out my talk by spending a little too much time on the slide design.. I've done a fun Node Land themed slide deck and I'd be happy for anyone (in this block or any other block) to steal it if it interests you--I'd actually appreciate contributions on making it more rad!

http://r.va.gg/presentations/nodeconfeu.2013.slidetheme/

Content is authored in a Markdown file with slide separator markers, there's a build script (with a --watch mode) that turns it into the HTML with syntax-highlighting and all the mod cons you'd expect.

if you're awesome enough to want to join me in this craziness, ping me and I'll add you to the source repo.

[mcollina]

I'm not sure about using it :) (I'm using Keynote most of the time), but I think the content I'm presenting fits well in a HTML5 presentation. So, could you please add me to the repo?

[dominictarr]

@rvagg ahah, cian will love this

[juliangruber]

since @dominictarr will show how to write levelup modules and @hij1nx will talk about a real peace of software built upon level-*, a talk about how to use levelup modules will fit nicely in between.

I'll do some live coding and create a simple application, while not overlapping too much with the workshops.

[juliangruber]

or what about this: A talk about the most important level-* modules, similar to @dominictarr's talk in dublin. Thinking of level-sublevel, map-reduce, level-live-stream, multilevel, bytewise, level-hooks, level-store, level-replicate/-master...

What do you guys think?

[juliangruber]

@substack are you going to do the reactive leveldb thing too?

[ghost]

I'm covering streaming shared rendering and then at the end I'll how to hook up the shared rendering to do live-updating realtime updates hooked up to leveldb.

[rvagg]

My stuff is only now starting to solidify, it's been kind of tough to refine what I'm doing. I've added y'all to my repo, you can see the basics of it in the markdown source here (might be easier to view it raw).

In summary, A Real Database Rethink:

• briefly look at the history of databases (50 years)
• try and narrow down what a "database" is--or at least a foundation that we're all likely to agree on (very difficult)
• talk a bit about the problem of a having a beautiful abstraction (like the relational data model) overtake our thinking and crowd out more suitable solutions to problems at hand.
• move on to Node and the level* ecosystem, rant a bit about modularity, small core, vibrant user-land, etc.
• ...

I'm starting to think that I should probably insert my usual "what is leveldb" spiel here somewhere since most people are still in the dark and a gentle introduction might go down well even though I'm trying to expand this beyond leveldb in particular.

I want the module-ecosystem stuff to lead in to @dominictarr's talk which builds on it I believe. I'm going to limit myself to talking more about the categories in the ecosystem (see the table/chart I've built), particularly the pluggable backend.

[rvagg]

Some semi-random thoughts on how to run the NodeBase/Level* workshops:

The stream-adventure / learnyounode inspired workshop tool is coming along OK, tho it's certainly not as as developed as I'd like it to be. stream-adventure got consistently high praise from NodeConf and learnyounode was by far the stand-out session at CampJS according to the follow-up questionnaires; so I'm really keen for this to be the core mechanism for the workshops.

@hij1nx has suggested pairing the more experienced people with the less experienced; I'm not sure how practical the mechanics of that is but we can try. He also thinks we should dedicate the first 10 minutes to getting people to talk about what they already know (@hij1nx do you mean talk to each other or to/with the whole group?). I'd rather be slightly more formal and organised, make sure people feel like they've got bang for their buck in choosing to attend this workshop over an alternative so I'd suggest that instead we go with something more like this:

• Have the installation instructions on the screen from the beginning so they can get started whenever they are ready.
• Introduce ourselves--whoever of this group can be there and anyone else who turns up that we know is experienced enough to be a help--and give a quick run-down of how the workshop application runs and how they can get help at any point along the way from one of the instructors.
• Spend a little while inviting questions that have arisen from the talks; I'm sure there will be lots of questions and it might lead to interesting discussion. But we should allow the workshops to proceed in the background if people don't want to discuss. Also the question-time shouldn't dominate, people can chase us up outside of the workshop if they really want to chat. Perhaps only 10 minutes max.
• As people start solving the problems, invite some up to share their solutions (the 'create a gist and give me the url' approach worked nicely at NodeConf I think... if we have decent internet). Although this needs to be done carefully because if you start showing advanced answers before enough people have got there then they won't be able to engage because their understanding hasn't advanced enough. Perhaps this should be reserved for the last 10 minutes or so.

[rvagg]

i.e. less of this

and more of this

[dominictarr]

and also

[rvagg]

my slides are close to complete, latest is published here: http://r.va.gg/presentations/nodeconfeu.2013/

[rvagg]

Hah! It's not that your advice doesn't sound good I'm just trying to reconcile it with the format. We should all talk today and come up with a clearer plan. I'd also like to hear from @substack and @maxogden what feedback they got from nodeconf regarding their format and what improvements they would make.