search

histograph / data

http://histograph.io
MIT License
0 stars 4 forks source link

Create content for repository data-bag #29

Open sbocconi opened 8 years ago

sbocconi commented 8 years ago

With npm -v 3.3.12 and node -v 5.3.0 npm install fails with

npm ERR! Darwin 15.2.0
npm ERR! argv "/usr/local/Cellar/node/5.3.0/bin/node" "/usr/local/bin/npm" "install"
npm ERR! node v5.3.0
npm ERR! npm  v3.3.12
npm ERR! code ELIFECYCLE

npm ERR! leveldown@0.10.5 install: `node-gyp rebuild`
npm ERR! Exit status 1

Upgrading to the latest release fixes the problem for me, but might break with other versions of node/npm?

Stefanos-MacBook-Pro:data SB$ git diff
diff --git a/package.json b/package.json
index afb2766..62b1d46 100644
--- a/package.json
+++ b/package.json
@@ -21,7 +21,7 @@
     "geojsonhint": "^1.0.0",
     "highland": "^2.5.1",
     "is-my-json-valid": "^2.10.1",
-    "level": "^0.19.1",
+    "level": "^1.4.0",
     "minimist": "^1.1.1",
     "ndjson": "^1.3.0",
     "pg": "^4.3.0",
bertspaan commented 8 years ago

I've been working on a new version of the data package: https://github.com/histograph/data/tree/data-modules

I've moved all dataset-specific logic out of data module, and into separate modules:

I'm using this version here in NYC, with newest Node + NPM. However, I've not changed NWB + BAG data modules. I can explain how to make one! It's easy! And then I can merge data-modules branch of this repo into master.

bertspaan commented 8 years ago

By the way: hello @sbocconi!

sbocconi commented 8 years ago

hello @bertspaan!!

bertspaan commented 8 years ago

This is how data module works in new data-modules branch:

Each data module should have the following API:

module.exports.steps = [
  step1, // step1 and step2 are functions inside the data module
  step2
];
sbocconi commented 8 years ago

I changed the title of this issue since the original one is fixed but the discussion here is useful to solve the issue described in the title.

sbocconi commented 8 years ago

https://github.com/reinvantveer is working on this:

Yes I'm doing BAG at the moment. Got as far as neatly recursively unpacking
those dreadful zips (at least the Amstelveen test set) , so I can get set
to streaming thise dreadful XMLs to ndjson :)
Greetings,
Rein
reinvantveer commented 8 years ago

Doing the BAG... :handbag:

reinvantveer commented 8 years ago

Dank @jobspierings, je mag me ook assignen als je wil, geloof niet dat ik dat kan... Of doet @sbocconi dit?

reinvantveer commented 8 years ago

Downscaling issue to BAG