ipfs-shipyard / ipfs-geoip

geoip lookup over DAG-CBOR dataset loaded from IPFS
MIT License
70 stars 21 forks source link

Copy to 0.4 network #6

Closed dignifiedquire closed 8 years ago

dignifiedquire commented 8 years ago

@krl we need to put the data onto the 0.4 network, any pointers on how to easiest do this

cc @whyrusleeping @lgierth for ideas

whyrusleeping commented 8 years ago

get it all on one v0.3.11 node, then run a migration on it and now its an 0.4.0 node

ghost commented 8 years ago

Isn't the geoip dag currently used by the webui broken? We'll need scripts to reproduce it from the source geoip data.

dignifiedquire commented 8 years ago

Nope @krl fixed it :) all the data is on the 0.3 network atm

daviddias commented 8 years ago

@dignifiedquire woot! :D

dignifiedquire commented 8 years ago

well looks like I was too excited trying to pull down all the refs fails on the 0.3 network :(

dignifiedquire commented 8 years ago

Okay nothing is working :( no idea why though I did the following

Run

$ node geoip-gen.js ../GeoLiteCity_20150407/GeoLiteCity-Blocks.csv ../GeoLiteCity_20150407/GeoLiteCity-Location.csv

Update the main hash with the return value QmQQ3BUpPjgYiTdhp4H9YWSCtoFXs8t91njhpvXNNLd3yB in lib/lookup.js

After that the look ups fail with

❯ node example/lookup.js 8.8.8.8
/Users/dignifiedquire/opensource/ipfs/ipfs-geoip/lib/lookup.js:30
      return memoized_lookup(ipfs, res.Links[child - 1].Hash, lookfor, cb)
                                                       ^

TypeError: Cannot read property 'Hash' of undefined
    at /Users/dignifiedquire/opensource/ipfs/ipfs-geoip/lib/lookup.js:30:56
    at finish (/Users/dignifiedquire/opensource/ipfs/ipfs-geoip/node_modules/wreck/lib/index.js:319:16)
    at wrapped (/Users/dignifiedquire/opensource/ipfs/ipfs-geoip/node_modules/hoek/lib/index.js:867:20)
    at onReaderFinish (/Users/dignifiedquire/opensource/ipfs/ipfs-geoip/node_modules/wreck/lib/index.js:365:16)
    at g (events.js:273:16)
    at emitNone (events.js:85:20)
    at emit (events.js:179:7)
    at finishMaybe (_stream_writable.js:468:14)
    at afterWrite (_stream_writable.js:347:3)
    at nextTickCallbackWithManyArgs (node.js:497:18)

and

$ ipfs refs -r QmQQ3BUpPjgYiTdhp4H9YWSCtoFXs8t91njhpvXNNLd3yB
Qmdc6WT9mjNbEqgXQZ4vrXKpua3gh1oY3uCDasxmX5DvF6
QmWNYkjWPJ4w3ejg5RHHaefS5bGt3e1DcoFf1BhSiU6P8B
QmTU5PW7ZX86K39FM84rWLSoReA84TDsQ9JDP3sQ29Jywj

just hangs :(

daviddias commented 8 years ago

Upon the update of js-ipfs-api to 2.13, I managed to successfully the lookup:

» node lookup.js 162.243.248.213
Result: {
  "country_name": "United States",
  "country_code": "US",
  "region_code": "NY",
  "city": "New York",
  "postal_code": "10118",
  "latitude": 40.7143,
  "longitude": -74.006,
  "metro_code": "501",
  "area_code": "212",
  "planet": "Earth"
}
Pretty result: New York, NY, United States, Earth

Also, with the js-ipfs-api update, the geoip-gen.js script worked and completed successfully, resulting in the following hash Qme3KxW193W3YVGZB8knWTeY3v5NzUdKeU8gfphgD2VyS8

After updating the GEOIP root hash to the new one, it thrown this error:

» node lookup.js 162.243.248.213
/Users/ground-control/code/protocol-labs/ipfs-geoip/lib/lookup.js:30
      return memoized_lookup(ipfs, res.Links[child - 1].Hash, lookfor, cb)
                                                       ^

TypeError: Cannot read property 'Hash' of undefined
    at /Users/ground-control/code/protocol-labs/ipfs-geoip/lib/lookup.js:30:56
    at finish (/Users/ground-control/code/protocol-labs/ipfs-geoip/node_modules/wreck/lib/index.js:319:16)
    at wrapped (/Users/ground-control/code/protocol-labs/ipfs-geoip/node_modules/hoek/lib/index.js:867:20)
    at onReaderFinish (/Users/ground-control/code/protocol-labs/ipfs-geoip/node_modules/wreck/lib/index.js:365:16)
    at g (events.js:273:16)
    at emitNone (events.js:85:20)
    at emit (events.js:179:7)
    at finishMaybe (_stream_writable.js:468:14)
    at endWritable (_stream_writable.js:478:3)
    at Writable.end (_stream_writable.js:443:5)
dignifiedquire commented 8 years ago

@diasdavid was this on 0.4 (which commit) or on 0.3.11?

daviddias commented 8 years ago
» git log | head -n1
commit 3deb40eb06004245dde12784fc8583532e09c36c
» ipfs version
ipfs version 0.4.0-dev