Add CSV URL importer to graph loader

[pudo]

Currently, a DB connection URI is required in DATASETS_YAML to map data into the entities search. Instead, this should also allow a user to specify the URL of a CSV file (possibly including credentials) from which the data should be loaded.

[smmbllsm]

See #168

[pudo]

This looks to be done, no? Do you think we could have one mapping with a public CSV url that ships with aleph by default? Both as a test case and demo.

[smmbllsm]

Any kind of data in particular?

On Wed, Apr 5, 2017 at 12:29 PM, Friedrich Lindenberg < notifications@github.com> wrote:

This looks to be done, no? Do you think we could have one mapping with a public CSV url that ships with aleph by default? Both as a test case and demo.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/alephdata/aleph/issues/158#issuecomment-291820556, or mute the thread https://github.com/notifications/unsubscribe-auth/AYUmNeKTYNS6Kw-1MsjOdt1HZ36NuOT8ks5rs2z3gaJpZM4MKnEi .

-- Lion Summerbell Investigative Data Engineer Organized Crime and Corruption Reporting Project Sarajevo / Bucharest / Tbilisi / Washington e: lion@occrp.org stella@occrp.org PGP Signature: 0D7E AA52 05CD FE32 2E8B DE84 7A62 FD2D 86E8 0F81 https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x7A62FD2D86E80F81 w: https://www.occrp.org/ skype: alphabet.citizen

[pudo]

Maybe US OFAC? It's in our datavault right now but would make a very good demo if dumped to CSV. Or even straight from the source:

https://www.treasury.gov/resource-center/sanctions/SDN-List/Pages/sdn_data.aspx

[smmbllsm]

Check check.

On Wed, Apr 5, 2017 at 12:33 PM, Friedrich Lindenberg < notifications@github.com> wrote:

Maybe US OFAC? It's in our datavault right now but would make a very good demo if dumped to CSV. Or even straight from the source:

https://www.treasury.gov/resource-center/sanctions/SDN- List/Pages/sdn_data.aspx

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/alephdata/aleph/issues/158#issuecomment-291821397, or mute the thread https://github.com/notifications/unsubscribe-auth/AYUmNWgXGg8-Tb0mWcyxUBOQCt6_HxXlks5rs233gaJpZM4MKnEi .

-- Lion Summerbell Investigative Data Engineer Organized Crime and Corruption Reporting Project Sarajevo / Bucharest / Tbilisi / Washington e: lion@occrp.org stella@occrp.org PGP Signature: 0D7E AA52 05CD FE32 2E8B DE84 7A62 FD2D 86E8 0F81 https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x7A62FD2D86E80F81 w: https://www.occrp.org/ skype: alphabet.citizen

[smmbllsm]

Just helping out with a research request and then will get this out, if that's alright.

On Wed, Apr 5, 2017 at 12:41 PM, Lion Summerbell lion@occrp.org wrote:

Check check.

On Wed, Apr 5, 2017 at 12:33 PM, Friedrich Lindenberg < notifications@github.com> wrote:

Maybe US OFAC? It's in our datavault right now but would make a very good demo if dumped to CSV. Or even straight from the source:

https://www.treasury.gov/resource-center/sanctions/SDN-List/ Pages/sdn_data.aspx

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/alephdata/aleph/issues/158#issuecomment-291821397, or mute the thread https://github.com/notifications/unsubscribe-auth/AYUmNWgXGg8-Tb0mWcyxUBOQCt6_HxXlks5rs233gaJpZM4MKnEi .

-- Lion Summerbell Investigative Data Engineer Organized Crime and Corruption Reporting Project Sarajevo / Bucharest / Tbilisi / Washington e: lion@occrp.org stella@occrp.org PGP Signature: 0D7E AA52 05CD FE32 2E8B DE84 7A62 FD2D 86E8 0F81 https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x7A62FD2D86E80F81 w: https://www.occrp.org/ skype: alphabet.citizen

-- Lion Summerbell Investigative Data Engineer Organized Crime and Corruption Reporting Project Sarajevo / Bucharest / Tbilisi / Washington e: lion@occrp.org stella@occrp.org PGP Signature: 0D7E AA52 05CD FE32 2E8B DE84 7A62 FD2D 86E8 0F81 https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x7A62FD2D86E80F81 w: https://www.occrp.org/ skype: alphabet.citizen

[pudo]

Take it easy, man :)

[smmbllsm]

This is up on datavault.

On Wed, Apr 5, 2017 at 12:52 PM, Friedrich Lindenberg < notifications@github.com> wrote:

Take it easy, man :)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/alephdata/aleph/issues/158#issuecomment-291825043, or mute the thread https://github.com/notifications/unsubscribe-auth/AYUmNcODFF3iBshCyJsesfZ2XnEp7sfzks5rs3JVgaJpZM4MKnEi .

-- Lion Summerbell Investigative Data Engineer Organized Crime and Corruption Reporting Project Sarajevo / Bucharest / Tbilisi / Washington e: lion@occrp.org stella@occrp.org PGP Signature: 0D7E AA52 05CD FE32 2E8B DE84 7A62 FD2D 86E8 0F81 https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x7A62FD2D86E80F81 w: https://www.occrp.org/ skype: alphabet.citizen

[pudo]

Sorry, I don't understand?

[smmbllsm]

Was responding to US OFAC, not the csv importer itself :)

On Fri, Apr 7, 2017 at 10:05 AM, Friedrich Lindenberg < notifications@github.com> wrote:

Sorry, I don't understand?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/alephdata/aleph/issues/158#issuecomment-292470337, or mute the thread https://github.com/notifications/unsubscribe-auth/AYUmNa57ZdR0_Bo0DOGl4CraTHs8esLuks5rte5jgaJpZM4MKnEi .

-- Lion Summerbell Investigative Data Engineer Organized Crime and Corruption Reporting Project Sarajevo / Bucharest / Tbilisi / Washington e: lion@occrp.org stella@occrp.org PGP Signature: 0D7E AA52 05CD FE32 2E8B DE84 7A62 FD2D 86E8 0F81 https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x7A62FD2D86E80F81 w: https://www.occrp.org/ skype: alphabet.citizen

[pudo]

Thanks, @smmbllsm. Good work!