Usage of hlib in Juypter

[xldrkp]

Hi!

Saw what you do with facet and love it! In order to automated annotation download in Jupyter Notebook I tried to use your hlib with a Javascript kernel but have not succeeded yet. I'm especially interessted in the CSV download.

Do you have any advise for me how to set up the library?

[judell]

I believe the Jupyter JS kernel is node-based? I haven't had a reason so far to make hlib node-compatible but this is a good one so I’ll look into it, thanks.

From: Axel Dürkop Sent: Sunday, November 3, 2019 3:05 AM To: judell/hlib Cc: Subscribed Subject: [judell/hlib] Usage of hlib in Juypter (#3)

Hi! Saw what you do with facet and love it! In order to automated annotation download in Jupyter Notebook I tried to use your hlib with a Javascript kernel but have not succeeded yet. I'm especially interessted in the CSV download. Do you have any advise for me how to set up the library? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[xldrkp]

Thanks! You're right, it is node-based. I followed these instructions for conda.

[judell]

Actually, for CSV download into Jupyter, it would probably make more sense to use a Python wrapper around the Hypothesis /api/search endpoint, e.g. https://github.com/judell/Hypothesis/blob/master/hypothesis.py#L69. Perhaps I should add an example there of how to do that?

[xldrkp]

Thanks @judell, I appreciate your offer to provide an example with Python. I started tinkering with the API and Python and was quite successful, but never came to the point e.g. to get the "level" data in the CSV export of your online tool. One of my research ideas is to classify annotations by this level. Then I thought that this is done by hlib.

[judell]

@xldrkp I have a question for you. Although the title of this thread reads Usage of hlib in Juypter I would venture the subtext is Need more/better access to Hypothesis data.

Do you think that's fair? In any case, I'm mulling whether to create a separate repository for a set of tools and strategies I've been pulling together in my work at Hypothesis. It would be about how to pull data from the Hypothesis API into postgres, and then visualize it using postgres functions and materialized views that work with metabase queries and dashboards. This approach obviates the need to build Hypothesis-specific analytics into multiple languages. The analytics core lives in postgres. Any language that connects to postgres can use it.

[xldrkp]

Hi Jon,

Merry Christmas and sorry for the delay. I was busy with other things and missed the notification of your reply. As far as I understand your idea is a much wider approach to the problem which gives the Hypothesis community access to data from whatever language they come from. Postgres would be some kind of data abstraction layer in between.

Personally, I'd be happy to stay with Python analyzing data from Hypothesis connecting to Postgres. Doing so, SQL would be the first step, right?

[judell]

@xldrkp thanks to Postgres' robust support for JSON/JSONB, you can dump JSON data pulled from Hypothesis and then, yes, query it with SQL, but a SQL that's extended with JSON operators. And can be further extended with functions written in the native language, plpgsql, or indeed Python. Here I'm referring to functions that live in the database. You can also, of course, connect to the database from any language, and issue queries that use JSON operators and/or db-resident functions.

[xldrkp]

Great approach IMHO! I did not know about these functions and possibilities in Postgres. Are you already working on this? And what kind of support is needed here?

[judell]

I think this set of capabilities is not well known, and am planning to write about them.

On Fri, Dec 27, 2019 at 8:54 AM Axel Dürkop notifications@github.com wrote:

Great approach IMHO! I did not know about these functions and possibilities in Postgres. Are you already working on this? And what kind of support is needed her?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/judell/hlib/issues/3?email_source=notifications&email_token=AAALLLIKZKZJINE3BQ4QX7LQ2YXL7A5CNFSM4JIKZ2G2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHXOXFQ#issuecomment-569306006, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAALLLPPMKX4YZXRNZC7DNLQ2YXL7ANCNFSM4JIKZ2GQ .

[xldrkp]

@judell The only valuable resource I found dealing with the features you mentioned is https://www.packtpub.com/big-data-and-business-intelligence/learning-postgresql-11-third-edition.