Many package incompatibilities, is this project abandoned?

[caschoener]

I tried the install process in the readme but I run into many python package incompatibilies. I worked through a few by downgrading flask, jinja, werkzeug to older versions but I think it'll take a more systematic effort.

Just curious if this project is abandoned or if there'd be interest in me going through and pinning all the packages / fixing code when needed. The idea of a open-source scuba is really appealing to me and I hate to see it so close to working.

[okayzed]

hi cash, the project is still running on my servers, but i have not updated the python dependencies in a while. if you are looking for open source scuba, i think a mix of ClickHouse + a UI is a good way to go

[mashlol]

I sadly had the same issue as @caschoener, and share the same sentiment, it's such an amazing project! @okayzed are you able to share the package versions used on the functioning demo? That could help us figure out which packages need downgrading to get it to work again.

Alternatively, if you know a similar UI to snorkel that works with ClickHouse, that would also be super helpful! Having a fast UI to analyze data quickly without needing to write SQL every time is a huge productivity booster, but most of the ones I saw for ClickHouse just support visualizing your results after running a SQL query.

[caschoener]

I ended up spinning up clickhouse pointed at my postgres DB and I'm just slowly building out the views I want as I need them. Thanks for the tip zed, it's remarkably fast.

I do agree that it'd be nice if someone made a snorkel UI for it, it would honestly only be a few days work with chatgpt etc.

[okayzed]

I will share the package versions on the weekend, I'm hoping it isn't too hard to make a clickhouse SQL query adapter

[caschoener]

Any luck?

[okayzed]

python2 deps:

(dev) okay@ubuntu:~/slite$ pip freeze
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
addict==2.2.0
Babel==2.6.0
bcrypt==3.1.6
blinker==1.4
certifi==2018.11.29
cffi==1.11.5
chardet==3.0.4
Click==7.0
diskcache==3.1.1
Flask==1.0.2
Flask-Admin==1.5.3
Flask-BabelEx==0.9.3
Flask-Dance==1.3.0
Flask-Login==0.4.1
Flask-Mail==0.9.1
Flask-OAuth==0.12
flask-peewee==3.0.3
Flask-Principal==0.4.0
Flask-Security==3.0.0
Flask-Social==1.6.2
Flask-WTF==0.14.2
httplib2==0.12.0
idna==2.8
itsdangerous==1.1.0
Jinja2==2.10
lazy==1.4
libsass==0.17.0
MarkupSafe==1.1.0
oauth2==1.9.0.post1
oauthlib==3.0.1
passlib==1.7.1
peewee==3.8.2
pkg-resources==0.0.0
Preparable==0.1.2
pudgy==0.1.5
pycparser==2.19
pystache==0.5.4
pytz==2018.9
requests==2.21.0
requests-oauthlib==1.2.0
six==1.12.0
speaklater==1.3
urllib3==1.24.1
URLObject==2.4.3
Werkzeug==0.14.1
wtf-peewee==3.0.0
WTForms==2.2.1

i think it will work with python3 as well, though (it is compatible with python2 + python3)

[okayzed]

e641f21d9c6f3fb95c259a5f106a0be7a2539661 adds requirements.py2.txt and requirements.py3.txt

i verified that py3 works by starting a new virtual env and then installing the requirements with pip install -r requirements.py3.txt, then starting the snorkel server and verifying that behavior works.

i think it makes sense to publish a snorkel docker that supports py3 and includes the sybil binary, but i don't have much time

(PS: sorry for the delays, it's been a crazy work year)