datopian / giftless

🎁 A pluggable Git LFS server written in Python. Highly customizable and easy to extend.
https://giftless.datopian.com
MIT License
124 stars 30 forks source link

pypi release (0.5.0) broken #102

Open yuv opened 2 years ago

yuv commented 2 years ago

I have setup an instance of giftless (pip) in an LXD container (Ubuntu 20.04) on my homelab. Using git-lfs, I can commit files, but cloning and pulling (git lfs pull) does not work. Log and relevant config files below. What am I doing wrong?

uwsgi log file:

Apr 19 21:36:03 giftless uwsgi: [pid: 1150|app: 0|req: 321/389] 192.168.50.50 () {36 vars in 1434 bytes} [Tue Apr 19 21:36:03 2022] GET /Orgz/Projz/objects/storage/9fc1a8cab7aec045ae158bf87139965dc0c9af92cb65dfa0dead9cfdc7274b8e?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiIsImtpZCI6ImdpZnRsZXNzLWludGVybmFsLWp3dC1rZXkifQ.eyJleHAiOjE2NTA0MDQyMTMsImlhdCI6MTY1MDQwNDE1MywibmJmIjoxNjUwNDA0MTUzLCJzdWIiOm51bGwsIm5hbWUiOiJhbm9ueW1vdXMiLCJzY29wZXMiOiJvYmo6WXV2YWxfTGV2eS9TZXBhcmF0aW9uX05pY29sZS85ZmMxYThjYWI3YWVjMDQ1YWUxNThiZjg3MTM5OTY1ZGMwYzlhZjkyY2I2NWRmYTBkZWFkOWNmZGM3Mjc0YjhlOnJlYWQifQ._CSLwDTEbkacsZU58WfnkLyFgL55N6ZJmPMTSmOraOg => generated 0 bytes in 6 msecs (HTTP/1.1 500) 0 headers in 0 bytes (0 switches on core 0) Apr 19 21:36:03 giftless uwsgi: 2022-04-19 21:36:03,696 giftless.auth.jwt DEBUG Allowing scopes: ('obj:Orgz/Projz/0d5838affc281e5ca492a350e2d7c274daf2da4b415d8c1d5d6339192aa1ea26:read',) Apr 19 21:36:03 giftless uwsgi: 2022-04-19 21:36:03,696 giftless.auth DEBUG Authenticated identity: Apr 19 21:36:03 giftless uwsgi: Traceback (most recent call last): Apr 19 21:36:03 giftless uwsgi: File "/home/ubuntu/giftless/giftless_env/lib/python3.8/site-packages/flask/app.py", line 2464, in call Apr 19 21:36:03 giftless uwsgi: return self.wsgi_app(environ, start_response) Apr 19 21:36:03 giftless uwsgi: File "/home/ubuntu/giftless/giftless_env/lib/python3.8/site-packages/flask/app.py", line 2450, in wsgi_app Apr 19 21:36:03 giftless uwsgi: response = self.handle_exception(e) Apr 19 21:36:03 giftless uwsgi: File "/home/ubuntu/giftless/giftless_env/lib/python3.8/site-packages/flask/app.py", line 1867, in handle_exception Apr 19 21:36:03 giftless uwsgi: reraise(exc_type, exc_value, tb) Apr 19 21:36:03 giftless uwsgi: File "/home/ubuntu/giftless/giftless_env/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise Apr 19 21:36:03 giftless uwsgi: raise value Apr 19 21:36:03 giftless uwsgi: File "/home/ubuntu/giftless/giftless_env/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app Apr 19 21:36:03 giftless uwsgi: response = self.full_dispatch_request() Apr 19 21:36:03 giftless uwsgi: File "/home/ubuntu/giftless/giftless_env/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request Apr 19 21:36:03 giftless uwsgi: rv = self.handle_user_exception(e) Apr 19 21:36:03 giftless uwsgi: File "/home/ubuntu/giftless/giftless_env/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception Apr 19 21:36:03 giftless uwsgi: reraise(exc_type, exc_value, tb) Apr 19 21:36:03 giftless uwsgi: File "/home/ubuntu/giftless/giftless_env/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise Apr 19 21:36:03 giftless uwsgi: raise value Apr 19 21:36:03 giftless uwsgi: File "/home/ubuntu/giftless/giftless_env/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request Apr 19 21:36:03 giftless uwsgi: rv = self.dispatch_request() Apr 19 21:36:03 giftless uwsgi: File "/home/ubuntu/giftless/giftless_env/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request Apr 19 21:36:03 giftless uwsgi: return self.view_functionsrule.endpoint Apr 19 21:36:03 giftless uwsgi: File "/home/ubuntu/giftless/giftless_env/lib/python3.8/site-packages/flask_classful.py", line 301, in proxy Apr 19 21:36:03 giftless uwsgi: response = view(request.view_args) Apr 19 21:36:03 giftless uwsgi: File "/home/ubuntu/giftless/giftless_env/lib/python3.8/site-packages/giftless/auth/init.py", line 90, in decorated_function Apr 19 21:36:03 giftless uwsgi: return f(*args, *kwargs) Apr 19 21:36:03 giftless uwsgi: File "/home/ubuntu/giftless/giftless_env/lib/python3.8/site-packages/flask_classful.py", line 269, in inner Apr 19 21:36:03 giftless uwsgi: return fn(args, kwargs) Apr 19 21:36:03 giftless uwsgi: File "/home/ubuntu/giftless/giftless_env/lib/python3.8/site-packages/giftless/transfer/basic_streaming.py", line 85, in get Apr 19 21:36:03 giftless uwsgi: filename = safe_filename(filename) Apr 19 21:36:03 giftless uwsgi: File "/home/ubuntu/giftless/giftless_env/lib/python3.8/site-packages/giftless/util.py", line 84, in safe_filename Apr 19 21:36:03 giftless uwsgi: return ''.join(c for c in original_filename if c in valid_chars) Apr 19 21:36:03 giftless uwsgi: TypeError: 'NoneType' object is not iterable

Nginx:

server {
    listen *:443 ssl;
    server_name giftless.lan;

    include ssl_params;
    ssl_certificate        /etc/ssl/private/key.pem;
    ssl_certificate_key    /etc/ssl/private/key.pem;

    error_log syslog:server=unix:/dev/log,facility=local6,tag=nginx,severity=debug;
    access_log syslog:server=unix:/dev/log,facility=local7,tag=nginx,severity=info haproxy_log;

    location / {
        client_max_body_size 10G;
        include uwsgi_params;
        uwsgi_buffer_size 32k;
        uwsgi_buffers 8 32k;
        uwsgi_busy_buffers_size 32k;
        uwsgi_pass unix:///home/ubuntu/giftless/giftless.sock;
    }

    root /home/ubuntu/giftless/lfs-storage;
    index index.php index.html index.htm;

}

giftless.conf.yaml

# Giftless configuration
AUTH_PROVIDERS:
  - giftless.auth.allow_anon:read_write

uwsgi_giftless.ini

[uwsgi]
module = giftless.wsgi_entrypoint
callable = app
manage-script-name
master = true
processes = 2
threads = 2
socket = /home/ubuntu/giftless/giftless.sock
chmod-socket = 666
uid = ubuntu
gid = ubuntu
pidfile = /tmp/giftless.pid
chdir = /home/ubuntu/giftless
vacuum = true
die-on-term = true
logger = syslog:uwsgi,local5

bash script to start giftless

#!/bin/bash
cd /home/ubuntu/giftless
source giftless_env/bin/activate
export GIFTLESS_CONFIG_FILE=giftless.conf.yaml
#export FLASK_ENV=development
#export FLASK_APP=giftless.wsgi_entrypoint
export GIFTLESS_DEBUG=1
/home/ubuntu/giftless/giftless_env/bin/uwsgi --ini /home/ubuntu/giftless/uwsgi_giftless.ini
xeechou commented 1 year ago

Hi, I ran into the same issue and went to debug the code myself. It seems the published package is outdated and I pip install path-to-giftless-repo-dir solved my problem. Hope this works for you as well.

yuv commented 1 year ago

thanks @xeechou ! pip install git+https://github.com/datopian/giftless.git@c0935cec1f13b71d39de7d95510bfab1cdad8767 worked for me. Let me know if I can buy you a drink next time I am in Toronto, and you can certainly ask me if you need an Ontario lawyer. I owe you one.

as for the giftless project, I wonder why the pypi version is stale? https://pypi.org/project/giftless/#history

or should I worry that the whole giftless project is becoming abandonware?