TeamPiped / Piped

An alternative privacy-friendly YouTube frontend which is efficient by design.
https://piped.video
GNU Affero General Public License v3.0
8.21k stars 671 forks source link

Feed not updating #896

Closed FuzzyGrim closed 2 years ago

FuzzyGrim commented 2 years ago

I am self-hosting Piped and I am really liking it but I am having the same problem as #299 and #289, my feed is not getting updated and it only seems to find new videos for a specific channel when I search for that channel.

I have set API_URL in config.properties, and when I go to https://pipedapi.domain.com I get the following error: io.activej.http.HttpError: HTTP code 404: Not found, and if I go to https://pipedapi.domain.com/trending?region=US I am receiving the correct JSON, which should mean that the API is working.

If I go to: https://pubsubhubbub.appspot.com/subscription-details?hub.callback=https%3A%2F%2Fpipedapi.domain.com%2Fwebhooks%2Fpubsub&hub.topic=https%3A%2F%2Fwww.youtube.com%2Fxml%2Ffeeds%2Fvideos.xml%3Fchannel_id%3D{channel_id}&hub.secret=

I get:

Last successful verification
    Sat, 05 Mar 2022 22:50:10 +0000
Expiration time
    Thu, 10 Mar 2022 22:50:09 +0000
Last subscribe request
    Thu, 10 Mar 2022 11:13:55 +0000
Last unsubscribe request
    n/a
Last verification error
    Thu, 10 Mar 2022 11:13:56 +0000 (Bad response code 404) 
Last delivery error
    Fri, 04 Mar 2022 00:59:25 +0000 (HTTP 530) 

Content received
    Wed, 09 Mar 2022 23:20:17 +0000
Content delivered
    Wed, 09 Mar 2022 23:20:20 +0000 

My docker-compose is similar to https://github.com/TeamPiped/Piped-Docker/issues/7 as I am also using traefik. I basically followed https://piped-docs.kavin.rocks/docs/self-hosting/#docker-compose-nginx-aio-script and then chose traefik.

services:
    pipedfrontend:
        image: 1337kavin/piped-frontend:latest
        restart: unless-stopped
        depends_on:
            - piped
        container_name: piped-frontend
        entrypoint: ash -c 'sed -i s/pipedapi.kavin.rocks/pipedapi.domain.com/g
            /usr/share/nginx/html/assets/* && /docker-entrypoint.sh && nginx -g
            "daemon off;"'
        networks:
            - proxy
        labels:
            - "traefik.enable=true"
            - "traefik.http.routers.piped.entrypoints=http"
            - "traefik.http.routers.piped.rule=Host(`piped.domain.com`)"
            - "traefik.http.middlewares.piped-https-redirect.redirectscheme.scheme=https"
            - "traefik.http.routers.piped.middlewares=piped-https-redirect"
            - "traefik.http.routers.piped-secure.entrypoints=https"
            - "traefik.http.routers.piped-secure.rule=Host(`piped.domain.com`)"
            - "traefik.http.routers.piped-secure.tls=true"
            - "traefik.http.routers.piped-secure.service=piped"
            - "traefik.http.services.piped.loadbalancer.server.port=80"
            - "traefik.docker.network=proxy"
            - "traefik.http.routers.piped-secure.middlewares=authelia@docker"

    ytproxy:
        image: 1337kavin/ytproxy:latest
        restart: unless-stopped
        networks:
            - default
        volumes:
            - ytproxy:/app/socket
        container_name: piped-ytproxy

    piped:
        image: 1337kavin/piped:latest
        restart: unless-stopped
        volumes:
            - ./config/config.properties:/app/config.properties:ro
        networks:
            - default
        depends_on:
            - postgres
        container_name: piped-backend

    varnish:
        image: varnish:7.0-alpine
        restart: unless-stopped
        networks:
            - default
            - proxy
        volumes:
            - ./config/default.vcl:/etc/varnish/default.vcl:ro
        container_name: piped-varnish
        depends_on:
            - piped
        labels:
            - "traefik.enable=true"
            - "traefik.http.routers.pipedapi.entrypoints=http"
            - "traefik.http.routers.pipedapi.rule=Host(`pipedapi.domain.com`)"
            - "traefik.http.middlewares.pipedapi-https-redirect.redirectscheme.scheme=https"
            - "traefik.http.routers.pipedapi.middlewares=pipedapi-https-redirect"
            - "traefik.http.routers.pipedapi-secure.entrypoints=https"
            - "traefik.http.routers.pipedapi-secure.rule=Host(`pipedapi.domain.com`)"
            - "traefik.http.routers.pipedapi-secure.tls=true"
            - "traefik.http.routers.pipedapi-secure.service=pipedapi"
            - "traefik.http.services.pipedapi.loadbalancer.server.port=80"
            - "traefik.docker.network=proxy"
        healthcheck:
            test: ash -c "wget --no-verbose --tries=1 --spider 127.0.0.1:80/feed ||
                (varnishreload && exit 1)"
            interval: 10s
            timeout: 10s
            retries: 1

    nginx:
        image: nginx:mainline-alpine
        restart: unless-stopped
        networks:
            - proxy
            - default
        volumes:
            - ./config/nginx.conf:/etc/nginx/nginx.conf:ro
            - ./config/pipedapi.conf:/etc/nginx/conf.d/pipedapi.conf:ro
            - ./config/pipedproxy.conf:/etc/nginx/conf.d/pipedproxy.conf:ro
            - ./config/pipedfrontend.conf:/etc/nginx/conf.d/pipedfrontend.conf:ro
            - ./config/ytproxy.conf:/etc/nginx/snippets/ytproxy.conf:ro
            - ytproxy:/var/run/ytproxy
        container_name: piped-nginx
        depends_on:
            - piped
            - varnish
            - ytproxy
            - pipedfrontend
        labels:
            - "traefik.enable=true"
            - "traefik.http.routers.pipedproxy.entrypoints=http"
            - "traefik.http.routers.pipedproxy.rule=Host(`pipedproxy.domain.com`)"
            - "traefik.http.middlewares.pipedproxy-https-redirect.redirectscheme.scheme=https"
            - "traefik.http.routers.pipedproxy.middlewares=pipedproxy-https-redirect"
            - "traefik.http.routers.pipedproxy-secure.entrypoints=https"
            - "traefik.http.routers.pipedproxy-secure.rule=Host(`pipedproxy.domain.com`)"
            - "traefik.http.routers.pipedproxy-secure.tls=true"
            - "traefik.http.routers.pipedproxy-secure.service=pipedproxy"
            - "traefik.http.services.pipedproxy.loadbalancer.server.port=80"
            - "traefik.docker.network=proxy"

    postgres:
        image: postgres:13-alpine
        restart: unless-stopped
        networks:
            - default
        volumes:
            - ./data/db:/var/lib/postgresql/data
        environment:
            - POSTGRES_DB=piped
            - POSTGRES_USER=piped
            - POSTGRES_PASSWORD={DB_PW}
        container_name: piped-postgres

volumes:
    ytproxy: null
networks:
    proxy:
        external: true

These are some other files that may be useful:

pipedapi.conf

server {
    listen 80;
    server_name pipedapi.domain.com;

    location / {
        proxy_pass http://varnish:80;
        proxy_http_version 1.1;
        proxy_set_header Connection "keep-alive";
    }
}

pipedproxy.conf

server {
    listen 80;
    server_name pipedproxy.domain.com;

    location ~ (/videoplayback|/api/v4/|/api/manifest/) {
        include snippets/ytproxy.conf;
        add_header Cache-Control private always;
    }

    location / {
        include snippets/ytproxy.conf;
        add_header Cache-Control "public, max-age=604800";
    }
}

config.properties

# The port to Listen on.
PORT: 8080

# The number of workers to use for the server
HTTP_WORKERS: 2

# Proxy
PROXY_PART: https://pipedproxy.domain.com

# Outgoing HTTP Proxy - eg: 127.0.0.1:8118
#HTTP_PROXY: 127.0.0.1:8118

# Captcha Parameters
CAPTCHA_BASE_URL: https://api.capmonster.cloud/
CAPTCHA_API_KEY: INSERT_HERE

# Public API URL
API_URL: https://pipedapi.domain.com

# Public Frontend URL
FRONTEND_URL: https://piped.domain.com

# Hibernate properties
hibernate.connection.url: jdbc:postgresql://postgres:5432/piped
hibernate.connection.driver_class: org.postgresql.Driver
hibernate.dialect: org.hibernate.dialect.PostgreSQL10Dialect
hibernate.connection.username: piped
hibernate.connection.password: long_string

Except for this problem, everything else seems to be working properly.

FireMasterK commented 2 years ago

Does https://pipedapi.kavin.rocks/webhooks/pubsub?hub.challenge=123456 on your instance show 123456?

FuzzyGrim commented 2 years ago

Everything is working correctly again without me changing any config file, maybe there was an update that fixed the bug.