search

mhdzumair / MediaFusion

Universal Stremio Add-On
MIT License
184 stars 27 forks source link

Selfhosted docker-compose method failed. #143

Closed ericvlog closed 3 months ago

ericvlog commented 3 months ago

failed to create network docker-compose_default: Error response from daemon: could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network

hello, I try selfhost with docker compose

as my system already have prowlarr and nginx proxy manager

so below is my docker-compose and .env

HOST_URL="https://mediafusion.mydomain.duckdns.org" MONGO_URI="mongodb://mongodb:27017/mediafusion" REDIS_URL="redis://redis:6379" PROWLARR_URL="http://192.168.1.100:9696" PROWLARR_API_KEY=66fab75acab44fb0a29fecc88c42c89f2 PROWLARR_IMMEDIATE_MAX_PROCESS=5 PROWLARR_IMMEDIATE_MAX_PROCESS_TIME=15 PROWLARR_SEARCH_INTERVAL_HOUR="24" IS_SCRAP_FROM_TORRENTIO="true" ENABLE_RATE_LIMIT="false" ENABLE_TAMILMV_SEARCH_SCRAPER="false"

version: '3.8'
services:
  mediafusion:
    image: mhdzumair/mediafusion:v3.7.4
    ports:
      - "8899:80"
    env_file:
      - .env
    depends_on:
      - mongodb
      - redis
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost/health"]
      interval: 1m
      timeout: 10s
      retries: 5
      start_period: 10s
  mongodb:
    image: mongo:4.4.6
    volumes:
      - mongo-data:/data/db
    ports:
      - "27017:27017"

  redis:
    image: redis:latest
    ports:
      - "6379:6379"

  dramatiq-worker:
    image: mhdzumair/mediafusion:v3.7.4
    command: ["pipenv", "run", "dramatiq-gevent", "api.task"]
    env_file:
      - .env
    depends_on:
      - mongodb

  scraper:
    image: mhdzumair/mediafusion:v3.7.4
    command: ["pipenv", "run", "python", "-m", "api.scheduler"]
    env_file:
      - .env
    depends_on:
      - mongodb

volumes:
  mongo-data:
mhdzumair commented 3 months ago

Its issue with your docker usable network ips, Try with following solutions

  1. Remove Unused Networks 
    
# To list all networks:
docker network ls

To remove a network:

docker network rm [NETWORK_ID]


2. Try to Restart Docker
3. Cleanup Docker System
```bash
docker system prune

  1. Manually Specify a Subnet

    
version: '3.8'
services:
mediafusion:
image: mhdzumair/mediafusion:v3.7.4
ports:
  - "8899:80"
env_file:
  - .env
depends_on:
  - mongodb
  - redis
healthcheck:
  test: ["CMD", "curl", "-f", "http://localhost/health"]
  interval: 1m
  timeout: 10s
  retries: 5
  start_period: 10s
networks:
  - mediafusion_network

mongodb:
image: mongo:4.4.6
volumes:
  - mongo-data:/data/db
ports:
  - "27017:27017"
networks:
  - mediafusion_network

redis:
image: redis:latest
ports:
  - "6379:6379"
networks:
  - mediafusion_network

dramatiq-worker:
image: mhdzumair/mediafusion:v3.7.4
command: ["pipenv", "run", "dramatiq-gevent", "api.task"]
env_file:
  - .env
depends_on:
  - mongodb
networks:
  - mediafusion_network

scraper:
image: mhdzumair/mediafusion:v3.7.4
command: ["pipenv", "run", "python", "-m", "api.scheduler"]
env_file:
  - .env
depends_on:
  - mongodb
networks:
  - mediafusion_network

volumes: mongo-data:

networks: mediafusion_network: driver: bridge ipam: driver: default config:

ericvlog commented 3 months ago

Hello, Tq so much for your reply. I managed install the docker compose but 3 container stopped.

[docker-compose-dramatiq-worker-1] [docker-compose-mediafusion-1] [docker-compose-scraper-1]

The error msg from logs is 

  File "<frozen runpy>", line 112, in _get_module_details
  File "/mediafusion/api/__init__.py", line 6, in <module>
    from db.config import settings
  File "/mediafusion/db/config.py", line 38, in <module>
    settings = Settings()
               ^^^^^^^^^^
  File "/root/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.11/site-packages/pydantic_settings/main.py", line 84, in __init__
    super().__init__(
  File "/root/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.11/site-packages/pydantic/main.py", line 171, in __init__
    self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for Settings
secret_key
  Field required [type=missing, input_value={'mongo_uri': 'mongodb://...ble_rate_limit': 'true'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.6/v/missing
mhdzumair commented 3 months ago

Please read the documentation for env configuration again. You missed to set the secret key.

ericvlog commented 3 months ago

Yup, it can already!! Just ask this addons is based on prowlarr izzit? So I need delete prowlarr indexer less for more efficient. So it no scraper data when idle? or just scraper prowlarr/torrentio when request.

Thanks for your kind help!! 👍

image image

ericvlog commented 3 months ago

It successful installed on stremio using nginx proxy manager(https). But when I clicked on movies It no come out. Belows is the logs files.

[docker-compose-dramatiq-worker-1]

[2024-03-15 02:07:09,997] [PID 9] [Thread-4] [dramatiq.worker.ConsumerThread(default.DQ)] [INFO] Restarting consumer in 3.00 seconds.
[2024-03-15 02:07:09,997] [PID 9] [Thread-4] [dramatiq.worker.ConsumerThread(default.DQ)] [CRITICAL] Consumer encountered a connection error: Error -2 connecting to redis:6379. Name or service not known.
[2024-03-15 02:07:12,967] [PID 10] [Thread-4] [dramatiq.worker.ConsumerThread(default.DQ)] [INFO] Restarting consumer in 3.00 seconds.
[2024-03-15 02:07:12,951] [PID 10] [Thread-3] [dramatiq.worker.ConsumerThread(default)] [INFO] Restarting consumer in 3.00 seconds.
[2024-03-15 02:07:12,967] [PID 10] [Thread-4] [dramatiq.worker.ConsumerThread(default.DQ)] [CRITICAL] Consumer encountered a connection error: Error -2 connecting to redis:6379. Name or service not known.
[2024-03-15 02:07:13,005] [PID 9] [Thread-4] [dramatiq.worker.ConsumerThread(default.DQ)] [CRITICAL] Consumer encountered a connection error: Error -2 connecting to redis:6379. Name or service not known.

[docker-compose-mediafusion-1]

  File "/mediafusion/db/crud.py", line 132, in get_cached_torrent_streams
    cached_data = await redis.get(cache_key)
  File "/root/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.11/site-packages/redis/asyncio/client.py", line 605, in execute_command
    conn = self.connection or await pool.get_connection(command_name, **options)
  File "/root/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.11/site-packages/redis/asyncio/connection.py", line 1076, in get_connection
  File "/root/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.11/site-packages/redis/asyncio/connection.py", line 1109, in ensure_connection
    await connection.connect()
  File "/root/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.11/site-packages/redis/asyncio/connection.py", line 282, in connect
    raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error -2 connecting to redis:6379. -2.
ericvlog commented 3 months ago

Finally I get it running well, the redis I set is incorrect.

WARNING::15-Mar-24 08:52:57 - Timeout exceeded for operation: scrap_movies_streams_from_prowlarr ('tt6723592', 'Tenet', 2020). Skipping.
INFO::15-Mar-24 08:52:48 - Found 205 streams for Tenet (2020) with IMDb ID
INFO::15-Mar-24 08:52:53 - Created movies stream 63ffcbdde3602b20453f320aae3f5c41340e0adc for tt6723592
INFO::15-Mar-24 08:52:53 - Created movies stream bf301d0ab6cd523f3dde9253da57b807376f7d15 for tt6723592

It stated my container mediafusion unhealthy due to this 

/root/.local/share/virtualenvs/mediafusion-ZmKbvzsi/src/cinemagoer/imdb/utils.py:990: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if beginTag is "":

Anyhow it can function now.

Btw It can scraper torrentio? I have set to true, but how I know it working?

mhdzumair commented 3 months ago

It stated my container mediafusion unhealthy due to this

No the log message you mention is not an error, its a warning in one of the library im using. You can discarded it.

Btw It can scraper torrentio? I have set to true, but how I know it working?

If you setup the config and url correctly, the scraper should work. Its only scrap the movie/series you click on the stremio. To validate whether its working, you can open any movie/series and then wait to return the streams data. It will show the streams and source as Torrentio/KingtCrowler based on the url you're configured.

Just ask this addons is based on prowlarr izzit? So I need delete prowlarr indexer less for more efficient. So it no scraper data when idle? or just scraper prowlarr/torrentio when request.

Mediafusion has several scrapers: prowlarr and torrenito scrapers are scrape based on the movie/series you're clicking in stremio. tamilblaster, tamilmv scraping scripts, mhtvworld, mhdtvsports, formula scrapy spiders are scheduler based scraping. You can see the more details on configurations doc. https://github.com/mhdzumair/MediaFusion/blob/main/docs/configuration.md#mediafusion-environment-configuration-guide

ericvlog commented 3 months ago

Stunner addons, with a lot of debrid services intergrade, I tested on PikPak it run very well.

Again, thanks for amazing addons.

mhdzumair commented 3 months ago

Closing as done