Does not download files to disk

Cronus89 commented 5 years ago

Describe the issue you are having

Moved from generic to this, cant get to work, resolves IP's correctly but doesnt seem to save any data to disk I get the below error messages in error.log  I see it creates the folders as the www-data user which doesnt exist on my docker host, except under uid 33 (default debian users?)

they are all chmod'd 777 however.  it is using a NFS Share on a NAS 

2018/12/20 03:24:13 [crit] 192#192: *35 open() "/data/cache/cache/99/ac/57f4a2d49f8869000a7ded344cc4ac99" failed (13: Permission denied), client: 10.0.0.114, server: , request: "GET /serverlist/77/20/ HTTP/1.1", host: "valve325.steamcontent.com"
2018/12/20 03:24:13 [crit] 193#193: *36 open() "/data/cache/cache/99/ac/57f4a2d49f8869000a7ded344cc4ac99" failed (13: Permission denied), client: 10.0.0.102, server: , request: "GET /serverlist/77/20/ HTTP/1.1", host: "valve2300.steamcontent.com"

How are you running the container(s)?

version: '2.4'
services:
    sniproxy:
        image: steamcache/sniproxy
        container_name: sniproxy
        restart: unless-stopped
        environment:
            - TZ=America/Chicago
        ports:
            - "10.0.0.82:443:443"
            - "10.0.0.83:443:443"
        mem_limit: 20M
    cache-monolithic:
        image: steamcache/monolithic
        container_name: cache-monolithic
        restart: unless-stopped
        depends_on:
            - sniproxy
        volumes:
            - /mnt/cache/data:/data/cache
            - /mnt/cache/logs:/data/logs
        environment:
            - TZ=America/Chicago
            - PGID=1000
            - PUID=1000
        ports:
            - "10.0.0.82:80:80"
        mem_limit: 1024M
        dns:
            - 1.1.1.1
            - 9.9.9.9

DNS Configuration

Output of container(s)

root@docker:~# docker-compose up cache-monolithic
sniproxy is up-to-date
Starting cache-monolithic ... done
Attaching to cache-monolithic
cache-monolithic    | Executing hook /hooks/entrypoint-pre.d/10_setup.sh
cache-monolithic    | Executing hook /hooks/entrypoint-pre.d/15_generate_maps.sh
cache-monolithic    | HEAD is now at 63a12a6 Merge pull request #47 from rotanid/wargaming-fixes
cache-monolithic    | Reading cache apple from apple.txt
cache-monolithic    | Reading cache arenanet from arenanet.txt
cache-monolithic    | Reading cache blizzard from blizzard.txt
cache-monolithic    | Reading cache daybreak from daybreak.txt
cache-monolithic    | Reading cache frontier from frontier.txt
cache-monolithic    | Reading cache hirez from hirez.txt
cache-monolithic    | Reading cache nexusmods from nexusmods.txt
cache-monolithic    | Reading cache nintendo from nintendo.txt
cache-monolithic    | Reading cache origin from origin.txt
cache-monolithic    | Reading cache renegadex from renegadex.txt
cache-monolithic    | Reading cache riot from riot.txt
cache-monolithic    | Reading cache rockstar from rockstar.txt
cache-monolithic    | Reading cache sony from sony.txt
cache-monolithic    | Reading cache steam from steam.txt
cache-monolithic    | Reading cache uplay from uplay.txt
cache-monolithic    | Reading cache twitch from twitchapp.txt
cache-monolithic    | Reading cache wargaming from wargaming.net.txt
cache-monolithic    | Reading cache wsus from windowsupdates.txt
cache-monolithic    | Reading cache xboxlive from xboxlive.txt
cache-monolithic    | map $http_host $cacheidentifier {
cache-monolithic    |     hostnames;
cache-monolithic    |     default $http_host;
cache-monolithic    |     assetcdn.101.arenanetworks.com arenanet;
cache-monolithic    |     assetcdn.102.arenanetworks.com arenanet;
cache-monolithic    |     assetcdn.103.arenanetworks.com arenanet;
cache-monolithic    |     dist.blizzard.com blizzard;
cache-monolithic    |     dist.blizzard.com.edgesuite.net blizzard;
cache-monolithic    |     llnw.blizzard.com blizzard;
cache-monolithic    |     edgecast.blizzard.com blizzard;
cache-monolithic    |     blizzard.vo.llnwd.net blizzard;
cache-monolithic    |     blzddist1-a.akamaihd.net blizzard;
cache-monolithic    |     blzddist2-a.akamaihd.net blizzard;
cache-monolithic    |     blzddist3-a.akamaihd.net blizzard;
cache-monolithic    |     blzddist4-a.akamaihd.net blizzard;
cache-monolithic    |     level3.blizzard.com blizzard;
cache-monolithic    |     nydus.battle.net blizzard;
cache-monolithic    |     edge.blizzard.top.comcast.net blizzard;
cache-monolithic    |     cdn.blizzard.com blizzard;
cache-monolithic    |     *.cdn.blizzard.com blizzard;
cache-monolithic    |     # Daybreak games daybreak;
cache-monolithic    |     # daybreak;
cache-monolithic    |     # Do NOT cache manifest.patch.daybreakgames.com daybreak;
cache-monolithic    |     # daybreak;
cache-monolithic    |     #PS2 daybreak;
cache-monolithic    |     pls.patch.daybreakgames.com daybreak;
cache-monolithic    |     ccs.cdn.wup.shop.nintendo.com nintendo;
cache-monolithic    |     pushmo.hac.lp1.eshop.nintendo.net nintendo;
cache-monolithic    |     ecs-lp1.hac.shop.nintendo.net nintendo;
cache-monolithic    |     receive-lp1.dg.srv.nintendo.net nintendo;
cache-monolithic    |     aqua.hac.lp1.d4c.nintendo.net nintendo;
cache-monolithic    |     atum.hac.lp1.d4c.nintendo.net nintendo;
cache-monolithic    |     bugyo.hac.lp1.eshop.nintendo.net nintendo;
cache-monolithic    |     tagaya.hac.lp1.eshop.nintendo.net nintendo;
cache-monolithic    |     # WARNING:  Origin has been seen downloading https client downloads on origin-a.akamaihd.net.  A solution should be in place to forward https to the origin server (eg sniproxy) origin;
cache-monolithic    |     origin-a.akamaihd.net origin;
cache-monolithic    |     akamai.cdn.ea.com origin;
cache-monolithic    |     lvlt.cdn.ea.com origin;
cache-monolithic    |     river.data.ea.com origin;
cache-monolithic    |     origin-a.akamaihd.net.edgesuite.net origin;
cache-monolithic    |     rxp-fl.cncirc.net renegadex;
cache-monolithic    |     rxp-chi.cncirc.net renegadex;
cache-monolithic    |     rxp-nz.cncirc.net renegadex;
cache-monolithic    |     rxp-bgr.cncirc.net renegadex;
cache-monolithic    |     rxp-fr.cncirc.net renegadex;
cache-monolithic    |     rxp-nyc.cncirc.net renegadex;
cache-monolithic    |     rxp-uk.cncirc.net renegadex;
cache-monolithic    |     rxp-sg.cncirc.net renegadex;
cache-monolithic    |     rxp-la.cncirc.net renegadex;
cache-monolithic    |     rxp-fin.cncirc.net renegadex;
cache-monolithic    |     denver1.renegade-x.com renegadex;
cache-monolithic    |     l3cdn.riotgames.com riot;
cache-monolithic    |     worldwide.l3cdn.riotgames.com riot;
cache-monolithic    |     riotgamespatcher-a.akamaihd.net riot;
cache-monolithic    |     riotgamespatcher-a.akamaihd.net.edgesuite.net riot;
cache-monolithic    |     pls.patch.station.sony.com sony;
cache-monolithic    |     gs2.ww.prod.dl.playstation.net sony;
cache-monolithic    |     *.content.steampowered.com steam;
cache-monolithic    |     content1.steampowered.com steam;
cache-monolithic    |     content2.steampowered.com steam;
cache-monolithic    |     content3.steampowered.com steam;
cache-monolithic    |     content4.steampowered.com steam;
cache-monolithic    |     content5.steampowered.com steam;
cache-monolithic    |     content6.steampowered.com steam;
cache-monolithic    |     content7.steampowered.com steam;
cache-monolithic    |     content8.steampowered.com steam;
cache-monolithic    |     cs.steampowered.com steam;
cache-monolithic    |     steamcontent.com steam;
cache-monolithic    |     client-download.steampowered.com steam;
cache-monolithic    |     *.hsar.steampowered.com.edgesuite.net steam;
cache-monolithic    |     *.akamai.steamstatic.com steam;
cache-monolithic    |     content-origin.steampowered.com steam;
cache-monolithic    |     clientconfig.akamai.steamtransparent.com steam;
cache-monolithic    |     steampipe.akamaized.net steam;
cache-monolithic    |     edgecast.steamstatic.com steam;
cache-monolithic    |     steam.apac.qtlglb.com.mwcloudcdn.com steam;
cache-monolithic    |     *.cs.steampowered.com steam;
cache-monolithic    |     *.edgecast.steamstatic.com steam;
cache-monolithic    |     *.steamcontent.com steam;
cache-monolithic    |     cdn1-sea1.valve.net steam;
cache-monolithic    |     cdn2-sea1.valve.net steam;
cache-monolithic    |     *.steam-content-dnld-1.apac-1-cdn.cqloud.com steam;
cache-monolithic    |     steam.apac.qtlglb.com steam;
cache-monolithic    |     edge.steam-dns.top.comcast.net steam;
cache-monolithic    |     edge.steam-dns-2.top.comcast.net steam;
cache-monolithic    |     steamcdn-a.akamaihd.net steam;
cache-monolithic    |     steam.naeu.qtlglb.com steam;
cache-monolithic    |     steampipe-kr.akamaized.net steam;
cache-monolithic    |     steam.ix.asn.au steam;
cache-monolithic    |     steam.eca.qtlglb.com steam;
cache-monolithic    |     *.cdn.ubi.com uplay;
cache-monolithic    |     d3rmjivj4k4f0t.cloudfront.net twitch;
cache-monolithic    |     addons.forgesvc.net twitch;
cache-monolithic    |     media.forgecdn.net twitch;
cache-monolithic    |     files.forgecdn.net twitch;
cache-monolithic    |     dl1.wargaming.net wargaming;
cache-monolithic    |     dl2.wargaming.net wargaming;
cache-monolithic    |     wg.gcdn.co wargaming;
cache-monolithic    |     wgusst-na.wargaming.net wargaming;
cache-monolithic    |     wgusst-eu.wargaming.net wargaming;
cache-monolithic    |     update-v4r4h10x.worldofwarships.com wargaming;
cache-monolithic    |     dl-wows-ak.wargaming.net wargaming;
cache-monolithic    |     wgus-wotasia.wargaming.net wargaming;
cache-monolithic    |     officecdn.microsoft.com wsus;
cache-monolithic    |     *.windowsupdate.com wsus;
cache-monolithic    |     windowsupdate.com wsus;
cache-monolithic    |     *.dl.delivery.mp.microsoft.com wsus;
cache-monolithic    |     dl.delivery.mp.microsoft.com wsus;
cache-monolithic    |     *.update.microsoft.com wsus;
cache-monolithic    |     *.do.dsp.mp.microsoft.com wsus;
cache-monolithic    |     *.microsoft.com.edgesuite.net wsus;
cache-monolithic    |     assets1.xboxlive.com xboxlive;
cache-monolithic    |     assets2.xboxlive.com xboxlive;
cache-monolithic    |     dlassets.xboxlive.com xboxlive;
cache-monolithic    |     xboxone.loris.llnwd.net xboxlive;
cache-monolithic    |     *.xboxone.loris.llnwd.net xboxlive;
cache-monolithic    |     xboxone.vo.llnwd.net xboxlive;
cache-monolithic    |     images-eds.xboxlive.com xboxlive;
cache-monolithic    |     xbox-mbr.xboxlive.com xboxlive;
cache-monolithic    | }
cache-monolithic    | Executing hook /hooks/entrypoint-pre.d/19_doc_root_setup
cache-monolithic    | /var/www/html already exists.
cache-monolithic    | Setting document root to /var/www/html
cache-monolithic    | Executing hook /hooks/entrypoint-pre.d/20_perms_check.sh
cache-monolithic    | Checking permissions (This may take a long time if the permissions are incorrect on large caches)...
cache-monolithic    | Permisions ok
cache-monolithic    | Executing hook /hooks/entrypoint-pre.d/20_ssl_setup
cache-monolithic    | Not enabling SSL as neither key nor cert provided.
cache-monolithic    | Executing hook /hooks/supervisord-pre.d/20_test_files_setup
cache-monolithic    | Checking if /var/www/html is empty - Directory not empty.. don't touch content
cache-monolithic    | Executing hook /hooks/supervisord-pre.d/21_cleanup_log_files
cache-monolithic    | Cleaning up log files older than 3560 days
cache-monolithic    | Executing hook /hooks/supervisord-pre.d/99_config_check.sh
cache-monolithic    | Checking nginx config
cache-monolithic    | nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
cache-monolithic    | nginx: configuration file /etc/nginx/nginx.conf test is successful
cache-monolithic    | Config check successful
cache-monolithic    | 2018-12-20 03:24:11,149 CRIT Set uid to user 0

VibroAxe commented 5 years ago

Interesting as this uses generic's cache engine under the box and others have it working fine, so probably a permissions error of some forms, lets see if we can debug it

can you try the following in a docker exec -it cache-monolithic bash

cat /data/logs/access.log
cat /data/logs/error.log
ls -al /data/cache
ls -al /data/cache/cache
touch /data/cache/test
touch /data/cache/cache/test
ls -al /data/cache
ls -al /data/cache/cache

And for good measure try the following on the host machine

ls -al /mnt/cache/data
touch /mnt/cache/data/test_host
ls -al /mnt/cache/data

Cronus89 commented 5 years ago

All logs were readable, output of the others look great, see below

root@83a1eab75562:/scripts# ls -al /data/cache
total 12
drwxrwxrwx 3 www-data www-data 4096 Dec 20 03:07 .
drwxr-xr-x 1 www-data www-data 4096 Dec 19 21:58 ..
drwxrwxrwx 2 www-data www-data 4096 Dec 20 03:07 cache
root@83a1eab75562:/scripts# ls -al /data/cache/cache
total 8
drwxrwxrwx 2 www-data www-data 4096 Dec 20 03:07 .
drwxrwxrwx 3 www-data www-data 4096 Dec 20 03:07 ..
root@83a1eab75562:/scripts# touch /data/cache/test
root@83a1eab75562:/scripts# touch /data/cache/cache/test
root@83a1eab75562:/scripts# ls -al /data/cache
total 12
drwxrwxrwx 3 www-data www-data 4096 Dec 22 05:10 .
drwxr-xr-x 1 www-data www-data 4096 Dec 19 21:58 ..
drwxrwxrwx 2 www-data www-data 4096 Dec 22 05:10 cache
-rwxrwxrwx 1 root     root        0 Dec 22 05:10 test
root@83a1eab75562:/scripts# ls -al /data/cache/cache
total 8
drwxrwxrwx 2 www-data www-data 4096 Dec 22 05:10 .
drwxrwxrwx 3 www-data www-data 4096 Dec 22 05:10 ..
-rwxrwxrwx 1 root     root        0 Dec 22 05:10 test

root@docker:~# ls -al /mnt/cache/data
total 16
drwxrwxrwx 3 www-data www-data 4096 Dec 21 23:10 .
drwxrwxrwx 5 deploy   deploy   4096 Dec 19 21:13 ..
drwxrwxrwx 2 www-data www-data 4096 Dec 21 23:10 cache
-rwxrwxrwx 1 root     root        0 Dec 21 23:10 test
root@docker:~# touch /mnt/cache/data/test_host
root@docker:~#
root@docker:~# ls -al /mnt/cache/data
total 16
drwxrwxrwx 3 www-data www-data 4096 Dec 21 23:13 .
drwxrwxrwx 5 deploy   deploy   4096 Dec 19 21:13 ..
drwxrwxrwx 2 www-data www-data 4096 Dec 21 23:10 cache
-rwxrwxrwx 1 root     root        0 Dec 21 23:10 test
-rwxrwxrwx 1 root     root        0 Dec 21 23:13 test_host
root@docker:~#

VibroAxe commented 5 years ago

Sorry @cronus89 I missed your reply to this in the Christmas rush. I'll see if I can have another think in the morning

VibroAxe commented 5 years ago

@Cronus89 Did you manage to find any solution to this, I cannot replicate this at all, even using another nfs share in the same manor you you appear to be using it. Can you try with the cache directories pointing to a local disk and see if you still have the same issues?

VibroAxe commented 5 years ago

Only other test I can think of is the following (we've only tested as the root user and your user in both cases. Again inside the docker container docker exec -it cache-monolithic bash:

su -s /bin/bash www-data 
ls -al /data/cache
ls -al /data/cache/cache
touch /data/cache/test_user
touch /data/cache/cache/test_user
mkdir -p /data/cache/cache/testdir
touch /data/cache/cache/testdir/test_user
ls -al /data/cache
ls -al /data/cache/cache
ls -al /data/cache/cache/testdir

Cronus89 commented 5 years ago

I ended up removing the nfs mount and it worked fine.

So its the mount somehow the issue.

I've tested your new command ideas, and the 2nd one fails on, is I cannot see the contents of /data/cache

I can see /data/ fine and also /data/cachedomains

The NFS Server is a Synology NAS and i have another nfs mount on this same VM that works fine (plex video files) Plex is also a docker container.

I'm lost as to why this one wouldnt work.

Cronus89 commented 5 years ago

Got it!....

Had to squash root to admin :\ Thanks for all the helpful commands. Love this new monolithic container!

VibroAxe commented 5 years ago

I didn't suggest root squash as I thought you had generic working in the same context. Also I'm slightly surprised the debug commands worked with root squash to nobody, but hey ho!

Glad to have got you sorted ;)

PazifistRules commented 5 years ago

Sorry for jumping on this old topic, but:

Got it!....

Had to squash root to admin :\ Thanks for all the helpful commands. Love this new monolithic container!

@Cronus89 Can you describe how exactly you got the container to work on a NFS share? (For a Linux and Docker Beginner, please: Full commands for the share mount and the container start please.)

I'm having exactly the same issue with a shared volume on a Synology Server. The logs write fine, the folder structure in cache creates fine, but no files are stored. I cannot seem to fix it, no matter SMB or NFS share. Using the cache with local storage works just fine.

lacion commented 5 years ago

i know this is an old issue, but i faced this while having an smb share and a docker container.

i fixed the issue by changing the WEBUSER to root.

VibroAxe commented 5 years ago

@lacion please be aware that is a terrible idea and incredibly bad security practice. Lancache do not recommend this course of action at all

lacion commented 5 years ago

@VibroAxe that is absolutely for a prod environment.

but my steam cache is part of a private VLAN not accessible to the outside world. so it works fine. (this is a DNS server on my home network not accessible externally.)

ideally, the user being run in the container should be able to use a custom UID and GID so we could match it to permissions on a user in the host. having to avoid running anything in root or just chmod 777 or 775 which is also a really bad idea.

but for now this works, i wish i had time to do something more or fork and PR... but i just blew out all my free time of the week trying to get this to work.

lancachenet / monolithic