Only getting 40Mbit-ish on initial caching

BenWoodford commented 4 years ago

Describe the issue you are having

Admittedly I am still on the old steamcache-steam container, but before I go through the process of updating I wanted to see if I was missing something first.

Speeds when downloading a cached game are great. Easy 500Mbps when it gets up to speed. However, the initial speeds I'm getting for an uncached game on our 750Mbit line are a mere 45Mbit. Which isn't ideal.

I'd like to not kill steamcache, as it will be useful to have in the office (albeit it's a small team), so I'd like to try and resolve that one.

Is this just a case of write speeds not being sufficient, or should they not affect the speed of the download?

How are you running the container(s)?

version: '2'
services:
 steamcache-steam:
   container_name: steamcache-steam
   hostname: steam.cache.domain.ext
   image: steamcache/steamcache:latest
   restart: unless-stopped
   volumes:
     - "/mnt/filestore/caches/steam/:/data/cache"
     - "/mnt/filestore/logs/caches/steam:/data/logs"
   ports:
     - '80:80'
   environment:
     - TZ=Europe/London
   networks:
     default:
       ipv4_address: 10.10.3.1
networks:
  default:
    external:
      name: macvlan-nat

DNS Configuration

version: '2'
services:
 steamcache-dns:
   container_name: steamcache-dns
   image: steamcache/steamcache-dns:latest
   restart: unless-stopped
   environment:
     - TZ=Europe/London
     - UPSTREAM_DNS=8.8.8.8
     - STEAMCACHE_IP=10.10.3.1
   networks:
     default:
       ipv4_address: 10.10.1.2
networks:
  default:
    external:
      name: macvlan-nat

fhibler commented 4 years ago

+1 here. Only about 10 Mbps for initial priming vs ~100 Mbps for direct connection. Tested for Blizzard/Steam

Fresh and clean install of lancachenet/monolithic

Lepidopterist commented 4 years ago

Steam in particular seems to be strictly adhering to the HTTP spec on concurrent connections per upstream IP since a patch about 6 months ago.

We've documented a workaround in the main readme here: https://github.com/lancachenet/monolithic#tuning-your-cache. Whilst the instructions do not /directly/ apply to steamcache, the same principle should be able to be used.

@fhibler , these instructions should apply directly to monolithic.

We've seen varying levels of success reported, so it will involve a bit of trial and error to determine the best number of IP's to add for your specific conditions.

(This is assuming you are not IO bound, which also a possibility. Use system tools like iotop / dstat / top to determine this on your setup)

BenWoodford commented 4 years ago

Any idea how to get that working with macvlan? Multiple macvlan interfaces with static IPs aren't really something I've managed to do before so it's a bit weird.

Why are we using macvlan? A lot of potential port collisions essentially. Just easier to have everything on its own IP

MattzFiber commented 4 years ago

Been fiddling with this for my entire weekend, and I cant get my cache to give me more than 10-30mbps on my 100mbps pipe. Tried different VMs and hardware, even set up NGINX without any Docker, and get the results were all the same. Used this cache at an event a few weeks back, and didn't have a single issue then without having to add extra IPs. Did Steam really break this somehow? I recall Steam downloads pumping though even MultiPlays configs years back, now, it seems broken and no one 100% knows why. Throwing IPs at the problem might not be a sustainable approach.

soja92 commented 4 years ago

Increasing the slice size from 1m to 16m solved the slow initial cache problem for me. With slice size set to 1m I was seeing 40-50Mb/s and at 16m I get 250-300Mb/s. I would only change this if your internet connection can support quickly filling the slice. The option is in the config file located at nginx/sites-available/generic.conf.d/root/20_cache.conf

unspec commented 4 years ago

Regarding slow initial blizzard downloads:

The latest version of monolithic/generic now supports changing the slice size used by nginx. We've found that increasing from 1m to 8m offers a small performance boost to specific use cases (single user initial downloads of blizzard games in particular). See http://lancache.net/docs/advanced/tuning-cache/#tweaking-slice-size for information on how to make use of this.

Please note that it does come with some potential downsides (discussed in the above link) and will invalidate any already cached data on your cache if you change the value.

To tidy up the issues, if you choose to test this please post any feedback on this issue: https://github.com/lancachenet/generic/issues/100

If you need any other support please see http://lancache.net/support/ or open a new issue.

lancachenet / monolithic