Stopping lbrynet fails on full disk

The Issue

Trying to stop the daemon when the disk is full fails. Trying to stop it using systemd also leaves a single process running which can only be solved with kill -9 Lbrynet seems to be using disk space at runtime?

Steps to reproduce

fill the disk
run brynet stop
when it fails, check disk space
forcefully terminate the process
watch disk space again

Expected behaviour

The daemon stops

Actual behaviour

./lbrynet stop [15609] INTERNAL ERROR: cannot create temporary directory!

$ sudo service lbrynet stop
$ ps x
  PID TTY      STAT   TIME COMMAND
...
14500 ?        Sl    45:37 /opt/lbry/lbrynet start
15964 pts/0    R+     0:00 ps x

System Configuration

{
  "id": null,
  "jsonrpc": "2.0",
  "result": {
    "build": "release",
    "desktop": "Unknown",
    "distro": {
      "codename": "Xenial Xerus",
      "id": "ubuntu",
      "like": "debian",
      "version": "16.04",
      "version_parts": {
        "build_number": "",
        "major": "16",
        "minor": "04"
      }
    },
    "lbrynet_version": "0.30.3",
    "lbryschema_version": "0.0.16",
    "os_release": "4.4.0-138-generic",
    "os_system": "Linux",
    "platform": "Linux-4.4.0-138-generic-x86_64-with-debian-stretch-sid",
    "processor": "x86_64",
    "python_version": "3.7.1"
  }
}

Anything Else

It seems that when I force stop the daemon, in some cases, a lot of space is freed up. Why is that?

[lbry@whatever:~] $ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        78G   78G     0 100% /

[lbry@whatever:~] $ sudo service lbrynet stop

... out of the 2 lbrynet processes only 1 terminates, so one needs to be force killed
[lbry@whatever:~] $ kill -9 14500

[lbry@whatever:~] $ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        78G   49G   29G  64% /

Internal Use

Acceptance Criteria

1. 2. 3.

Definition of Done

[ ] Tested against acceptance criteria
[ ] Tested against the assumptions of user story
[ ] The project builds without errors
[ ] Unit tests are written and passing
[ ] Tests on devices/browsers listed in the issue have passed
[ ] QA performed & issues resolved
[ ] Refactoring completed
[ ] Any configuration or build changes documented
[ ] Documentation updated
[ ] Peer Code Review performed

lbryio / lbry-sdk

Stopping lbrynet fails on full disk #1738