proycon / LaMachine

LaMachine - A software distribution of our in-house as well as some 3rd party NLP software - Virtual Machine, Docker, or local compilation/installation script
https://proycon.github.io/LaMachine
GNU General Public License v3.0
68 stars 20 forks source link

Out of space when adding tscan to Lamachine on Windows #148

Open pboot opened 5 years ago

pboot commented 5 years ago

I installed LaMachine on Windows 10, with some bumps along the way, but it runs. I can e.g. use Frog as a webservice. Now I want to add Tscan, which is what I really want to use.

I do this: lamachine-add tscan lamachine-update

then after many other messages:

Cannot write to ‘latest.tar.gz’ (No space left on device). MSG: non-zero return code PLAY RECAP *** stable : ok=857 changed=285 unreachable=0 failed=1 skipped=813 rescued=0 ignored=1

The disk is indeed full:

vagrant@stable:/usr/local/src/LaMachine$ df -h Filesystem Size Used Avail Use% Mounted on udev 992M 0 992M 0% /dev tmpfs 201M 6.8M 194M 4% /run /dev/sda1 8.7G 8.3G 1.5M 100% / tmpfs 1003M 0 1003M 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 1003M 0 1003M 0% /sys/fs/cgroup tmpfs 201M 0 201M 0% /run/user/1000

I have tried to increase the disk size by running vagrant plugin install vagrant-disksize and adding config.disksize.size = '50GB' in the vagrant file

This has however no effect on the disk size.

I suppose this is a beginner's question, but how should I go about increasing the available space?

P.S. I should have attached the update's logfile to this issue, but I'm afraid I don't know how to get that out of the virtual machine. I tried scp vagrant@127.0.0.1:2222:/usr/local/src/LaMachine/lamachine-stable-20190619_153529.log d:/even The answer I got was ssh: connect to host 127.0.0.1 port 22: Connection refused

kosloot commented 5 years ago

Well, it is up to @proycon to get into the gory LaMachine details, but wouldn't it be much easier to use the Tscan service @ru?

To cite the Tscan README:

T-Scan is available as a web application and webservice through https://webservices-lst.science.ru.nl , register for a (free) account there first, and then access T-Scan through https://webservices-lst.science.ru.nl/tscan/ .

proycon commented 5 years ago

When you install a custom LaMachine, you can opt for extra diskspace during the bootstrap process, this is indeed needed when installing certain extra software. Increasing the disksize as you did with the vagrant plugin is possible too, but requires you to also resize the partition manually (using parted for example).

pboot commented 5 years ago

Well, it is up to @proycon to get into the gory LaMachine details, but wouldn't it be much easier to use the Tscan service @ru?

The texts I have are confidential, so I am somewhat reluctant to send them over the web and put them on a server that I have no control over. Another reason is that the analysis there takes very long and I wondered, perhaps naively, whether my own machine might be faster (or at least could process part of the large volume of text in parallel to the web service).

I will try parted.

pboot commented 5 years ago

I tried parted but the other partitions are in the way:

(parted) resizepart Partition number? 1 Warning: Partition /dev/sda1 is being used. Are you sure you want to continue? Yes/No? y End? [9559MB]? 45000 Error: Can't have overlapping partitions.

Also, you're not supposed to use resizepart on a partition in use.

For now I give up and will try a Linux workstation.

proycon commented 5 years ago

I'm reopening this as it does address an important issue that needs fixing.

The LaMachine bootstrap procedure has functionality for adding extra disk space when building a LaMachine image, and this option is preferred to attempting to resize the partition yourself (because that is far from trivial as the testimony in this issue confirms).

This extra diskspace option LaMachine provides for the VM, however, requires users to actually go through the bootstrap procedure to build their custom LaMachine image rather than select the pre-built option (which is too small), but the bootstrap procedure does not run on Windows!. Windows users aiming for the VM are always recommended to use a pre-built option (sorry I overlooked this in my earlier comment).

So this currently poses a bit serious problem for Windows users who want to install more than the default VM image allows, and a solution will need to be found. (but porting the whole bootstrap procedure to Windows is something I want to desperately avoid as it is a platform I'm not very familiar with)

pboot commented 5 years ago

One option would be to run the bootstrap procedure on a Linux box in VirtualBox. That is what I set out to do, but then I decided to sort out the issues that I met in a native environment first.

proycon commented 5 years ago

In the above mentioned issue I hint at another workaround, but that requires some linux administration expertise. Creating the VM on a linux machine and then copying the images (plural because of the extra disk!) to a Windows system would indeed be a viable strategy as well.

I don't think there will be an easier solution for the Windows images for now.