cmmr / virmap

GNU Affero General Public License v3.0
25 stars 14 forks source link

How do I create Database #12

Open yeoeunyun opened 4 years ago

yeoeunyun commented 4 years ago

Hi, I gonna create database but I got some problems.

here is my command log.

.
.
.
rm: cannot remove '/tmp/eMRJikpDGC/gbpln214.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpat58.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpat58.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbvrt29.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbvrt29.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbvrt35.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbvrt35.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbbct22.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbbct22.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpat130.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpat130.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpat52.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpat52.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpln125.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpln125.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpat136.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpat136.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbenv15.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbenv15.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbbct93.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbbct93.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbbct394.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbbct394.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon92.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon92.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbenv26.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbenv26.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpri21.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpri21.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbmam45.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbmam45.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbbct393.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbbct393.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon37.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon37.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon117.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon117.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpat186.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpat186.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpat15.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpat15.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon20.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon20.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbmam50.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbmam50.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbvrt14.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbvrt14.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpln173.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpln173.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon175.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon175.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpat117.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpat117.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbbct392.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbbct392.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbvrt42.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbvrt42.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbvrt28.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbvrt28.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon4.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon4.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbmam49.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbmam49.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpln170.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpln170.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpln171.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpln171.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbrod11.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbrod11.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon118.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon118.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbsyn28.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbsyn28.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon30.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon30.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon143.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon143.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon195.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon195.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon210.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon210.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon137.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbcon137.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbmam13.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbmam13.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbuna1.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbuna1.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbhtg6.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbhtg6.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpln100.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpln100.seq.gz.nuc.fna.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpln99.seq.gz.aa.faa.zst': No such file or directory
rm: cannot remove '/tmp/eMRJikpDGC/gbpln99.seq.gz.nuc.fna.zst': No such file or directory
Finished building virDmnd.dmnd
Finished building virBbmap
Finished building gbBlastx.dmnd
sh: 1: cannot create /scratch/VirmapDb/Taxonomy.virmap: Permission denied
sh: 1: makeStruct.pl: not found
Finished building taxonomy structure
print() on closed filehandle CS at /home/minsookim/msk/Virmap/virmap-master/DatabaseConstructor/makeVirmapDb.pl line 379.
Finished building all virmap databases

(I just bring a part of the command log because there were too much lines.)

I wonder if this problem is because my system is Ubuntu, not Amazon Linux 2 OR I just missed some steps that you give.

Could you give me any advice?

Thanks in advance Yun

torptube commented 4 years ago

Did you create and mount an SSD filesystem onto /scratch and them set TMPDIR to some location on /scratch?

The installer has only been tested on Amazon Linux 2, and only on a fresh machine. There is code to support Debian, but it's not extensively tested. Regardless, you need to be in an AWS environment and on a machine with an SSD instance store. I would recommend at least 400GB on the instance store.

I haven't tested it on a pre-existing linux system. The install script makes a lot of assumptions on binaries and resources that are pre-installed on an Amazon Linux 2 image.

It also looks line your install isn't complete as not all the scripts installed.

What set of commands did you run to get to this point?

torptube commented 4 years ago

OK, I tested super quickly on a fresh ubuntu system, there was an error in the instruction that told you to set TMPDIR after it was already expected to have been set. I put the instruction in the proper spot.

yeoeunyun commented 4 years ago

Thank you for your response.

I downloaded the files you provided, and followed the steps in README

First, I ran these commands as below, and there were not any problems yet.

chmod 0775 virmapInstall.pl
./virmapInstall.pl

export TMPDIR=/scratch/tmp

But, when I ran this command, we got the problems about dependencies.

makeVirmapDb.pl --outputDir /scratch/VirmapDb

So, I installed some dependencies required in makeVirmapDb.pl and also Virmap.pl.

After that, I ran the Virmap.pl to make sure it was installed properly and there were no problems with it. (I mean I could see the usage)

I tried to create database again, using 'makeVirmapDb.pl', but I got that command log as above, and no files were created.

This is what I'm in right now.

I just wonder that there is the other way to use VirMap in Ubuntu.

Thanks in advance.

Yun

torptube commented 4 years ago

Are you running this on a fresh Ubuntu 18.04 image on AWS?

It looks to me like you may be trying to install this on a pre-existing Ubuntu set up (your username would be "ubuntu' on a vanilla AWS image). That scenario has not been tested and is unlikely to work, as the install script assumes certain AWS environment features and also assumes sudo with no password.

yeoeunyun commented 4 years ago

Okay, I got it.

Thank you for your reply.