os.remove error

[dstern]

I have installed JustOrthologs from github and tried to run the wrapper with recommended command from inside JustOrthologs folder. Throws error from line 240. I have tried this on two machines with independent installs of python 2.7 and get same error.

JustOrthologs sternd$ python2.7 wrapper.py -g1 smallTest/wrapperTest/small_pan.gff3 -g2 smallTest/wrapperTest/small_human.gff3 -r1 smallTest/wrapperTest/small_pan.fasta.gz -r2 smallTest/wrapperTest/small_human.fasta.gz -all -o output Traceback (most recent call last): File "wrapper.py", line 240, in os.remove(extract1) OSError: [Errno 2] No such file or directory: '.extract_1_1605371'

[ridgelab]

Hello, Thank you for your interest in JustOrthologs. I have previously encountered this error when the required python libraries are not installed. From README_WRAPPER, Python libraries that must be installed include:

1. sys
2. os
3. multiprocessing
4. argparse
5. gzip
6. biopython

If biopython is not installed, then one of the files is not created and cannot be removed by os.remove. Thank you for letting me know that you had this issue. Please let me know if installing the libraries (with pip install) solves the issue for you. Then I'll add some error checking in the wrapper script to ensure all python libraries are properly installed so other people don't have the same issue.

[muecker]

Hey @ridgelab , I ran into the same issue as dstern. With the command bash run_multiple_species.sh -i smallTest/wrapperTest/ -o outputDir/ -e output_ortholog_groups -f .fasta.gz -g .gff3 (all scripts adjusted to call python2 instead of just python), I got the following output:

Traceback (most recent call last): File "wrapper.py", line 240, in os.remove(extract1) OSError: [Errno 2] No such file or directory: '.extract_1_2113848' Traceback (most recent call last): File "wrapper.py", line 240, in os.remove(extract1) OSError: [Errno 2] No such file or directory: '.extract_1_1597833' Traceback (most recent call last): File "wrapper.py", line 240, in os.remove(extract1) OSError: [Errno 2] No such file or directory: '.extract_1_1589643' Traceback (most recent call last): File "wrapper.py", line 240, in os.remove(extract1) OSError: [Errno 2] No such file or directory: '.extract_1_5593029' Traceback (most recent call last): File "wrapper.py", line 240, in os.remove(extract1) OSError: [Errno 2] No such file or directory: '.extract_1_3831114' Traceback (most recent call last): File "wrapper.py", line 240, in os.remove(extract1) OSError: [Errno 2] No such file or directory: '.extract_1_1896815'

All required python modules are installed (see list below). To double check, I also tried every import function from your scripts on the python2.7 console and they worked. I tried just the wrapper as well but it throws the same error.

I am not sure what the error could be but I would love to resolve it. Your software seems to be the perfect fit to analyse my data. Do you have an idea what else I could try/look at to troubleshoot?

Best, Merle

---

Python 2.7.15+ (default, Nov 27 2018, 23:36:35) 
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> help ("modules")

Please wait a moment while I gather a list of all available modules...

/usr/lib/python2.7/dist-packages/Bio/GA/__init__.py:14: BiopythonDeprecationWarning: Bio.GA has been deprecated, and we intend to remove it in a future release of Biopython. Please consider using DEAP instead.  If you would like to continue using Bio.GA, please contact the Biopython developers via the mailing list or GitHub.
  BiopythonDeprecationWarning)
/usr/lib/python2.7/dist-packages/Bio/NeuralNetwork/__init__.py:14: BiopythonDeprecationWarning: Bio.NeuralNetwork has been deprecated, and we intend to remove it in a future release of Biopython. Please consider using scikit-learn or TensorFlow instead.  If you would like to continue using Bio.SomeModule, please contact the Biopython developers via the mailing list or GitHub.
  BiopythonDeprecationWarning)
INFO:rdflib:RDFLib Version: 4.2.1
/usr/lib/python2.7/dist-packages/Bio/SearchIO/__init__.py:211: BiopythonExperimentalWarning: Bio.SearchIO is an experimental submodule which may undergo significant changes prior to its future official release.
  BiopythonExperimentalWarning)
/usr/lib/python2.7/dist-packages/Bio/codonalign/__init__.py:27: BiopythonExperimentalWarning: Bio.codonalign is an experimental module which may undergo significant changes prior to its future official release.
  BiopythonExperimentalWarning)
/usr/lib/python2.7/dist-packages/Bio/phenotype/__init__.py:101: BiopythonExperimentalWarning: Bio.phenotype is an experimental submodule which may undergo significant changes prior to its future official release.
  BiopythonExperimentalWarning)
INFO:keyring.backend:Loading Windows (alt)
INFO:keyring.backend:Loading pyfs
INFO:keyring.backend:Loading multi
INFO:keyring.backend:Loading Google
INFO:keyring.backend:Loading Gnome
INFO:keyring.backend:Loading keyczar
INFO:keyring.backend:Loading file
/usr/lib/python2.7/dist-packages/matplotlib/cbook/deprecation.py:106: MatplotlibDeprecationWarning: The mpl_toolkits.axes_grid module was deprecated in version 2.1. Use mpl_toolkits.axes_grid1 and mpl_toolkits.axisartist provies the same functionality instead.
  warnings.warn(message, mplDeprecation, stacklevel=1)
/usr/lib/python2.7/dist-packages/statsmodels/compat/pandas.py:56: FutureWarning: The pandas.core.datetools module is deprecated and will be removed in a future version. Please use the pandas.tseries module instead.
  from pandas.core import datetools
BaseHTTPServer      _threading_local    hotshot             repr
Bastion             _tkinter            html5lib            requests
Bio                 _warnings           htmlentitydefs      resource
CDROM               _weakref            htmllib             rexec
CGIHTTPServer       _weakrefset         httplib             rfc822
Canvas              abc                 idna                rlcompleter
ConfigParser        aifc                ihooks              robotparser
Cookie              antigravity         imaplib             ruamel
Crypto              antlr               imgfuncs            runpy
Cython              anydbm              imghdr              sampletau
DLFCN               argparse            imp                 sched
DbgPlugInDiggers    array               importlib           schema_salad
Dialog              asn1crypto          imputil             scipy
DocXMLRPCServer     ast                 indicator_keyboard  scour
FileDialog          asynchat            inspect             seaborn
FixTk               asyncore            io                  secretstorage
HTMLParser          atexit              ipaddress           select
IN                  atk                 isodate             sets
MimeWriter          attr                itertools           setuptools
OpenSSL             audiodev            jdcal               sgmllib
PIL                 audioop             joblib              sha
PyQt4               avro                json                shellescape
Queue               backports           keyring             shelve
SPARQLWrapper       base64              keyrings            shlex
ScrolledText        bdb                 keyword             shutil
SimpleDialog        binascii            lib2to3             signal
SimpleHTTPServer    binhex              libexiv2python      simplejson
SimpleXMLRPCServer  bisect              libvboxjxpcom       sip
SocketServer        bs4                 linecache           sipconfig
StringIO            bsddb               linuxaudiodev       sipconfig_nd
TYPES               bz2                 locale              site
Tix                 cPickle             lockfile            sitecustomize
Tkconstants         cProfile            logging             six
Tkdnd               cStringIO           lsb_release         smtpd
Tkinter             cachecontrol        lxml                smtplib
UserDict            cairo               macpath             sndhdr
UserList            calendar            macurl2path         socket
UserString          certifi             mailbox             spwd
VBoxAuth            cgi                 mailcap             sqlite3
VBoxAuthSimple      cgitb               markupbase          sre
VBoxDD              chardet             marshal             sre_compile
VBoxDD2             checkm              math                sre_constants
VBoxDDU             chunk               matplotlib          sre_parse
VBoxDbg             cmath               md5                 ssl
VBoxDragAndDropSvc  cmd                 mhlib               stat
VBoxGlobal          code                mimetools           statsmodels
VBoxGuestControlSvc codecs              mimetypes           statvfs
VBoxGuestPropSvc    codeop              mimify              string
VBoxHeadless        collections         mirage              stringold
VBoxHostChannel     colorsys            mistune             stringprep
VBoxKeyboard        commands            mlt                 strop
VBoxNetDHCP         compileall          mmap                struct
VBoxNetNAT          compiler            mock                subprocess
VBoxOGLhostcrutil   contextlib          modulefinder        subprocess32
VBoxOGLhosterrorspu cookielib           monotonic           sunau
VBoxOGLrenderspu    copy                mpl_toolkits        sunaudio
VBoxPython          copy_reg            multifile           symbol
VBoxPython2_7       crypt               multiprocessing     symtable
VBoxREM             cryptography        mutex               sys
VBoxRT              csv                 netrc               sysconfig
VBoxSDL             ctypes              new                 syslog
VBoxSVGA3D          curses              nis                 tables
VBoxSharedClipboard cvxopt              nntplib             tabnanny
VBoxSharedCrOpenGL  cwltool             ntpath              talloc
VBoxSharedFolders   cycler              nturl2path          tarfile
VBoxVMM             cython              numbers             telnetlib
VBoxXPCOM           datetime            numexpr             tempfile
VBoxXPCOMC          dateutil            numpy               termios
VirtualBoxVM        dbhash              olefile             test
_LWPCookieJar       dbm                 opcode              textwrap
_MozillaCookieJar   dbus                openpyxl            this
__builtin__         decimal             operator            thread
__future__          decorator           optparse            threading
_abcoll             dendropy            os                  time
_ast                desman              os2emxpath          timeit
_bisect             difflib             ossaudiodev         tkColorChooser
_bsddb              dircache            pandas              tkCommonDialog
_cffi_backend       dis                 pango               tkFileDialog
_codecs             distutils           pangocairo          tkFont
_codecs_cn          doctest             parser              tkMessageBox
_codecs_hk          dsextras            patsy               tkSimpleDialog
_codecs_iso2022     dumbdbm             pbr                 toaiff
_codecs_jp          dummy_thread        pdb                 token
_codecs_kr          dummy_threading     pickle              tokenize
_codecs_tw          duplicity           pickletools         trace
_collections        easy_install        pip                 traceback
_csv                email               pipes               ttk
_ctypes             enchant             pkg_resources       tty
_ctypes_test        encodings           pkgutil             turtle
_curses             ensurepip           platform            types
_curses_panel       enum                plistlib            typing
_dbus_bindings      errno               pluggy              unicodedata
_dbus_glib_bindings et_xmlfile          popen2              unittest
_elementtree        ete3                poplib              urllib
_functools          exceptions          posix               urllib2
_hashlib            fasteners           posixfile           urllib3
_heapq              fcntl               posixpath           urlparse
_hotshot            filecmp             pprint              user
_io                 fileinput           profile             uu
_json               fnmatch             pstats              uuid
_locale             formatter           pty                 vboxapi
_lsprof             fpformat            pwd                 vboxshell
_md5                fractions           py                  warnings
_mlt                ftplib              py_compile          wave
_multibytecodec     funcsigs            pyclbr              weakref
_multiprocessing    functools           pydoc               webbrowser
_osx_support        future_builtins     pydoc_data          webencodings
_posixsubprocess    gc                  pyexiv2             wheel
_pyio               genericpath         pyexpat             whichdb
_pytest             getopt              pygtk               wsgiref
_random             getpass             pygtkcompat         xdg
_ruamel_yaml        gettext             pylab               xdrlib
_sha                gi                  pylocales           xlrd
_sha256             gio                 pyparsing           xlwt
_sha512             glib                pysam               xml
_socket             glob                pytest              xmllib
_sqlite3            gobject             pytz                xmlrpclib
_sre                grp                 pyximport           xmouse
_ssl                gtk                 quopri              xxsubtype
_strptime           gtkspellcheck       random              zim
_struct             gtkunixprint        rdflib              zipfile
_symtable           gzip                rdflib_jsonld       zipimport
_sysconfigdata      hashlib             re                  zlib
_sysconfigdata_nd   heapq               readline            
_testcapi           hmac                reportlab 

[miller34]

Hello Merle, Thank you for your interest in JustOrthologs. Which operating system are you using? If you are using Windows, then it is possible that the multiprocessing module isn't working correctly. We recently ran into that issue on a different software package that we have, and we developed a fix for that software package. We are working on implementing the fix for JustOrthologs as well, although we haven't pushed it yet.

If you aren't on Windows, try running the following command: python2 gff3_parser.py -g smallTest/wrapperTest/small_human.gff3 -f smallTest/wrapperTest/small_human.fasta.gz -o output.fasta

A similar command is called by both the wrapper and run_multiple_species.sh to create the extracted file that is later removed. My guess is that something is happening there (maybe permission denied for writing?). Please let me know what happens because I cannot replicate the error on my side.

Thanks.

[muecker]

Hi, thanks for the quick reply. I am using Ubuntu 18.04. I ran the command and it ran without any error message.

[ridgelab]

Ok. Thanks for the update. The wrapper simply takes the individual programs and runs them. So, each of the programs listed in README_OTHER_PROGRAMS and README are run. If those commands run without error on your system, then the wrapper should also run without error. Obviously that is not occurring, though. Would you mind checking those individual commands and seeing if any of those commands throw an error? If we're not able to find where the error is occurring, I would be happy to check the files that you want to run through JustOrthologs. You can email me at jmiller@byu.edu with the files, and I'll see if they're formatted correctly for JustOrthologs to run.

[lucasrocmoreira]

Hi,

I'm having the exact same error. Did you find a solution to the problem?