Summarizer fails without filelock package

[aggreen]

I recently created a new conda environment and installed evcouplings using pip install -e path/to/development/repo.

Using this environment, the summarizer functionality of the pipeline was not outputting any files because the filelock package was not installed. No error was thrown, and I confirmed that the function was being called and doing computation, but not writing to file. Neither the plots nor the csv file were created. This surprises me, it seems like it should have failed on import or on line 114 with the call to filelock.

After running conda install filelock the function works perfectly. Conda says this was a new package for my environment so I don't think it's a version issue.

we should definitely add filelock to the setup script, and potentially add some sort of error to the summarizer so that new users are prompted to install the software.

[thomashopf]

filelock is already in the dependencies and should be installed during a regular pip install evcouplings, so it very much looks like the -e option is the reason: https://github.com/pypa/pip/issues/4780

So I suppose nothing we can or need to do here, that is just setuptools behaviour that fortunately does not affect regular users.

[thomashopf]

I'll close for now since only relevant to developers, but not a problem of evcouplings -if you should ever see this happening with a regular pip install (without -e) please reopen.

[thomashopf]

Actually - the part that puzzles me though is that there you cannot see an ImportError? If the summarizer fails, that output should just be in the stderr dump of the pipeline...

[aggreen]

Okay good to know, we can close this then.

On Jan 26, 2018 9:34 AM, "thomashopf" notifications@github.com wrote:

filelock is already in the dependencies and should be installed during a regular pip install evcouplings, so it very much looks like the -e option is the reason: pypa/pip#4780 https://github.com/pypa/pip/issues/4780

So I suppose nothing we can or need to do here, that is just setuptools behaviour that fortunately does not affect regular users.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/debbiemarkslab/EVcouplings/issues/114#issuecomment-360800440, or mute the thread https://github.com/notifications/unsubscribe-auth/AHimXnTLEnxgURYRTnszLsg_HVBdwpyKks5tOeKEgaJpZM4RuXt8 .

[aggreen]

Apologies for that earlier reply - I was on mobile and didn't see that you had reopened the request. I'm content to just leave this as a mystery since it's now working on my machine and no one else has had this problem. Here's a bit more of a summary in case this ever comes up again.

In my orchestra anaconda environment from /groups/marks.users.agreen.evcoupligns_data/elongasome_monomers, I executed the summarizer command:

/groups/marks/software/anaconda/envs/agg_evcomplex/bin/python /groups/marks/users/agreen/dev/EVcouplings/evcouplings/utils/summarize.py protein_monomer /groups/marks/users/agreen/evcouplings_data/elongasome_monomers/output/FTSW_BACSU_1-403 /groups/marks/users/agreen/evcouplings_data/elongasome_monomers/output/FTSW_BACSU_1-403_b0.1_config.txt /groups/marks/users/agreen/evcouplings_data/elongasome_monomers/output/FTSW_BACSU_1-403_b0.2_config.txt /groups/marks/users/agreen/evcouplings_data/elongasome_monomers/output/FTSW_BACSU_1-403_b0.3_config.txt /groups/marks/users/agreen/evcouplings_data/elongasome_monomers/output/FTSW_BACSU_1-403_b0.5_config.txt /groups/marks/users/agreen/evcouplings_data/elongasome_monomers/output/FTSW_BACSU_1-403_b0.8_config.txt

code ran successfully with no errors, but produced no summary stats file or graph. I even added some temporary print statements to verify that the code is in fact finding the correct files and calculating the summary table. I grabbed this command directly from the job_database.txt file.

then I ran

(agg_evcomplex) ag300@loge:/groups/marks/users/agreen/evcouplings_data/elongasome_monomers$ conda install filelock
Fetching package metadata .........
Solving package specifications: .

Package plan for installation in environment /groups/marks/software/anaconda/envs/agg_evcomplex:

The following NEW packages will be INSTALLED:

    filelock: 2.0.7-py36_0

Proceed ([y]/n)? y

filelock-2.0.7 100% |#######################################################################################################################################| Time: 0:00:00 622.13 kB/s
(agg_evcomplex) ag300@loge:/groups/marks/users/agreen/evcouplings_data/elongasome_monomers$ /groups/marks/software/anaconda/envs/agg_evcomplex/bin/python /groups/marks/users/agreen/dev/EVcouplings/evcouplings/utils/summarize.py protein_monomer /groups/marks/users/agreen/evcouplings_data/elongasome_monomers/output/FTSW_BACSU_1-403 /groups/marks/users/agreen/evcouplings_data/elongasome_monomers/output/FTSW_BACSU_1-403_b0.1_config.txt /groups/marks/users/agreen/evcouplings_data/elongasome_monomers/output/FTSW_BACSU_1-403_b0.2_config.txt /groups/marks/users/agreen/evcouplings_data/elongasome_monomers/output/FTSW_BACSU_1-403_b0.3_config.txt /groups/marks/users/agreen/evcouplings_data/elongasome_monomers/output/FTSW_BACSU_1-403_b0.5_config.txt /groups/marks/users/agreen/evcouplings_data/elongasome_monomers/output/FTSW_BACSU_1-403_b0.8_config.txt

And the summary statistics file and pdf are produced.

I agree with you that it is puzzling.