Closed dputhier closed 3 years ago
dputhier
commented
3 years ago In fact, at least under OSX, when you run it several times you get the error rather frequently. Below is some example of what I got:
In [13]: set(interesting_combis)
Out[13]: {(0, 0, 0, 0, 1, 1), (1, 1, 0, 0, 0, 1), (1, 1, 1, 1, 0, 0)}
In [16]: set(interesting_combis)
Out[16]: {(0, 0, 0, 0, 1, 1), (1, 1, 0, 0, 1, 1), (1, 1, 1, 1, 0, 0)}
qferre
commented
3 years ago I tried running it too in Linux. I think I know what happened : it's the noise parameter. On rare occasions, it seems that 10% noise is too much, and this will drown the original combinations of (110000), (111100) and (000011) and MODL returns different combinations.
With 5% noise, it seems correct, so I'll reduce the noise for this unitary test. It happens rarely, hence why we did not see it earlier. I'll investigate more.
dputhier
commented
3 years ago To elaborate a little bit more it seems that, at least under OSX, the failure rate is nb_failure = 58%
from pygtftk.stats.intersect.modl.dict_learning import Modl, test_data_for_modl
import numpy as np
np.random.seed(42)
nb_successes = 0
nb_failure = 0
for i in range(100):
try:
flags_matrix = test_data_for_modl(nflags = 1000, number_of_sets = 6, noise = 0.1, cor_groups = [(0,1),(0,1,2,3),(4,5)])
combi_miner = Modl(flags_matrix, multiple_overlap_max_number_of_combinations = 3)
interesting_combis = combi_miner.find_interesting_combinations()
assert set(interesting_combis) == set([(1,1,0,0,0,0),(1,1,1,1,0,0),(0,0,0,0,1,1)])
nb_successes += 1
except:
nb_failure += 1I was lucky when I ran nose...
qferre
commented
3 years ago I just ran 100 tests with noise too. With 0.05 noise, all are correct, so I'll use that noise rate.
58% failure rate on 0.1 seems rather high though, it did not do that in my earlier tests. When you look at the matrix itself it's not really surprising : 0.1 noise means each element of the matrix has a 10% probability of being flipped, which is high. But still.
qferre
commented
3 years ago My own failure rate with 0.1 noise on Linux was 23 percent.
qferre
commented
3 years ago I just ran it again. 21 percent.
qferre
commented
3 years ago This might actually be due to the Convergence errors that keep plaguing me, and not to the algorithm itself. This might explain why it fails much more often on OSX. Investigating.
qferre
commented
3 years ago Okay, I found the problem. It is due to the matrix smothering that quadratically reduces the abundance of each unique row. While it is interesting to ensure the rarer combinations are still found, in this case it magnified the rate of noise by making the noisy lines much more frequent.
I changed the rate of noise in the unitary test to 5% in commit XXX based on that. 5% never failed in my tests. I have also added a discussion of this phenomenon in the paper and the documentation.
@dputhier: However, I did this in the develop branch and have merged back develop into release/1.2.0, before I realized that you hade made more commits in develop. I have not pushed the merged version yet. Should I ?
dputhier
commented
3 years ago In fact you should put this in master so that I will do a hotix. Develop is too advanced now because I have pushed modifications related to support for Python 3.8.
Le mar. 15 sept. 2020 à 15:11, Quentin Ferré notifications@github.com a écrit :
Okay, I found the problem. It is due to the matrix smothering that quadratically reduces the abundance of each unique row. While it is interesting to ensure the rarer combinations are still found, in this case it magnified the rate of noise by making the noisy lines much more frequent.
I changed the rate of noise in the unitary test to 5% in commit XXX based on that. 5% never failed in my tests. I have also added a discussion of this phenomenon in the paper and the documentation.
@dputhier https://github.com/dputhier: However, I did this in the develop branch and have merged back develop into release/1.2.0, before I realized that you hade made more commits. I have not pushed the merged version yet. Should I ?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dputhier/pygtftk/issues/139#issuecomment-692703957, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAN7CHVITAW56IGR5LTZ3H3SF5RXRANCNFSM4RMYKL6A .
Denis Puthier laboratoire INSERM TAGC/INSERM U 1090 Parc Scientifique de Luminy case 928 163, avenue de Luminy 13288 MARSEILLE cedex 09 FRANCE Mail: denis.puthier@univ-amu.fr Tel: (National) 04 91 82 87 31 / (International) 33 4 91 82 87 31 Fax: (National) 04 91 82 87 01 / (International) 33 4 91 82 87 01
Web:
http://tagc.univ-mrs.fr/tagc/index.php/research/network-bioinformatics/dputhier
====================================================================
qferre
commented
3 years ago Done in commit b21465b. Is everything okay for you ?
Just a question : I fixed another bug on master while I was doing it, so master is more advanced than develop on one point. When we will merge back develop into master, if I understand Git correctly, the more recent fix will be kept ?
dputhier
commented
3 years ago I am currently on hotfix/1.2.1 starting from master so integrating your modifications (which seem to fix #139).
$ git hf hotfix start 1.2.1Normally I should go for a release from this branch. When invoking the following line hubflow should normally merge hotfix/1.2.1 to develop (if I'm right). I will check it.
$ git hf hotfix finish 1.2.1Le mar. 15 sept. 2020 à 16:45, Quentin Ferré notifications@github.com a écrit :
Done in commit b21465b https://github.com/dputhier/pygtftk/commit/b21465b3170306659d655a4e703bf108c8c35cbc. Is everything okay for you ?
Just a question : I fixed another bug on master while I was doing it, so master is more advanced than develop on one point. When we will merge back develop into master, if I understand Git correctly, the more recent fix will be kept ?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dputhier/pygtftk/issues/139#issuecomment-692763230, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAN7CHQB64SLCZERP3KFY7LSF54W7ANCNFSM4RMYKL6A .
Denis Puthier laboratoire INSERM TAGC/INSERM U 1090 Parc Scientifique de Luminy case 928 163, avenue de Luminy 13288 MARSEILLE cedex 09 FRANCE Mail: denis.puthier@univ-amu.fr Tel: (National) 04 91 82 87 31 / (International) 33 4 91 82 87 31 Fax: (National) 04 91 82 87 01 / (International) 33 4 91 82 87 01
Web:
http://tagc.univ-mrs.fr/tagc/index.php/research/network-bioinformatics/dputhier
====================================================================
guillaumecharbonnier
commented
3 years ago It seems that the same error occurs with the hotfix 1.2.1
qferre
commented
3 years ago Okay, I am starting to panic a little bit. This is definitely not supposed to happen.
I cannot open the link you provided, could you copy the log in a text file ?
guillaumecharbonnier
commented
3 years ago No need to panic, Linux version looks good !
Sorry I did not realized circleci logs are not public. This one seems to be public: https://circleci.com/api/v1.1/project/github/bioconda/bioconda-recipes/123601/output/107/0?file=true&allocation-id=5f61f47994499101e4b93bdb-0-build%2F7A31176
04:32:03 BIOCONDA INFO (ERR) /Users/distiller/project/miniconda/conda-bld/pygtftk_1600255475165/test_tmp/run_test.sh: line 11: 1607 Segmentation fault: 11 nosetests --with-doctest $iThanks, my blood pressure is back down to normal levels ;)
But I don't see which command causes the error in the log ? What am I missing ?
guillaumecharbonnier
commented
3 years ago For a reason I ignore, I only get stderr at that point in the log so I can not precisely tell you which nosetest fails.
I am tempted to just edit https://github.com/bioconda/bioconda-recipes/blob/update_pygtftk_v1_2_0/recipes/pygtftk/run_test.sh to redirect the echo to stderr and build again to get a more insightful log.
dputhier
commented
3 years ago If I'm right, it seems that the error differs (it is not related this time to dict_learning.py but to ./fasta_interface.py. I think this may be related to BioPython version. I have pushed version 1.2.2 (that does not integrate the latest upcoming modifications of Quentin but should be functional). Best
Le mer. 16 sept. 2020 à 13:56, guillaumecharbonnier < notifications@github.com> a écrit :
It seems that the same error occurs with the hotfix 1.2.1
— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/dputhier/pygtftk/issues/139#issuecomment-693356793, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAN7CHWNH457JWEQGJVLQWLSGCRV7ANCNFSM4RMYKL6A .
Denis Puthier laboratoire INSERM TAGC/INSERM U 1090 Parc Scientifique de Luminy case 928 163, avenue de Luminy 13288 MARSEILLE cedex 09 FRANCE Mail: denis.puthier@univ-amu.fr Tel: (National) 04 91 82 87 31 / (International) 33 4 91 82 87 31 Fax: (National) 04 91 82 87 01 / (International) 33 4 91 82 87 01
Web:
http://tagc.univ-mrs.fr/tagc/index.php/research/network-bioinformatics/dputhier
====================================================================
guillaumecharbonnier
commented
3 years ago Yes you are right Denis, it is related to fasta_interface.py. https://circleci.com/api/v1.1/project/github/bioconda/bioconda-recipes/123628/output/107/0?file=true&allocation-id=5f62156a651d6e58c511eae9-0-build%2F3DDB1256
guillaumecharbonnier
commented
3 years ago However, even with biopython 1.78 I get the error : https://circleci.com/api/v1.1/project/github/bioconda/bioconda-recipes/123628/output/107/0?file=true&allocation-id=5f62156a651d6e58c511eae9-0-build%2F3DDB1256 Note in the log build in the message above I already had biopython 1.78 even without the explicit requirement in meta.yaml.
dputhier
commented
3 years ago I just confirmed that run_test.sh is working on my OSX machine (no issue with fasta_interface.py...). Could you try to modify run_test.sh in order to try to capture the error message:
for i in `find . -name "*.py" | perl -ne 'print unless(/(setup)|(plugin)|(bwig)|(libgtftk.py)|(__)/)'`; do echo "================="; echo $i; nosetests --with-doctest $i ; done &> /tmp/gtftk_built.logI don't know whether you have any access to /tmp/gtftk_built.log after error is encountered...
guillaumecharbonnier
commented
3 years ago I did the similar following modification yesterday to be able to see that the error was related to fasta_interface.py in the circleci log. https://github.com/bioconda/bioconda-recipes/commit/4a77eb40b6fa42e5f8ff298c6a1563251a976e6d
Could you share your successful conda-build log so I may track down discrepancies in package versions?
dputhier
commented
3 years ago But you did not return the nosetest error :
nosetest --with-doc-test $i
Should be
nosetest --with-doc-test $i 2>Or I missed something...
Best
Le jeu. 17 sept. 2020 à 16:02, guillaumecharbonnier < notifications@github.com> a écrit :
I did the similar following modification yesterday to be able to see that the error was related to fasta_interface.py in the circleci log. bioconda/bioconda-recipes@4a77eb4 https://github.com/bioconda/bioconda-recipes/commit/4a77eb40b6fa42e5f8ff298c6a1563251a976e6d
Could you share your successful conda-build log so I may track down discrepancies in package versions?
— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/dputhier/pygtftk/issues/139#issuecomment-694257351, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAN7CHSJKICDQCII6WCZM3TSGIJJBANCNFSM4RMYKL6A .
guillaumecharbonnier
commented
3 years ago I don't understand why your suggestion should matter. By default the stderr should be stored in the circleci log I have access to. If something insightful was provided it should already be here in the log I shared above. I now even redirect nosetest stdout to the circleci log but I still get the same segfault with no additional insights. https://circleci.com/api/v1.1/project/github/bioconda/bioconda-recipes/123928/output/107/0?file=true&allocation-id=5f6469862ca38311aa60a811-0-build%2F1613C10A
guillaumecharbonnier
commented
3 years ago Segfault at ./fasta_interface.py nose test is back on bioconda CI for v1.2.7.
Highlights from: https://circleci.com/api/v1.1/project/github/bioconda/bioconda-recipes/127761/output/107/0?file=true&allocation-id=5f8d613d10b48c53a9d40fe6-0-build%2F18634D68 below
03:02:56 BIOCONDA INFO (ERR) /Users/distiller/project/miniconda/conda-bld/pygtftk_1603101310369/test_tmp/run_test.sh: line 11: 1634 Segmentation fault: 11 nosetests --with-doctest $i03:02:57 BIOCONDA INFO (OUT) ./cmd_manager.py
03:02:57 BIOCONDA INFO (OUT) =================
03:02:57 BIOCONDA INFO (OUT) ./cmd_object.py
03:02:57 BIOCONDA INFO (OUT) =================
03:02:57 BIOCONDA INFO (OUT) ./fasta_interface.py
03:03:00 BIOCONDA ERROR COMMAND FAILED (exited with 1): /Users/distiller/project/miniconda/bin/conda build --override-channels --no-anaconda-upload -c conda-forge -c bioconda -c defaults -e /Users/distiller/project/miniconda/conda_build_config.yaml -e /Users/distiller/project/miniconda/lib/python3.7/site-packages/bioconda_utils/bioconda_utils-conda_build_config.yaml recipes/pygtftk/meta.yamlI have added a skip to ./fasta_interface.py nose test here
https://github.com/bioconda/bioconda-recipes/pull/24913/commits/2c382d871d3993468e134c8ae23ef7013636be8b
and waiting for CI.
dputhier
commented
3 years ago Thank you Guillaume. I had already skipped this test in a previous recipe but I don't understand why fasta_interface is tested again... Also I don't understand why osx tests appeared as successful in : https://github.com/bioconda/bioconda-recipes/runs/1260148623
Best
Le lun. 19 oct. 2020 à 12:24, guillaumecharbonnier notifications@github.com a écrit :
Reopened #139 https://github.com/dputhier/pygtftk/issues/139.
— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/dputhier/pygtftk/issues/139#event-3892619434, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAN7CHSA43DHV5OJBL5HW4TSLQHV7ANCNFSM4RMYKL6A .
Denis Puthier
Directeur Adjoint formation Institut MarMaRa https://www.univ-amu.fr/marmara Co-directeur du TGML https://tagc.univ-amu.fr/en/resources/transcriptomic-genomic-platform-marseille-luminy (Transcriptomic & Genomic Platform Marseille Luminy)
laboratoire INSERM TAGC/INSERM U 1090 Parc Scientifique de Luminy case 928 163, avenue de Luminy 13288 MARSEILLE cedex 09 FRANCE Mail: denis.puthier@inserm.fr Tel: (National) 04 91 82 87 11 / (International) 33 4 91 82 87 11 Fax: (National) 04 91 82 87 01 / (International) 33 4 91 82 87 01
Web: https://tagc.univ-amu.fr/en/user/645
====================================================================
guillaumecharbonnier
commented
3 years ago If you look at the commit used for the "successful" OSX build you are referring https://github.com/bioconda/bioconda-recipes/commit/30cc88c44d8932bf7aada835e4f92dc0304e7e75 you can see there is the "skip: osx". So the CI is telling you it has successfully skipped the build...
Now that I have skipped fasta_interface.py test by editing https://github.com/bioconda/bioconda-recipes/commits/guillaumecharbonnier-patch-2/recipes/pygtftk/run_test.sh with:
for i in `find . -name "*.py" | perl -ne 'print unless(/(fasta_interface.py)|(setup)|(plugin)|(bwig)|(libgtftk.py)|(__)/)'`I get the segfault for gtf_interface.
Do you remember if I can just remove the test for this one and we should be done, or if there are also other tests to skip because they segfault too ? I want to avoid wasting again CI resources with failed builds. I have looked for your edited version of run_test.sh in the history but I can't see it. Maybe you trigger the script elsewhere.
dputhier
commented
3 years ago Guillaume, Just delete also the tests for gtf_interface so that it should be OK. Just add to the list of files to be avoid:
... perl -ne 'print unless(/(gtf_interface)|(fasta_interface.py)|(setup)|(plugin)|(bwig)|(libgtftk.py)|(__)/)'`
There something really weird on the Circle CI side but we don't have enough information to understand. Tests are passing on my own mac and are regularly tested before any release.
Best
guillaumecharbonnier
commented
3 years ago OSX build done, waiting for review. It should be good this time.
@qferre . We have an issue in nose tests. The error is erratic but I think this is what impedes the proper building under Bioconda (OSX).
Make nose