search

dib-lab / kProcessor

kProcessor: kmers processing framework.
https://kprocessor.readthedocs.io
BSD 3-Clause "New" or "Revised" License
11 stars 1 forks source link

update kmerDecoder #99

Closed mr-eyes closed 2 years ago

mr-eyes commented 2 years ago

Trying to Fix #98 but will need @shokrof approval.

shokrof commented 2 years ago

did you check the test cases after your edits?

mr-eyes commented 2 years ago

did you check the test cases after your edits?

There's a problem in building the test cases

lto1: internal compiler error: Segmentation fault
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://gcc.gnu.org/bugs/> for instructions.
lto-wrapper: fatal error: /home/mabuelanin/miniconda3/envs/kProcessor/bin/x86_64-conda-linux-gnu-c++ returned 1 exit status
compilation terminated.
/home/mabuelanin/miniconda3/envs/kProcessor/bin/../lib/gcc/x86_64-conda-linux-gnu/9.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
make[2]: *** [tests/kProcessorLibTests/CMakeFiles/testKprocessorLib.dir/build.make:121: tests/kProcessorLibTests/testKprocessorLib] Error 1
make[1]: *** [CMakeFiles/Makefile2:11524: tests/kProcessorLibTests/CMakeFiles/testKprocessorLib.dir/all] Error 2
make: *** [Makefile:166: all] Error 2

I tried with and without conda. Can you try building tests for this branch?

shokrof commented 2 years ago

is this error reproducible? I think the problem is in the compiler not the code

shokrof commented 2 years ago

I build it on my machine and test cases for this part didn't pass. my compiler version is "g++ (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0"

mr-eyes commented 2 years ago

Great it compiled. Would appreciate your help in resolving this issue with the latest parallel hashmap library so I can continue developing on v2.

On Mon, Apr 25, 2022 at 1:35 PM Mostafa Shokrof @.***> wrote:

I build it on my machine and test cases for this part didn't pass. my compiler version is "g++ (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0"

— Reply to this email directly, view it on GitHub https://github.com/dib-lab/kProcessor/pull/99#issuecomment-1109015168, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABWVPKCEEH5OCT7UZMNGZ5TVG36Y7ANCNFSM5UHSPXHA . You are receiving this because you authored the thread.Message ID: @.***>

-- Mohamed Abuelanin Ph.D. Student GRA at the DIB-LAB http://ivory.idyll.org/lab/ UC Davis, Computer Science Dept.

shokrof commented 2 years ago

I can fix it, but I suggest you try first. It is good to learn about how to use parallel hashmap in a parallel context. and this is an example. take a look at the code and make sure that you understand what Is the main objective of the function and how it is achieved. and also read the documentation for try_emplace function in parallel hashmap

mr-eyes commented 2 years ago

I was not planning to learn how to use it properly for now. Yesterday I tried to understand and went through the scattered docs I mentioned in the comments here https://github.com/dib-lab/kProcessor/blob/1da00371bb7c766ff11d346f5234aa87c03ac760/src/algorithms.cpp#L642-L643 but it's gonna take a while to figure out because I need to revise how multi processing/mutex/etc... is working.

mr-eyes commented 2 years ago

The emplace function here is to either insert a new kmer with a new default kmerOrder if the hash value didn't exist. Or update its kmerOrder if the hash val exist. try_emplace_l is used because you're expecting that you find the same kmer while parallel joining the kDataFrames.

shokrof commented 2 years ago

That's great! what try_emplace is doing is either put the kmer in the hashmap or call a function that is passed as a parameter. the lambda function being passed in the last commit is empty unlike the original code before the update. please fix it and make sure the test cases are running

mr-eyes commented 2 years ago

@shokrof fixed, tests are passing now.

./testKprocessorLib --gtest_filter="testSetFunctions/setFunctionsTest.parallelinnerJoinTest*"

shokrof commented 2 years ago

Good job ya Mohamed, please make sure to test the code next time before making the pull request

mr-eyes commented 2 years ago

Thanks! I will run the tests in github actions so we can develope while testing every commit.

On Wed, Apr 27, 2022 at 6:04 PM Mostafa Shokrof @.***> wrote:

Good job ya Mohamed, please make sure to test the code next time before making the pull request

— Reply to this email directly, view it on GitHub https://github.com/dib-lab/kProcessor/pull/99#issuecomment-1111626050, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABWVPKHJVRLGJTYF2BEGHPTVHHPX7ANCNFSM5UHSPXHA . You are receiving this because you authored the thread.Message ID: @.***>

-- Mohamed Abuelanin Ph.D. Student GRA at the DIB-LAB http://ivory.idyll.org/lab/ UC Davis, Computer Science Dept.