Closed sclamons closed 3 weeks ago
I am having some trouble reproducing this. Basically, I am using your working example and then just running
df.with_columns(alignscore=pla.check_for_const_seq_match("Barcode", target_seq)) # <- Run this line multiple times, it eventually segfaults
in a loop to see if it fails. If you find any other information that might be useful just let me know. I see you are trying to do striped local alignment, so for now I am working out a potential alternative solution.
Ok, I am able to reproduce this finally. Working on a fix, but it might take some time.
Thank you so much for putting the time into tracking down - and hopefully fixing, fingers crossed - this issue! It's truly heroic of you.
I'm not exactly sure where how Polars' internals work, but the aligner should get built either once for the whole column of data, or one for each chunk that Polars' decides to split it into.
On Sat, Aug 10, 2024, 10:17 AM Nicolas Buitrago @.***> wrote:
Ok, I am able to reproduce this finally. Working on a fix, but it might take some time.
— Reply to this email directly, view it on GitHub https://github.com/nsbuitrago/parasail-rs/issues/7#issuecomment-2282214827, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAM7DU3S7DYOWQ7BVKRU3L3ZQZDMPAVCNFSM6AAAAABMBGEBTWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBSGIYTIOBSG4 . You are receiving this because you authored the thread.Message ID: @.***>
I've double checked for any memory issues with parasail-rs but haven't been able to find anything specific. This problem may be related to a stack overflow in polars. A couple things that could help are setting RUST_MIN_STACK
(as noted in https://github.com/pola-rs/polars/issues/12268). There may also be some additional optimization that can be done on the extension side. For example, with check_for_const_seq_match
as is I get a segfault at around 100000 iterations when adjusting RUST_MIN_STACK. This issue is solved if we cache the aligner to avoid rebuilding it every time the target sequence changes or for each iteration in my example.
use once_cell::sync::Lazy;
use parking_lot::Mutex;
//... rest of implementation
struct CachedAligner {
aligner: Aligner,
target: String,
}
static ALIGNER_CACHE: Lazy<Mutex<Option<CachedAligner>>> = Lazy::new(|| Mutex::new(None));
#[polars_expr(output_type=UInt8)]
pub fn check_for_const_seq_match(inputs: &[Series], kwargs: AlignKwargs) -> PolarsResult<Series> {
let aligner = {
let mut cache = ALIGNER_CACHE.lock();
match cache.as_ref() {
Some(cached) if cached.target == kwargs.target => cached.aligner.clone(),
_ => {
let new_aligner: Aligner = build_aligner(&kwargs.target);
*cache = Some(CachedAligner {
aligner: new_aligner.clone(),
target: kwargs.target.clone(),
});
new_aligner
}
}
};
let seqs = inputs[0].str()?;
let match_scores: ChunkedArray<UInt8Type> =
seqs.try_apply_generic(|data: Option<&str>| -> PolarsResult<Option<u8>> {
match data {
Some(s) => Ok(Some(check_alignment_score(s, &aligner))),
None => Ok(None),
}
})?;
println!("Finished check_for_const_seq_match: {:?}", match_scores);
Ok(match_scores.into_series())
}
This leads me to think this is just an issue with the underlying polars extension. Of course this is just a working example I found to avoid the issue but may not be the best. The above example does rely on being able to clone the aligner which is not available as of 0.7.3. I've included for the next update in 0.7.4.
This could be a bit annoying to reproduce.
I've been trying to build an extension for Polars that applies a parasail alignment to each element of a column. This consistently leads to a segfault, unless I limit Polars to one thread (using the
POLARS_MAX_THREADS
env variable). With unlimited threads, the extension segfaults very consistently; with two threads, it segfaults eventually but takes more tries.I'm not certain where the segfault is happening, but I think this is the relevant stack trace, as reported by pystack:
My extension code (apologies for the very-much-not-minimal example!):
expressions.rs:
lib.rs
init.py
Cargo.toml
pyproject.toml
And finally the script code to reproduce: