search

scverse / squidpy_notebooks

Tutorials for Squidpy
https://squidpy.readthedocs.io/en/stable/
MIT License
30 stars 17 forks source link

Refactor testing #40

Closed michalk8 closed 3 years ago

michalk8 commented 3 years ago

Tries to resolve the issue in #39

giovp commented 3 years ago

I agree it's a resource issue, and probably due to memory. But couldn't it be due to the image processing steps in the tutorial? Also, we could remove that tutorial and put it in external. All the functions are tested anyway in examples

michalk8 commented 3 years ago

I agree it's a resource issue, and probably due to memory. But couldn't it be due to the image processing steps in the tutorial? Also, we could remove that tutorial and put it in external. All the functions are tested anyway in examples

Problem is running all tests, just running the tutorial is fine. I am trying garbage collecting, maybe some process isolation would also be good.

giovp commented 3 years ago

what do you mean by process isolation? really happy to have a look at this and help out

michalk8 commented 3 years ago

By that I mean having 2 tox calls in 1 CI step (1 for tutorials, 1 for example), will try it now.

michalk8 commented 3 years ago

Other alternatives:

michalk8 commented 3 years ago

If you want, you can try the binary search (i.e. split the file in half and let the 1st half run on the CI with the current setup). If it runs, run 3/4 of the whole file, if not, run 1/4 of the file, etc.

michalk8 commented 3 years ago

@giovp I think the moran is the issue, look at the flame graph: profile.svg.txt

giovp commented 3 years ago

@giovp I think the moran is the issue, look at the flame graph: profile.svg.txt

oh yeah crazy, then for sure reduce perms and also n_genes if useful (like top 1k HVG, I can do that of you want)

giovp commented 3 years ago

it failed again. Give me an hour and can shortlist genes

michalk8 commented 3 years ago

it failed again. Give me an hour and can shortlist genes

It didn't matter (I've removed half bottom half of the file, still fails). I suspect leiden issue, let's see.

michalk8 commented 3 years ago

Wasn't that either, that leaves the segmentation (just noticed it's done of the full image, reduced the crop size to 1000).

michalk8 commented 3 years ago

@giovp seems to have worked, did the same changes in #39 and there it runs as well (touch wood).

michalk8 commented 3 years ago

I am closing this and will redo the test in #39 and the merge it.