Closed sergpolly closed 1 year ago
anecdotally - simply removing max_chunk=np.inf
argument from clr.matrix()
call: https://github.com/higlass/clodius/blob/dceb4bcb225e31f6669cc8be22485b8e56e2c082/clodius/tiles/cooler.py#L98 fixes tests in tests/tiles/cooler.py
todo - test higlass-python with such modified clodius for serving local coolers
I know @nvictus was prototyping a new tile fetcher for clodius > 0.9. Maybe that’s in a state he can share?
hm - so far I can confirm that, the simple fix was enough ... and higlass-python is happily serving me a local cooler from within open2c examples :
It's not clear to me what this used to do because the max_chunk
parameter isn't included in the old docstring.
I guess we could either choose to remove the parameter or have the newer cooler API accept **kwargs
as a final catch all for kwargs. My feeling is to just remove max_chunk
as you have, and pin cooler>=0.9.0
in the pyproject.toml for clodius.
here is what I found in the old version:
CSRReader
is where matrix
method uses max_chunk
parameter - so I think it would be fair to say that it used to control performance/memory-footprint tradeoff ...
It is unclear however what happened to this "contol knob" in the 0.9.0...
I know that 0.9.0 was a major refactoring of the query engine...
Wait... - the new API (e.g. matrix
method) has chunksize
(!) which remains undocumented, - perhaps it does the similar thing as max_chunk
before
Yes, removing max_chunk
should do the trick until the new tile fetcher is merged.
Just wanted to upvote this issue, especially since the fix appears to be simple.
there is a little conundrum in the higlass-python/clodius/cooler/cooltoos interoperability now:
what is incompatible between cooler 0.9.1 and clodius at the moment ? is it just tiles/cooler.py ? I'm willing to look at it - just trying to narrow down the issue ...