search

intake / intake-xarray

Intake plugin for xarray
https://intake-xarray.readthedocs.io/
BSD 2-Clause "Simplified" License
76 stars 36 forks source link

Adding image driver and restructuring #25

Closed jsignell closed 5 years ago

jsignell commented 5 years ago

This PR adds an image reader for xarray and does a restructure to have a common structure for all plugins in intake-xarray. In particular this adds the ability to return a dataset rather than a dataarray using the merge_dim option.

I think this will help justify the existence of an intake-xarray plugin at all since it can now be used directly and the other plugins are just helper for using specific readers.

It is likely that this needs lots of work. I rewrote the example notebook to try to explain some of the behavior.

screen shot 2018-12-13 at 1 52 15 pm
martindurant commented 5 years ago

Are you around the coming week to take me through the proposal here?

philippjfr commented 5 years ago

Just tried playing with this, but I can't figure out why intake.open_image doesn't exist for me.

martindurant commented 5 years ago

@philippjfr , I believe @jsignell is not yet back from holidays. Maybe you get an explicit error when trying to import ImageSource?

philippjfr commented 5 years ago

Thanks for letting me know. No error when I try to import ImageSource directly. At some point I should clearly read some of the internals of the plugin system to understand how the function gets registered.

martindurant commented 5 years ago

That I can answer: Intake tries to import any package with the name intake_*, and looks for subclasses of DataSource in the top level. For each found, it registers the class by the name class attribute and generates the open function. They should appear in the top-level registry dict, and the functions that do the importing are in source.discovery.

jsignell commented 5 years ago

@martindurant I forgot to mention in our chat that this PR also adds the ability to return datasets using the kwarg merge_dim rather than only dataarrays. I am not sure if that is an overstep for intake or not, but it does seem like a bit of munging that it would be very handy to be able to encode in a catalog.

martindurant commented 5 years ago

I would say that adding extra capabilities that may be useful for some is totally in scope, so long as it doesn't ass complexity for those that don't need it.

jsignell commented 5 years ago

After conversation with Martin this chunk of work seems unreasonably big. So I am going to make a new PR that just adds an ImagePlugin.

martindurant commented 5 years ago

Closed in favour of #28 . The refactor may become necessary again at a later stage.