EcoJulia / RasterDataSources.jl

Easily download and use raster data sets in Julia
MIT License
21 stars 10 forks source link

Raster path #21

Closed tpoisot closed 3 years ago

tpoisot commented 3 years ago

I'm not a big fan of requiring a path - what about by default using somewhere in the julia repo (so that any project using the rasters draw from the same source), with the possibility of using a specific path if required?

rafaqz commented 3 years ago

The reason for forcing users to define a path is how large some of the weather datasets are - 100s of GB. Downloading even part of them to the repo probably means into a linux/macos home directory, which can break a lot of things when you run out of hard drive space. If you have a solution to that we can change things, I just don't want to break peoples systems. In future we can use the new package preferences instead of ENV?

tpoisot commented 3 years ago

What about some types have a flag for the specific paths mandatory, others don't? I don't feel like setting an environment variable just to get bioclim data, that's going to be a huge hassle for most users just to accomodate the specifics of a few?

My solution would be that everything < 100M goes into a default position.

tpoisot commented 3 years ago

In fact we can even make it into a type trait: _requires_raster_path returns nothing by default, nothing is the type needs a path and it is set, and an error message explaining how to set it if it isn't.

jamesmaino commented 3 years ago

I agree that the system PATH variable is a bit of a nuisance for first time users. I think @tpoisot solutions sounds sensible - having the type trait seems like a nice compromise for larger datasets.

rafaqz commented 3 years ago

I also agree a trait sounds nice, it's overkill for BioClim users to have to set a path. 100MB cutoff sounds reasonable too. PR?

rafaqz commented 3 years ago

@tpoisot thanks for deleting the old package, I've moved this to EcoJulia and put in a PR to the registry. Can you make me an EcoJulia owner so I can do admin things here?

tpoisot commented 3 years ago

Yes, will do!

tpoisot commented 3 years ago

@mkborregaard is there a way to give @rafaqz admin privileges without the owner role? I think we can have things like "groups" with elevated permissions for some repos, but can't find it. Or we can just make @rafaqz an owner.

mkborregaard commented 3 years ago

I think groups is a good idea. I did that for JuliaPlots once. @rafaqz what kind of admin things do you want to do? just for your packages or for the whole org?

rafaqz commented 3 years ago

I'm just an owner in 2 other orgs where I have or share a package, I'm not sure what the concern is honestly.

But primarily I need admin on RDS.

Edit: and I would of course be happy doing occasional admin/mandatory PR reviews, version bumps etc that need ownership. And I'll probably be in this space for a while ;)

mkborregaard commented 3 years ago

Hi @rafaqz right you're an owner now. I don't think any of those activities require ownership though :-) There's no issue at all - it's just a question of using these statuses to signal how people want to be using the org. Far too many open source orgs end up with bloated lists of non-active members, and it weakens the org team spirit imho :-)

rafaqz commented 3 years ago

Admin of Documenter/tagbot whatever apps that need secrets, and mandatory PR approval do require ownership of the package, I just experienced these and made someone an owner. Registrator youre right. But thanks anyway for the add.

mkborregaard commented 3 years ago

Great, welcome :-)

In my view a member is someone who engages actively with issues, takes new development initiatives, makes PRs and reviews others. An owner is someone who see themselves developing new packages under the org and initiates and engages in conversations about where to take the overall ecosystem in the future. Oh and single package admin and org ownership are distinct. Does that make sense to you guys?

rafaqz commented 3 years ago

That makes sense to me, I was definately intending on the second option :)

Also just to clarify my position in this a little, most of the code currently here now was written as part of my paid work at cesar, and it's something cesar increasingly depends on in modelling work and published papers.

In the long term this package should really live here and get broader contributions, which will be good for cesar as well, as are some of the other packages here like NeutralLandscapes.jl. But I feel like moving registration from an org where everyone in cesar has ownership permissions to one where noone does would be a questionable decision for me to make.

Anyway, I reopened the registration PR ;)

And I'm glad to have this here, and look forward to more collaboration.

mkborregaard commented 3 years ago

That makes sense for sure!

tpoisot commented 3 years ago

@rafaqz is the admin stuff sorted out? I'll try adding the trait function this evening, as I'd like to have the next big release of SimpleSDMLayers depend on this package.

rafaqz commented 3 years ago

Yep, sounds good. I also have a few more PRs I'll push through that I didn't realise I hadn't pushed earlier, just minor refactors and adding docstrings basically.

rafaqz commented 3 years ago

You should also have a look at the GeoData.jl integration. Pretty easy way to download and use one or hundreds of rasters with a one liner, and |> plot to plot.