search

ioos / erddapy

Python interface for ERDDAP
https://ioos.github.io/erddapy/
BSD 3-Clause "New" or "Revised" License
77 stars 30 forks source link

GSoC ideas #165

Closed ocefpaf closed 3 years ago

ocefpaf commented 3 years ago

What is erddapy?

erddapy takes advantage of ERDDAP's RESTful web services to create the ERDDAP URLs. The users can create virtually any request like, searching for datasets, acquiring metadata, downloading data, etc.

What is ERDDAP?

ERDDAP is a data server that provides a consistent way to download subsets of scientific datasets.

There are many scientific data server available, like OPeNDAP, WCS, SOS, OBIS, etc. They all have their advantages and disadvantages, ERDDAP goal is to fill the gaps and unify most of the advantages in a single service. The main advantages of ERDDAP are:

See https://coastwatch.pfeg.noaa.gov/erddap/index.html for more information.

Ideas for GSoC

erddapy supports only tabledap protocol but ERDDAP also provides a griddap protocol that servers gridded data like models and satellite images. One could extend erddapy's API to support griddap and expand the amount of data Python users can obtain with this library.

Issue: https://github.com/ioos/erddapy/issues/32

The library has two types of constraints, "regular ones" that are parsed as expected by Python users, and the relative_constraints, which are passed directly to ERDDAP without any pre-processing. Ideally the API should be unified to avoid confusing the users.

Issue: https://github.com/ioos/erddapy/issues/164

The current API is a bit too verbose, for example: 


e.dataset_id = "whoi_406-20160902T1700"

e.variables = [
    "depth",
    "latitude",
    "longitude",
    "salinity",
    "temperature",
    "time",
]

e.constraints = {
    "time>=": "2016-09-03T00:00:00Z",
    "time<=": "2016-09-04T00:00:00Z",
    "latitude>=": 38.0,
    "latitude<=": 41.0,
    "longitude>=": -72.0,
    "longitude<=": -69.0,
}

while that makes a good base for other libraries to be built on top of erddapy, like argopy and gliderpy, it makes it hard to use high level objects as constraints. For example, a shapefile are any GIS WKT-like object.

The idea would be to create a higher level API that could consume these object and output a similar query as the one described above.

Issue: https://github.com/ioos/erddapy/issues/96

kstonekuan commented 3 years ago

Hi @ocefpaf, my name is Kingston and I am a Computer Engineering Undergraduate from the National University of Singapore. When I was exploring research using flood datasets previously, I learnt how difficult gathering data is and am excited by your mission to support ocean-related research. I was intrigued by the work done for erddapy and would love to explore some of the ideas here for GSoC 2021. In particular, "Idea 2: High level data queries" has caught my attention as I have experience with REST APIs as well as pandas. Would it be possible to discuss this further with you? Thank you for your time!

ocefpaf commented 3 years ago

Would it be possible to discuss this further with you?

Sure. At this point we are asking prospective students to get familiar with the project(s). Please read the documentation, check if there are any low hanging fruit in issues list that you could tackle. The GSoC application will start on March 29th. By then hopefully you will know if you want to work with this code base and have a draft proposal.

kstonekuan commented 3 years ago

Would it be possible to discuss this further with you?

Sure. At this point we are asking prospective students to get familiar with the project(s). Please read the documentation, check if there are any low hanging fruit in issues list that you could tackle.

Thank you for your guidance, I will take a look at the current issues and contribute wherever I can!

HenryKobin commented 3 years ago

@ocefpaf I've been browsing around a couple of the IOOS GSoC project ideas, and had two quick questions.

  1. Can we propose to work on more than one codebase (I.E. an "idea" on this project and an "idea" on Colocate)?
  2. If we consider one of the ideas on this issue thread to potentially be low hanging fruit, can we try forking this project and working on it? I think I might have a decent solution to Idea #1. I understand if we should hold off on any of those until after submitting our proposals, though.

Thanks for your time!

ocefpaf commented 3 years ago

@HenryKobin combining projects, as long as you have a feasible project, is fine and actually encouraged since we don't know how many slots Google will give us and some projects may not get funds.

The low hanging ideas are there just to help prospective students to write their proposals. A proposal that will be done in a few days probably won't get selected. TL;DR build up on those ideas and/or, to show engagement, even solve some of them before submitting your project.

avishmehta68710 commented 3 years ago

@ocefpaf Hi The chat channel mentioned in GSOC page of IOOS shows it has no rooms and no people. can you provide me a valid chat channel link for communication. image

ocefpaf commented 3 years ago

The chat channel mentioned in GSOC page of IOOS shows it has no rooms and no people. can you provide me a valid chat channel link for communication.

We'll use the issues here for now. The gitter, or some other chat technology, will be used when we start GSoC. The main reason is Gitter's move to matrix, which complicated our communications a little bit.

avishmehta68710 commented 3 years ago

Hi @ocefpaf in present situation we need to operate it from shell and IDE. i am thinking of integrating it with HTML,CSS,JS and Django for creating the webpages so that if a user does not know coding he can still get the knowledge. making a website so so that user can view it on web. and for the api of griddap protocols can i use python with django i wanted your advice on my idea. Correct me if i am wrong Thanks

ocefpaf commented 3 years ago

@avishmehta68710 it is not clear to me what you are planning to do. Do you mind creating a new issue with tour idea? Or better yet, a Google Doc with a draft for your project?