hadekale
commented
3 months ago Hi! My team and I are very interested in making a proposal for challenge 10.
We are a team consisting of high school students from Turkey currently in the process of graduating, however we have plenty of free time thanks to the way our school term is structured. What we don't have is plenty of experience, though we aim to make up for it with our dedication and passion. During our discussion on how to make a good proposal, a few points of uncertainty arose.
Firstly, we are aware of the need to come up with a timeline and plan in general. Yet it is not clear how to make a plan without having a clear vision, which will apparently be discussed after the application as "(you will) work with (us) to define a "minimum viable product" (MVP) and a list of "nice-to-haves" that could be completed if time allows." and as "the project would start with some initial intensive discussions to communicate the vision and answer questions.". From this, we understand that making a proposal for the structure of such a web page is practically pointless, as it will be discussed later with the mentors.
The lack of information on proposal details (as well as the desire to prove our abilities) pushed us in the direction of making a demo. The idea is to make a basic demo with exemplary data showing the plot types given in the github page of the challenge. Would this indeed be helpful or would you suggest us to rather focus on another aspect? Should we still come up with some suggestions for the web page structure or what would you recommend us to mainly include in our proposal? We would appreciate some guidance on this issue.
Furthermore, you have stated that the verification software generates unnecessary plots and you want to rather supply the user with the plot on demand. If the plots generated by your verification software are publicly accessible we would like to have a look at them.
As an explanation, we searched the data page of the CAMS website however couldn't find any pre-generated verification plots. The plots in here, which are indeed verification plots, seem to be generated with server-side data and client-side visualisation via highcharts. If you mean there is pre generated data for cities, this normally shouldn't be a problem as the data must be a few megabytes per date (3 [models] 4 [pollutants] ~50 [cities] 151002 [data points per city per 4 day period] 4 [bytes per float] = 7.200.000 bytes ~7 megabytes and if done dynamically, 1/6 as much).
Of course, the application could be made more versatile, nonetheless, we fail to see why it would take your software a long time to generate the data for the plots on this website. As such we ask for a clarification of your plot generation problem, if this is indeed important.
Thanks for your answer in advance, Hakan Deniz Kale, on behalf of my team
LukeJones123
PostholerCom
Challenge 10 - A Web Application for Interactive Visualisation of Verification Results
Goal
Develop a single-page web application and associated backend to allow the flexible exploration and plotting of scientific data. The primary use case will CAMS forecast verification data, i.e. making plots that compare forecasts with observations, but ideally the system will be general enough to have wide applicability.
Mentors and skills
Skills required:
Challenge description
The Copernicus Atmospheric Monitoring Service (CAMS) makes daily forecasts and historic reanalyses of the planet's atmospheric composition, e.g. pollution, dust, greenhouse gas concentrations, etc. Verification is the process of comparing computer model forecasts with real-world observations so we can see how well the model is performing.
Our verification software currently spends a long time generating a huge number of plots, most of which don't get looked at. A solution to this problem would be a web application which would display a series of menus allowing a user to select precisely what they are interested in, and then allows them plot the associated data on demand.
This would necessitate a single-page frontend application with a Python-based backend. In order to be consistent with our other developments we envisage the frontend written in Javascript, preferably using Material UI on top of React. The backend could use Django, Flask, or anything similar. The job of producing the actual verification data would be ours - your backend would just have to call our Python to obtain the data, before preparing it for transfer to the frontend and rendering it as a plot.
Ideally, the code will be as general as possible. That is to say, the names and contents of the navigation menus will not be hard-coded anywhere in your app but will be provided by our backend code. If done this way this app would be useful beyond the immediate use case and could find widespread use across ECMWF.
We anticipate you would probably use Plotly to make the plots. The types we need include:
These plots should be as interactive as possible, allowing a user to change scale, zoom in, etc. An important feature will be that a click on a point in a map plot ( :earth_americas: ) would generate a plot showing the time-series at that site ( :round_pushpin: ).
The number of features we'd like to have is extensive and accomplishing them all would probably be unfeasible in the time available, so we'll work with you to define a "minimum viable product" (MVP) and a list of "nice-to-haves" that could be completed if time allows. It may be important to consider the nice-to-haves even if they can't be achieved as they may influence the design of the MVP. For example, we would very much like the option to make a plot non-interactively (i.e. with a batch script that saves the plot to file without ever using a browser) which may have important implications for the system design.
We envisage that the project would start with some initial intensive discussions to communicate the vision and answer questions, with bi-weekly meetings thereafter.
This project, if successful, would be invaluable to CAMS scientists and possibly to many others in ECMWF. We really hope you can help us with our goals and that you get some rewarding experience in return!