search

The-Academic-Observatory / observatory-platform

Observatory Platform Package
https://docs.observatory.academy
Apache License 2.0
18 stars 5 forks source link

Decide what data sources and telescopes to include in the Apache Airflow PoC #50

Closed jdddog closed 4 years ago

jdddog commented 4 years ago

Decide what data sources and telescopes to include in the Apache Airflow POC

aroelo commented 4 years ago

As discussed briefly over slack with @rhosking a suggestion for different telescopes: PoC 1

PoC 2

PoC 3

Would be good to further discuss suggestions here on github.

I will create an issue for each telescope so we can track progress.

rhosking commented 4 years ago

These make sense to me, @jdddog if you are happy with these i'll let you close the ticket as done.

@aroelo I was just playing, and it seems that issues can be placed in multiple projects. Just in regards to telescopes, and back to the discussion around giving visibility to the team, I wonder if we misuse the projects feature a little and create a 'Telescopes' Project, where others can easily see all the currently planned telescopes, and we can add notes and other things there to make that clear. Thoughts?

aroelo commented 4 years ago

Is it an idea to create a milestone per telescope in the 'Telescopes' project?

We can then create issues for smaller problems, for example the openssl error I have for crossref. These issues can be linked to the milestone. It is then possible to discuss solutions within the issue and close it with a corresponding commit that fixed the issue.

If you want to get an overview of what telescopes we're working on you can then just look at the milestones, so the issues don't clutter this overview.

rhosking commented 4 years ago

Minor update, can we move FundRef to PoC 1, and Crossref Events to PoC 2?

rhosking commented 4 years ago

So there are a couple of things here with regards to managing all the issues. I think yes, to begin with specific bugs or issues should be tracked separate from the main issue (build a telescope). Though, will appropriate cross-referencing.

@jdddog with regards to milestones, do you have a thoughts around the relationship between projects and milestones already? The projects feature is new to me, so still getting my head around the most appropriate use of it. On the surface, I think Anieks idea is good though.

The core point of a Telescopes project, is more around communication outside the development team. While it makes sense internally for telescopes to be linked with specific PoC's or later releases, having a clean place to track their progress (aka go to this link to see what's on track and scheduled to be developed) is key

jdddog commented 4 years ago

I think it is OK to make a telescopes project, as long as there isn't one project per telescope haha

For the second PoC it would be good to add support for the Unpaywall Data Feed to the Unpaywall telescope. Unpaywall snapshots are only updated a few times a year, wheras the Unpaywall Data Feed provides weekly diffs for the last Unpaywall snapshot.

rhosking commented 4 years ago

I think that makes sense. @aroelo let us know if you have any further questions on this ticket, else, I think we can make those changes and call this issues closed

aroelo commented 4 years ago

Sure, so we agree we create a separate telescopes project that is kind of kept 'clean' to keep an overview of the telescopes.

Just not sure what our conclusion is about the milestones, I think one milestone per telescope in this new telescope project would be nice to try out. I like that it makes it easier to reference smaller issues to a milestone.

rhosking commented 4 years ago

@aroelo let's try your idea of milestones per telescope. I think that is going to help us most quickly define how we will manage this. I suspect we will end up evolving this over time though.