search

nasa / Transform-to-Open-Science

Transformation to Open Science
Other
699 stars 143 forks source link

Module 1, Lesson 1 --> Presents an oversimplification of Stargazing Live / Exoplanet Explorers #703

Open jessie-dotson opened 4 months ago

jessie-dotson commented 4 months ago

I love that we're using the Stargazing Live / Exoplanet Explorers example as a way to demonstrate the power of open science. It's a fantastic example about the power of open science / citizen science! But the way it is currently described in the training material isn't technically accurate...

This experiment was a grand success that failed to answer it's key questions... The questions the team were trying to address were statistical demographics questions (as are listed in the current online training material). The team worked hard to design an experiment that was appropriate to address the demographics questions -- including injecting known signals into the dataset so they'd have a control sample that could be used to calibrate the detections from the citizen scientists. The citizen scientists were fantastic at identifying planets in the data -- and indeed identified a lot of planets (including a really awesome multi-planet system). Unfortunately the results were not appropriate for statistical demographics studies. In order to do demographics studies, you need to measure and balance completeness and reliability. If your sample is complete, but unreliable or if it's incomplete but reliable -- you can't get solid statistics out. As it turns out, the citizen scientists were very complete (found everything) and very unreliable (including things that weren't real) -- so the team wasn't able to answer the demographics questions they started with.

So -- how to make the training material more accurate? On one hand it'd be cool to have the time to get into this level of detail about designing and executing a citizen science project -- but that's out of scope for this training material. Here's some suggested changes to keep the example, make it technically correct, and not dig deep into the details...

bressler95tops commented 3 months ago

Thank you for submitting this suggestion @jessie-dotson. As we mentioned in our talk, we are actively working on a review process for OS101 and we should be reviewing these in the coming weeks. Stay tuned and feel free to keep submitting more suggestions in the meantime.

bressler95tops commented 2 months ago

Noting that we are tabling this for the next disposition.

katblanchette commented 1 month ago

@bressler95tops - Please make the updates per Jesse's recommendations. Do you need more detail from @jessie-dotson ?

jessie-dotson commented 1 month ago

Received request for additional suggestions via email. answered there, but putting them here also for tracking...

Suggestion for second sentence after the graphic... Instead of "The hosts invited viewers to contribute to their research question by classifying solar systems from an open access data set." How about: "The hosts invited viewers to identify exoplanets in an open access dataset."

Second sentence in the following paragraph... Instead of "On the second night, enough people had participated that the researchers were able to share the demographics of the planet candidates that had already been flagged and were undergoing additional analysis:..." How about "On the second night, enough people had participated that the researchers were able to share that 44 Jupiter-size candidate planets, 72 Neptune-size candidate planets, 53 sub-Neptune size candidate planets (larger than Earth but smaller than Neptune), and 44 Earth-size candidate planets had already been found and were undergoing additional analysis."

katblanchette commented 1 month ago

@jessie-dotson - Thank you for the updated sentence. @bressler95tops - Will you update the content with @jessie-dotson new sentences? Thank you!

bressler95tops commented 1 month ago

Thanks @katblanchette for jumping on this one while I make my way through everyone's suggestions and thank you @jessie-dotson for carrying over those suggested content snippets over to GitHub, they are very thorough. I have addressed those sentences and the original issue that suggests removing the three questions above. You can view the changes in PR #757, specifically under the files changed tab. Let me know if anything needs further tweaking. Otherwise, I am happy to check this one off once the MOOC developers are able.

katblanchette commented 1 month ago

@bressler95tops - Thank you for updating the content. I have shared this information with the MOOC developers and will notify you when the change is made.