search

NSF-Polar-Cyberinfrastructure / datavis-hackathon

http://nsf-polar-cyberinfrastructure.github.io/datavis-hackathon
42 stars 11 forks source link

Data Science Publication for NSF Polar Cyberinfrastructure #74

Open brandniemann opened 9 years ago

brandniemann commented 9 years ago

"data science" and "proposed session" http://semanticommunity.info/Data_Science/Data_Science_for_NSF_Polar_Cyberinfrastructure

brandniemann commented 9 years ago

and "demo"

chrismattmann commented 9 years ago

Thanks @brandniemann

brandniemann commented 9 years ago

Explanation of Data Science Publication and Spotfire Visualizations

I think that NIH’s Associate Director for Data Science, Dr. Phil Bourne, really invented the data science publication with his journal (PLOS Computational Biology) and was hired to change the data culture to that at NIH: http://semanticommunity.info/Data_Science/Data_Culture_at_the_NIH#Story

He views data science publications as the building blocks for a “data commons” which is like a sandbox where researchers can come and play with others scientific data with their own tools or tools that are part of the “data commons”

His best explanation of a data science publication I have found is in his slides at: http://semanticommunity.info/Data_Science/Data_Science_for_RDA#Best_Practices_for_Data:_A_Biologists_View

Especially the one called: The Knowledge and Data Cycle

I am doing essentially the same thing by putting the content in MindTouch (a state-of-the-art Wiki) with structure and embedding the Spotfire file in MindTouch with a link to the Web Player version

For example, the NSF/NSB Indicators Digest 2014 text and graphics are in MindTouch with well-defined URLs in a searchable index which is integrated with the data tables for the graphics which are integrated by topic in multiple adjacent visualizations as Tufte suggests so the user can more easily compare trends, etc.

See: http://semanticommunity.info/Data_Science/Data_Science_for_Big_Data_Analytics#Data_Science_Data_Publication_for_National_Science_Board

More importantly, the Spotfire visualizations are more than static graphs, they are dynamically linked to one another and the underlying data, and also to the metadata and story in MindTouch.

We have done Data Science Publications for many senior government leaders: http://semanticommunity.info/Data_Science/NSF_Funding_for_BIG_DATA_and_Data_Science/NSF_Grant_Proposal_Guide#Conclusion

Another example: Data Science (publication) for the NOAA Chief Data Officer

http://semanticommunity.info/Data_Science/Data_Science_for_the_NOAA_Chief_Data_Officer

for our November 3rd Meetup: http://www.meetup.com/Federal-Big-Data-Working-Group/events/213175262/

Spotfire TIBCO Spotfire designs, develops and distributes in-memory analytics software for next generation business intelligence.

TIBCO Spotfire® Ranked Highest “Current Offering” in Forrester Wave for Agile BI 2014 Source: http://spotfire.tibco.com/

Complimentary Subscription: http://spotfire.tibco.com/tsc/donate What I have as a Journalist, Professor, and Non-Profit Organization

Free One Month Trial: https://spotfire.cloud.tibco.com/tsc/#!/tryspotfire Cloud Personal, Cloud Work Group, and Cloud Enterprise

See Slides 5 and 6. Is there a live link for the Spotfire to do analysis? https://spotfire.cloud.tibco.com/public/ViewAnalysis.aspx?file=/users/bniemann/Public/NSBIndicators2014-Spotfire&waid=eaccd3aaab73f89cda578-26211723b2ccba

You can download my Spotfire File (and/or my Excel Spreadsheets) and use it in your own Spotfire Client or another tool of your choosing so it is open in that sense.

chrismattmann commented 9 years ago

Thanks @brandniemann I checked out the Spotfire description for the NOAA meetup. Looks really neat! Did it build the wiki/website for that automatically?

brandniemann commented 9 years ago

No, there is considerable science and art involved in building the knowledge base (in MindTouch) and spreadsheet (in Excel) first, which then makes the Spotfire (data browser) application easier to “storify” the results.

These tools make it easier, but still require data science, statistics, visualization, data journalism talents and experience.

This is what I did in two phases for the NOAA work over a period of about a month and will do for the Polar Data in the next week for a demo and more after that to prepare for another EarthCube Meetup we have been planning for some time now:

http://semanticommunity.info/Data_Science/EarthCube_Data_Science_Publications

From: Chris Mattmann [mailto:notifications@github.com] Sent: Tuesday, October 28, 2014 2:47 PM To: NSF-Polar-Cyberinfrastructure/datavis-hackathon Cc: brandniemann Subject: Re: [datavis-hackathon] Data Science Publication for NSF Polar Cyberinfrastructure (#74)

Thanks @brandniemann https://github.com/brandniemann I checked out the Spotfire description for the NOAA meetup. Looks really neat! Did it build the wiki/website for that automatically?

— Reply to this email directly or view it on GitHub https://github.com/NSF-Polar-Cyberinfrastructure/datavis-hackathon/issues/74#issuecomment-60809801 . https://github.com/notifications/beacon/AA-W4NFJrn8UuOvngrlwlchnYdGS0BrGks5nH9wbgaJpZM4CzaJ6.gif

brandniemann commented 9 years ago

Starting to data mine for data sets at: http://semanticommunity.info/Data_Science/Data_Science_for_NSF_Polar_Cyberinfrastructure#Data_Sets

And document what I find for demo: http://semanticommunity.info/@api/deki/files/31200/BrandNiemann11042014.pptx

Looking for: “We will provide some of this prepared data to interested parties ahead of the workshop in the next few weeks in case folks want to start hacking early.”

Since I am new to this data domain.

From: Chris Mattmann [mailto:notifications@github.com] Sent: Tuesday, October 28, 2014 2:47 PM To: NSF-Polar-Cyberinfrastructure/datavis-hackathon Cc: brandniemann Subject: Re: [datavis-hackathon] Data Science Publication for NSF Polar Cyberinfrastructure (#74)

Thanks @brandniemann https://github.com/brandniemann I checked out the Spotfire description for the NOAA meetup. Looks really neat! Did it build the wiki/website for that automatically?

— Reply to this email directly or view it on GitHub https://github.com/NSF-Polar-Cyberinfrastructure/datavis-hackathon/issues/74#issuecomment-60809801 . https://github.com/notifications/beacon/AA-W4NFJrn8UuOvngrlwlchnYdGS0BrGks5nH9wbgaJpZM4CzaJ6.gif

brandniemann commented 9 years ago

More progress in finding, documenting, and preparing data sets (Text to Excel, Access, and Shape) in a Knowledge Base: http://semanticommunity.info/Data_Science/Data_Science_for_NSF_Polar_Cyberinfrastructure

Spreadsheet: http://semanticommunity.info/@api/deki/files/31201/NSFPolarCI.xlsx?origin=mt-web

Slides: http://semanticommunity.info/@api/deki/files/31200/BrandNiemann11042014.pptx?origin=mt-web

In preparation for visualizations in Spotfire.

From: Chris Mattmann [mailto:notifications@github.com] Sent: Tuesday, October 28, 2014 2:47 PM To: NSF-Polar-Cyberinfrastructure/datavis-hackathon Cc: brandniemann Subject: Re: [datavis-hackathon] Data Science Publication for NSF Polar Cyberinfrastructure (#74)

Thanks @brandniemann https://github.com/brandniemann I checked out the Spotfire description for the NOAA meetup. Looks really neat! Did it build the wiki/website for that automatically?

— Reply to this email directly or view it on GitHub https://github.com/NSF-Polar-Cyberinfrastructure/datavis-hackathon/issues/74#issuecomment-60809801 . https://github.com/notifications/beacon/AA-W4NFJrn8UuOvngrlwlchnYdGS0BrGks5nH9wbgaJpZM4CzaJ6.gif