license spreadsheet and sherpa/info.json are out of sync

[pixelzoom]

The license spreadsheet (on Google Drive) and sherpa/info.json are out of sync.

These are missing from info.json:

grunt html5-shiv modernizr node.js PxLoader reveal.js stats.min.js uglify2

Also, stats.min.js has an entry in the spreadsheet, but no info associated with it.

Highly recommended to consolidate this info into info.json and delete the Google spreadsheet. Until that's done, these are more likely to be out-of-sync than in-sync.

[pixelzoom]

We should also answer the question of what licenses need to be bundled with the HTML file, since those must be in info.js: https://github.com/phetsims/chipper/issues/32.

[samreid]

@kathy-phet has previously requested the google doc as a convenient way to share licensing information with other parties. @pixelzoom, what do you think about keeping things synchronized by generating the Google Spreadsheet from the info.js? We would still have to remember to re-sync when (a) info.js changes or (b) when @kathy-phet is about to share the google doc to another party.

Another possibility would be for @kathy-phet to share https://github.com/phetsims/sherpa/blob/master/info.json with 3rd parties directly--I wonder if this would be possible or if we would have to make it even more human-readable-friendly.

[pixelzoom]

5/28/15 dev meeting:

@pixelzoom: Move all info from spreadsheet into sherpa/info.json. Delete libs that do not include code in the html file (i.e., no licenses for stuff that is exclusively part of toolchain). Update chipper/build.json to include the proper keys in licenseText that are required by all sims.

@aaronsamuel137: Write a script to generate something that is human-readable from info.json (GitHub markdown?). Make the output accessible at the URL that has been published for the spreadsheet, replace contents of the spreadsheet with URL (do not delete spreadsheet). Consider running script automatically when a build is done, so that human-readable form stays up-to-date.

Google license doc: https://docs.google.com/spreadsheets/d/1XWlHFidztUT6ARb7yHQ1Naepd6L0x9LF94AAoKHGaWg/edit#gid=0

[pixelzoom]

I'll assign this issue to @aaronsamuel137 when I'm done with my piece.

[pixelzoom]

@kathy-phet wrote in email:

Hi Aaron,

This is the link I will use.

http://phet.colorado.edu/third-party-libraries

Look OK? Kathy

[pixelzoom]

Libraries in spreadsheet that are not in info.json:

grunt html5-shiv modernizr node PxLoader reveal uglify2

I added reveal-3.1.0 to info.json. I deleted the others from the spreadsheet, since they are either no longer used or are not included in the html deliverable.

[pixelzoom]

Moved all license URLs from spreadsheet to info.json, optional field for when license has a URL that is different from project URL.

[pixelzoom]

I am not going to move the "In which sims is it used?" column from the spreadsheet to info.js. This is unlikely to be kept up to data. And it can be easily determined by searching in build.json and package.json. If it's in build.json, then it's used by all sims. If it's in package.json, then it's used by those specific sims.

[pixelzoom]

@samreid I need to discuss the Google license spreadsheet with you. The semantics of a few columns are unclear: "Compliance in Beer's Law Lab", "Compliance on GitHub", "Added 7/21/2014". And it's unclear whether those columns (and a couple of others) are needed.

[pixelzoom]

Remaining questions about the spreadsheet:

• What is the "Compliance in Beer's Law Lab" column? • What is the "Compliance on GitHub" column? • What is the "Added 7/21/2014" column? • Does info in any of the above columns need to be migrated to info.json? • Since only libs whose code appears in deliverables need to be included, I'm assuming that the "Does it appear in run-time HTML5 code?" column does not need to be migrated to info.json. Is that correct?

Remaining questions about info.json:

• I've added a "deprecated" field for entries that may be in published sims but are not longer used. It's only present for deprecated libs, and its value is true. This feels a little weird. Is there a better solution? • I moved the "how is it used?" info to "notes". This info is pretty sketchy, sometimes missing, and in some cases it's sim-specific. If we're showing this to the general public, should we clean this up? • Is requirejs-text still used? Was it ever used? • This file only needs to include libs whose code appears in deliverables. Which entries can be deleted?

[pixelzoom]

I've been asked to complete this sooner rather later (high priority). @samreid when is a good time to discuss the questions above?

[pixelzoom]

What is the "Compliance in Beer's Law Lab" column? • What is the "Compliance on GitHub" column? • What is the "Added 7/21/2014" column?

These columns are related to contracts that @kathy-phet was working on, check with her.

• Does info in any of the above columns need to be migrated to info.json?

Possibly need to archive the spreadsheet. Unlikely to propagate to info.json.

[pixelzoom]

• Since only libs whose code appears in deliverables need to be included, I'm assuming that the "Does it appear in run-time HTML5 code?" column does not need to be migrated to info.json. Is that correct?

We'll keep the things that are distributed via sherpa, but we'll ditch the others in info.json.

[pixelzoom]

• I've added a "deprecated" field for entries that may be in published sims but are not longer used. It's only present for deprecated libs, and its value is true. This feels a little weird. Is there a better solution?

Leave as is for now.

• Is requirejs-text still used? Was it ever used?

Replaced by PhET's own text.js plugin (see chipper).

• This file only needs to include libs whose code appears in deliverables. Which entries can be deleted?

Also keep things like documentation tools.

[pixelzoom]

• I moved the "how is it used?" info to "notes". This info is pretty sketchy, sometimes missing, and in some cases it's sim-specific. If we're showing this to the general public, should we clean this up?

Leave as is for now. Revisit when it's put in human-readable format.

[pixelzoom]

Commits not attached to this issue:

Removed "usage" field: https://github.com/phetsims/sherpa/commit/4d7db8b25b411085fc8e90999293df5eece70cfb Fixed lodash entry: https://github.com/phetsims/sherpa/commit/202fcf39f50ea4b1c740771044a92672fdd889a0 delete requirejs-text, delete selectedLicense field, touch up license fields, pull licenseURL out of license field: https://github.com/phetsims/sherpa/commit/333b91755ad452c3de87b7f68ddb56ac9f43cc97

[pixelzoom]

@samreid and I worked on this.

Open issues:

1. Is it OK to put license info into html deliverable as JSON?
2. Verify with @kathy-phet that the omitted spreadsheet columns were specific to contracts, and see if she needs a snapshot before deleting the spreadsheet.

[jonathanolson]

• Is requirejs-text still used? Was it ever used?

Replaced by PhET's own text.js plugin (see chipper).

It seems string.js uses the text plugin directly:

var text = require( 'text' );

I also don't see a 'text.js' in chipper, can you clarify the comment @pixelzoom?

[samreid]

I see, it does look like text-2.0.12.js has many usages.

[pixelzoom]

I'll restore the entry for text-2.0.12.js.

[pixelzoom]

There are 2 libs that have entries in info.json, but no actual file in sherpa/lib/.

"reveal-3.1.0" is addressed in https://github.com/phetsims/scenery/issues/435

"pegjs-0.7.0" is addressed in https://github.com/phetsims/kite/issues/48

[pixelzoom]

There are entries in info.json that are not bundled into the html deliverable, but are instead used in unit testing or to generate documentation. It's appropriate to have them in info.json because there are associated files in sherpa/lib/. Currently, only the "notes" field indicates how libs are used.

Related questions: • What is the audience for the human-readable version of info.json? • Do we need to filter out entries that don't appear in the html deliverable, or is it OK to show all entries in the human-readable version? If we need to filter, then we need an additional field in info.json that indicates whether an entries may be bundled in an html deliverable.

[pixelzoom]

Email to @kathy-phet:

Hi Kathy,

The Google spreadsheet for 3rd-party libraries is at: https://docs.google.com/spreadsheets/d/1XWlHFidztUT6ARb7yHQ1Naepd6L0x9LF94AAoKHGaWg/edit?usp=drive_web

As we discussed recently, this information is being consolidated in sherpa/info.json. Aaron will be generating human-readable output from info.json. And the Google spreadsheet will be deleted. This is being tracked in https://github.com/phetsims/sherpa/issues/22

There are 3 columns that have not been propagated to info.json: “Compliance in Beer’s Law Lab”, “Compliance on GitHub” and “Added 7/21/2014”. I asked Sam about these columns. He couldn’t recall their purpose or semantics, and thought that perhaps they were requested by you for specific contracts/agreements.

Questions: • If you recall their purpose, please describe these 3 columns. • Is it OK that the information in these 3 columns is not propagated to info.json? • Do you need a snapshot of the Google spreadsheet prior to deleting it?

You can reply to this email, or add a comment to https://github.com/phetsims/sherpa/issues/22

Thanks,

Chris

[pixelzoom]

Still waiting for feedback from @kathy-phet on this. This is a prerequisite to moving forward with "Summer 2015 redeploy".

[kathy-phet]

Hi Chris et al.,

The Google Spreadsheet should not be deleted. The content can be cleared and a redirect link put into the first cell -- but the spreadsheet link needs to live on.

The purpose of these 3 columns were for specific contracts, but maintaining this information in not needed. The new information will be sufficient - as long as a link is used to redirect anyone with this google doc link.

It would be good to capture and archive the Google Spreadsheet information before clearing, for prosperity. But again, please do not delete as we will need a way to redirect those with the Google link to the new approach.

Thanks, Kathy

[pixelzoom]

Got it.

@aaronsamuel137 when you create the human-readable form, please replace the contents of the Google spreadsheet with a link to the URL where the human-readable form can be viewed.

[aaronsamuel137]

Sounds good. Feel free to assign to me when it is ready to go.

[pixelzoom]

@kathy-phet would like this comment added:

// Licenses for third-party software used by this simulation. 
// The format is JSON, where each entry describes a third-party license.
window.phet.chipper.thirdPartyLicenses = {

[samreid]

I added comment text above the 3rd party licenses, as requested in this comment https://github.com/phetsims/sherpa/issues/22#issuecomment-113234292

I tried building hookes-law and it looked like so:

<script type="text/javascript">
  window.phet = {};
  window.phet.chipper = {};
  window.phet.chipper.project = 'hookes-law';
  window.phet.chipper.version = '1.0.0-dev.17';
  window.phet.chipper.buildTimestamp = '2015-06-22 23:37:13 UTC';
  window.phet.chipper.locale = 'en';

  // Licenses for third-party software used by this simulation. 
  // The format is JSON, where each entry describes a third-party license.
  window.phet.chipper.thirdPartyLicenses = {
  "almond-0.2.9": {
    "text": [
      "almond 0.2.9",
      "Copyright (c) 2011-2014, The Dojo Foundation All Rights Reserved."
    ],
    "license": "The MIT License",
    "projectURL": "https://github.com/jrburke/almond",
    "notes": "Allows a RequireJS-based simulation to run in its compiled (smaller, faster) form"
  },

[samreid]

@aaronsamuel137 what additional work or information are you waiting on before you start work on this? Also, I recall at a recent meeting that JSON text like in the above comment is not completely opaque--should we discuss just using the JSON instead of filtering + maintaining it? What did you have in mind for human-readable?

[aaronsamuel137]

I am not waiting for anything except the issue being assigned to me. I haven't followed the details too closely, so I haven't been sure if everything was ready yet.

I agree that the JSON is not too opaque, but I think the idea is to get a nice looking md file that we can redirect to from http://phet.colorado.edu/third-party-libraries. (see https://github.com/phetsims/website/issues/208).

I haven't thought too much about how it will look visually. Maybe a table? I am open to suggestions if you have some.

[samreid]

Reading through the comments from above, this seemed like a good idea:

@aaronsamuel137: Write a script to generate something that is human-readable from info.json (GitHub markdown?). Make the output accessible at the URL that has been published for the spreadsheet, Make the output accessible at the URL that has been published for the spreadsheet, replace contents of the spreadsheet with URL (do not delete spreadsheet). Consider running script automatically when a build is done, so that human-readable form stays up-to-date.

Github markdown provides a nice table output, so perhaps that would work well here. @aaronsamuel137 please feel free to start on this at your convenience and let me know if/when you have questions or comments.

[samreid]

@aaronsamuel137 please see related issue https://github.com/phetsims/chipper/issues/158

[pixelzoom]

Re the spreadsheet...

@kathy-phet said:

But again, please do not delete as we will need a way to redirect those with the Google link to the new approach.

So I edited the original instructions and @samreid's quote directly above to say:

Make the output accessible at the URL that has been published for the spreadsheet, replace contents of the spreadsheet with URL (do not delete spreadsheet).

[aaronsamuel137]

Added iteration 1 of the table markdown generator. To use, run

grunt generate-markdown

from chipper root. This will commit and push to sherpa.

Currently, the format is a markdown table which requires horizontal scrolling. It might be a good idea to look into other formats that don't require scrolling.

[samreid]

@aaronsamuel137 & I reviewed the sample output at https://github.com/phetsims/sherpa/blob/a1cd1f4d425c127226635229f7aa06d096395f92/info.md and decided to investigate a non-tabular MD output. Assigned to @samreid to look into this since it relates significantly to https://github.com/phetsims/chipper/issues/162

[samreid]

I've been working on a non-tabular output and it seems like a good direction to go in. I'm getting tired of having changes on my working copy so I'm going to commit what I have so far even though this is not yet complete.

[samreid]

I committed my partial progress above, reassigning to myself to continue work in the future.

[samreid]

Here is a sample of the current output:

text-2.0.12.js RequireJS text 2.0.12
Copyright (c) 2010-2014, The Dojo Foundation All Rights Reserved. http://github.com/requirejs/text

• Available via the MIT or new BSD license. undefined
• Loads strings to be displayed in the sim, including internationalized strings.

three-r68.js Three.js r68
Copyright © 2010-2014 three.js authors http://threejs.org

• The MIT License https://github.com/mrdoob/three.js/blob/master/LICENSE
• For the 3D view (WebGL and Canvas)

Tween-r12.js Tween.js r12
Copyright (c) 2010-2012 Tween.js authors. https://github.com/sole/tween.js/

• The MIT License. Easing equations Copyright (c) 2001 Robert Penner under BSD license see http://robertpenner.com/easing_terms_of_use.html
• Used to smoothly animate the speedometer and accelerometer to the left and back in Forces and Motion Basics.

typedarray-44f09cf.js typedarray
Copyright © 2010 Linden Research, Inc., Copyright © 2014 Joshua Bell https://github.com/substack/typedarray

• The MIT License https://github.com/substack/typedarray/blob/master/LICENSE
• Typed array polyfill

[samreid]

Still waiting for feedback from @kathy-phet on this. This is a prerequisite to moving forward with "Summer 2015 redeploy".

I'm not sure why this was tagged as for Summer 2015 Redeploy since it seems to be about updating the metadata outside of the simulations rather than the actual build artifacts. @pixelzoom can you comment on why the S2015R label was added and whether it is still necessary?

[samreid]

Here's another way of indenting it, but unfortunately relies on a bunch of   for whitespace which shouldn't be used directly in the HTML.

three-r68.js   Three.js r68
  Copyright © 2010-2014 three.js authors   http://threejs.org   The MIT License https://github.com/mrdoob/three.js/blob/master/LICENSE   For the 3D view (WebGL and Canvas)

Tween-r12.js   Tween.js r12
  Copyright (c) 2010-2012 Tween.js authors.   https://github.com/sole/tween.js/   The MIT License. Easing equations Copyright (c) 2001 Robert Penner under BSD license see http://robertpenner.com/easing_terms_of_use.html   Used to smoothly animate the speedometer and accelerometer to the left and back in Forces and Motion Basics.

typedarray-44f09cf.js   typedarray
  Copyright © 2010 Linden Research, Inc., Copyright © 2014 Joshua Bell   https://github.com/substack/typedarray   The MIT License https://github.com/substack/typedarray/blob/master/LICENSE   Typed array polyfill

[samreid]

Here's the same thing without indents:

three-r68.js Three.js r68
Copyright © 2010-2014 three.js authors http://threejs.org The MIT License https://github.com/mrdoob/three.js/blob/master/LICENSE For the 3D view (WebGL and Canvas)

Tween-r12.js Tween.js r12
Copyright (c) 2010-2012 Tween.js authors. https://github.com/sole/tween.js/ The MIT License. Easing equations Copyright (c) 2001 Robert Penner under BSD license see http://robertpenner.com/easing_terms_of_use.html Used to smoothly animate the speedometer and accelerometer to the left and back in Forces and Motion Basics.

typedarray-44f09cf.js typedarray
Copyright © 2010 Linden Research, Inc., Copyright © 2014 Joshua Bell https://github.com/substack/typedarray The MIT License https://github.com/substack/typedarray/blob/master/LICENSE Typed array polyfill

[pixelzoom]

@pixelzoom can you comment on why the S2015R label was added and whether it is still necessary?

I believe the goal was to have licensing correct and complete before new sims were released. Verify with @ariel-phet, because this is not my call.

[samreid]

In @aaronsamuel137's original work, this was set to automatically commit every time the task grunt generate-markdown is run. I restored that functionality above.

[samreid]

I think we should give this a more specific name such as generate-third-party-license-markdown. Or, since it is committing it as well, perhaps update-third-party-license-spreadsheet

[samreid]

@jonathanolson @jbphet and @pixelzoom do you want this to auto-commit or just update your working copy and put a message on the screen to remind you to commit?

[samreid]

Also to discuss. Who/when should this task run? Possibly with every deploy, just so we never forget?

[samreid]

I believe the goal was to have licensing correct and complete before new sims were released. Verify with @ariel-phet, because this is not my call.

Reassigning to @ariel-phet to comment on whether this issue should be labeled as Summer 2015 Redeploy.

[samreid]

More generally, here are all of our 10 open licensing issues: https://github.com/issues?utf8=%E2%9C%93&q=is%3Aissue+sort%3Aupdated-desc+is%3Aopen+label%3Alicensing+user%3Aphetsims+

We should decide which, if any, are prerequisite for the Summer 2015 Redeploys.

[pixelzoom]

@jonathanolson @jbphet and @pixelzoom do you want this to auto-commit or just update your working copy and put a message on the screen to remind you to commit?

I'm ok with an auto commit and push.