New data to be uploaded Inca City Season 4 observations

[mschwamb]

Please upload take these HiRISE images of Inca City Season 4 and make them into cutouts and uploaded onto P4 as the next data set to be classified (should they be it's own group?)

ESP_030084_0985 ESP_030229_0985 ESP_029886_0985 ESP_029596_0985 ESP_029240_0985 ESP_029227_0985 ESP_029095_0985 ESP_028911_0985 ESP_028910_0985 ESP_028884_0985 ESP_028752_0985 ESP_030163_0985 ESP_029807_0985 ESP_029741_0985 ESP_029662_0985 ESP_029108_0985

Stuart has the original code that made the cutouts for launch - there a few issues related to data upload if possible to be fixed at the same time

• [ ] See #115
• [ ] See #97

Can we aim for an mid to end of October date for getting the data live if that works with the development team schedule? If not, can you please let us know when would it might be possible to get the data prepared and uploaded?

[chrissnyder]

Is there any other metadata associated with each of those regions? Last round had a CSV file that looked something like https://docs.google.com/spreadsheets/d/1CXFAixqbi0WPiLpgfvFz5RAEjp4FvvgbDSq4y-YlO-4/edit?usp=sharing

[mschwamb]

Yeah, but unfortunately you need to scrape the HiRISE website pages. @parrish should have a script that parses the html of the HiRISE page ( ie http://hirise.lpl.arizona.edu/PSP_003309_0945) to get the metadata. I believe this is what was done at launch, and I think as well for the Season 1 data. Though one note, that script grabs the incorrect sub-solar azimuth and north azimuth (there is two on the HiRISE page we want the one for non-map projected ticket #97). The script currently grabs the map projected values for sub-solar azimuth and north azimuth. Everything else the script grabs for metadata is correct.

[chrissnyder]

Not to create additional steps, but here's all the metadata for the cutouts:

http://zooniverse-demo.s3.amazonaws.com/cutouts.json

If that appears correct, I'll crop them and get them prepped for uploading.

[mschwamb]

Happy to check. I looked at the first entry and then a random one lower in the list. The information is correct compared to the HiRISE webpages. Only thing to note is that in the api status called for a cutout some of the values you report as strings with the degree symbol are floats.

APF000013y

{"id":"50e73bf75e2ed2124000059e","activated_at":"2014-04-21T16:52:25Z","classification_count":30,"coords":[-81.321,295.719],"created_at":"2013-01-06T21:31:12Z","location":{"standard":"http://www.planetfour.org/subjects/standard/50e73bf75e2ed2124000059e.jpg"},"metadata":{"name":"ESP_013113_0985","area":265.501,"time":"2009-05-14T11:13:50Z","acquisition_date":"2009-05-14T00:00:00Z","local_mars_time":"4:05 PM","lat_centered":-81.329,"lng_east":295.74,"emission_angle":2.8,"phase_angle":63.2,"north_azimuth":205.7,"sub_solar_azimuth":140.2,"region":[[-81.157501220703,295.67498779297],[-81.181900024414,295.15301513672],[-81.500099182129,295.80200195312],[-81.474899291992,296.33999633789]],"link":"http://hirise-pds.lpl.arizona.edu/download/PDS/EXTRAS/RDR/ESP/ORB_013100_013199/ESP_013113_0985/ESP_013113_0985_RGB.NOMAP.JP2","x_tile":1,"y_tile":12,"filename":"_r12_c1.jpg"},"project_id":"50e9e3d33ae740f1f3000001","random":0.4626432333365924,"state":"complete","updated_at":"2013-01-06T21:31:12Z","workflow_ids":["50e9e3d33ae740f1f3000002"],"zooniverse_id":"APF000013y"}

[chrissnyder]

Just a matter of parsing out the degree symbol mostly.

Will do that and get example metadata for potential subjects up this weekend sometime.

[mschwamb]

A quick note to say whatever time works for development team works for us for when these go live. The public vote images are almost all out so we can send a combined newsletter out announcing the vote images and the new data whenever fits with the Zooniverse's timeline.

. To summarize the previous and recent listserv conversation, there was talk about adding in the few season 5 images we are getting made public from the vote but we we're leaning towards about adding those to the site as a birthday present for the 2nd anniversary (if the development team agrees and its not too much trouble to get them in in early January).

[chrissnyder]

Do we use the area and region data contained in the metadata of a subject?

If not, I'm ready to process all the cutouts into subjects. I'll post an example here as soon as the cutouts from HiRise are done downloading.

[parrish]

Do we use the area and region data contained in the metadata of a subject?

I'm not exporting them in the data and we don't use them in the app as far as I'm aware.

[mschwamb]

I don't know quite what you mean by region and area? I've never seen a copy of the mongo db. As long as for consistency the new cutouts look like the old ones with the info when you querry the api than I'm happy:

{"id":"50e73bf75e2ed2124000059e","activated_at":"2014-04-21T16:52:25Z","classification_count":30,"coords":[-81.321,295.719],"created_at":"2013-01-06T21:31:12Z","location":{"standard":"http://www.planetfour.org/subjects/standard/50e73bf75e2ed2124000059e.jpg"},"metadata":{"name":"ESP_013113_0985","area":265.501,"time":"2009-05-14T11:13:50Z","acquisition_date":"2009-05-14T00:00:00Z","local_mars_time":"4:05 PM","lat_centered":-81.329,"lng_east":295.74,"emission_angle":2.8,"phase_angle":63.2,"north_azimuth":205.7,"sub_solar_azimuth":140.2,"region":[[-81.157501220703,295.67498779297],[-81.181900024414,295.15301513672],[-81.500099182129,295.80200195312],[-81.474899291992,296.33999633789]],"link":"http://hirise-pds.lpl.arizona.edu/download/PDS/EXTRAS/RDR/ESP/ORB_013100_013199/ESP_013113_0985/ESP_013113_0985_RGB.NOMAP.JP2","x_tile":1,"y_tile":12,"filename":"_r12_c1.jpg"},"project_id":"50e9e3d33ae740f1f3000001","random":0.4626432333365924,"state":"complete","updated_at":"2013-01-06T21:31:12Z","workflow_ids":["50e9e3d33ae740f1f3000002"],"zooniverse_id":"APF000013y"}

[mschwamb]

Not sure if you're using Stuart's script or not. If you're not there should be something like ~100 pixel overlap on two sides of the cutout with an adjacent cutout.

[chrissnyder]

I used Stuart's script as a starting point, but re-wrote bits of it. Yes, cutouts will still have overlaps.

[mschwamb]

Cool. Thanks.

[chrissnyder]

http://zooniverse-demo.s3.amazonaws.com/manifest.json

If these look correct, we can import whenever.

[mschwamb]

there's a weird -0600 in the acquisition date - any chance we can remove that so it matches season 1, 2, and 3?

I've confirmed the overlap by eye as well. So looks fab other than the -0600.

Thank you Chris. Since you did all the hard, after the import do you want send the email to the P4 list to say the data is ready to go live so Candy can give the say so for the subjects to be unpaused?

~Meg

[parrish]

Imported and active. Also, @mschwamb, I fixed the acquisition date time zones.