Uploading Terrains subjects to panoptes

[mschwamb]

@camallen and @michaelaye I also made a github ticket for uploading the subject set for P4:Terrains. Cam, Michael sent the uploaded links for downloading the data. Please let us know if you have any questions or need anything else from us.

[mschwamb]

I figured this was the right place to put this so it wasn't sitting in the pantopes support tickets.

[camallen]

@mschwamb I've got the data, it's on my list of things todo.

[camallen]

@mschwamb prob easiest if you add me (camallen) and the zooniverse user as collaborators on the project.

[mschwamb]

@camallen This is the Zooniverse public repo for main P4 site - so you should have access to the repo. I figured it was easier to keep Terrains specific issues on the main P4 github issues. Looks like you have owner status for the repo

[camallen]

@mschwamb i meant in the project on panoptes, sorry.

[mschwamb]

oh got it! done! (once it stops giving an Error:0 when I got tot he preview site)

[mschwamb]

@camallen you're added, but no zooniverse account for me to add. Not showing up in the menu list when I type in Zooniverse.

[camallen]

No probs - thanks.

[camallen]

@mschwamb re https://github.com/zooniverse/Panoptes/issues/982, should i split the data into subject sets so you the ability to activate / deactivate data?

[mschwamb]

@camallen If there seriously isn't a way to pause specific subjects (even through direct interaction with the api skipping the project worklfow) then probably yes.

I did the math assuming 365 days and a conservative/optimistic 1000 classifications per day. If that were lower than that we might want to focus on less data.

Maybe by CTX image? which you don't actually have in the metadata because we can't release that on the interface, so we had to not include the data in the manifest file. I'll send you the metadata file

[mschwamb]

@camallen I just sent you the CTX_image to subject image conversion if you want to break them by CTX image or however else you want to divide the subjects is fine I think (we can work around it if we need to 'pause' stuff)

[mschwamb]

@camallen I know you're super busy. If you need anything else for us please let us know. Thanks!

[mschwamb]

Hey @camallen, I note to say that we've gone through beta and that we're planning on launching with the new system and will need these subjects. Is this still doable? Thanks for all the help

[camallen]

@mschwamb timelines are pretty tight but i'll hopefully get something worked out maybe a limited set to launch with. If you had to prioritize different subject sets which would they be?

[mschwamb]

Thanks @camallen . I can only imagine how busy y'all must be. We appreciate any help you can give right now.

there is no strict priority. Also thinking about, we don't need as fine as each CTX image as a separate subset. I imagine eventually there will be a pause feature, so if it's easier. We could break into three big subsets. I or Michael can divide the manifest into three chunks if that would help and if we can get one in for launch that would likely be more than enough to start with and the others can get uploaded post launch if that helps.

[camallen]

@mschwamb the best way would be to split the manifest into the resolution you want to pause subjects, we won't be able to pause so think about the desired sets to recreate this functionality. Get those new manifests to me and i'll get at least one done for launch.

[mschwamb]

@camallen I split the data into thirds based on the original parent images.

https://www.dropbox.com/s/s4of1hsa8lmo34t/manifest1.csv?dl=0 https://www.dropbox.com/s/3nuuu211izz2hw2/manifest2.csv?dl=0 https://www.dropbox.com/s/g3ce9lx2m0wdhsz/manifest3.csv?dl=0

That should provide the minimum flexibility for our needs right now.

If you can at least manifest1 that would be great and should hold us through launch.

[camallen]

@mschwamb i've got the new manifests. Just confirming, there is no additional metadata just the filename. Is this correct?

[mschwamb]

No other metadata's included since we don't want the metadata shown in the interface and currently everything in the manifest files shows up in the metadata since that would reveal what camera image we're looking at and could bias the result

[mschwamb]

@camallen Forgot to say - yep just the filename for now. in the future we might revisit when there are options in the front end/back end for specifying what you show the volunteer and what you store in the database in terms of metadata - thanks for the help

[mschwamb]

Hey @camallen - sounds like launch is tomorrow, but I haven't explicitly been told. I imagine you're super super busy. Please let me know if you think can't get to this and that I need to try uploading part of the first set. when I wake up since I've got a good 6hours ahead of UK awake up to try

[camallen]

@mschwamb i'm just starting it now

[mschwamb]

I owe you a few beers @camallen when I see you next

[camallen]

@mschwamb data from manifest 1 is all uploaded and in the system, 8436 in total.

[michaelaye]

Thanks Cam!

Sent from my iPhone

On Jun 25, 2015, at 08:02, Campbell Allen notifications@github.com wrote:

@mschwamb data from manifest 1 is all uploaded and in the system, 8436 in total.

— Reply to this email directly or view it on GitHub.

[mschwamb]

Thanks @camallen!

[mschwamb]

@camallen feel free to say no to this - absolutely your call.

I'm noticing when I classify that the same two types of terrains showing up. I'm seeing alot of swiss cheese terrain and south polar layered deposits. It might be since no one else is classifying and small number stats, but I think it might be luck of the random draw when I divided up the full frame images into the 3 manifests. Uploading the second manifest might give more images to choose from and make it less repeaty If launch is delayed a few days and you've got time to stick a script going.

I fully understand things are extremely busy so no is perfectly acceptable response to this request

[mschwamb]

Closing - since the science team can do this without intervention from the development team. Thanks for the help @camallen