Method to Export Cropped Screenshots of All Labeled Artifacts

[jonfroehlich]

For future CV work, we need a method that exports a cropped rectangle of all labeled artifacts along with metadata about that rectangle (e.g., similar metadata to what we had for Tohme for labelled spots).

[jonfroehlich]

This issue is now a huge priority as @Soben713 and @manaswis have both noticed that some previously labeled image data is no longer available (see https://github.com/ProjectSidewalk/SidewalkWebpage/issues/423). It's not possible with our current system to infer how much data is affected.

Consequently, to eliminate this problem in the future, this is what I think we should do:

• On the backend, when labels are uploaded, we should automatically extract a bounding box (of sufficient size) cropped image where that label exists (along with position information of that label within that cropped image)
• We should also simulataneously implement this for all previously labeled data.
• We could also start automatically saving full panoramas like we did for the Tohme project; however, this may break Google's terms of service and it's also a lot of data

I would like @tongning to take the lead on this with support from @Soben713

[tongning]

Are there any tried-and-tested tools for extracting street view images? I found an experimental node package but I'm wondering if something better exists.

[jonfroehlich]

No idea. @kotarohara made something for Tohme that he could likely send you. We can scrape both the panorama and the 3D depth LiDar data.

[tongning]

Thanks Jon, @kotarohara sent me some scripts. Hope to try them out later tonight.

[jonfroehlich]

Ok. Great!

Sent from my iPhone

On Nov 17, 2016, at 2:13 PM, Anthony Li notifications@github.com wrote:

Thanks Jon, @kotarohara sent me some scripts. Hope to try them out later tonight.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[misaugstad]

This is covered by issue #539 and a pano scraper has been set up, with the code in this repo, so I'm closing this issue.

[manaswisaha]

This issue is different from just availability of a pano (#539). It is about automatically extracting the part of the pano that relates to the associated label. It is a CV task. So in my opinion, this should remain open.

[jonfroehlich]

Agreed. It's also not a super straightforward task. @tongning actually worked on this quite a bit. I don't see a report of the current state of things anywhere on this though. @tongning can you update this thread, perhaps, with the current state and what you've learned thus far through your experimentation and analysis?

[misaugstad]

Looks like we also have a method of extracting crops from the full panos now too, so I'm closing this now.