Soapie

[daeus]

Project Lead: @cheukting @daeus

Mentor: @petergrabitz

Welcome to OL7, Cohort D! This issue will be used to track your project and progress during the program. Please use this checklist over the next few weeks as you start Open Leadership Training :tada:.

---

Before Week 1 (Jan 30): Your first mentorship call

• [x] Complete the OLF self-assessment (online, printable). If you're a group, each teammate should complete this assessment individually. This is here to help you set your own personal goals during the program. No need to share your results, but be ready to share your thoughts with your mentor.
• [x] Make sure you know when and how you'll be meeting with your mentor.

Before Week 2 (Feb 6): First Cohort Call (Open by Design)

• [x] Complete Assignment 'Write Your Vision or Mission Statement'. Leave a comment on this issue with your draft vision statement & be ready to share this on the call.
• [x] Watch 'Mozilla & Working Open'.
• [x] Check the Syllabus for notes and connection info for all the cohort calls.

Before Week 3 (Feb 13): Mentorship call

• [x] Look up two other projects and comment on their issues with feedback on their vision statement.
• [x] Complete your Open Canvas (instructions, canvas). Comment on this issue with a link to your canvas.
• [x] Start your Roadmap. Comment on this issue with your draft Roadmap.

Before Week 4 (Feb 20): Cohort Call (Build for Understanding)

• [ ] Look up two other projects and comment on their issues with feedback on their open canvas.
• [ ] Pick an open license for the work you're doing during the program.
• [ ] Use your canvas to start writing a README, or landing page, for your project. Link to your README in a comment on this issue.

Week 5 and more

• [ ] Create a repository for your project. New to GitHub? You can fork our templates.
• [ ] Add Contributing Guidlines
• [ ] Add a Code of Conduct
• [ ] Submit your project to Mozilla Pulse, add the tag #mozsprint
• [ ] Invite new contributors to into your work!

This issue is here to help you keep track of work as you start Open Leaders. Please refer to the OL7 Syllabus for more detailed weekly notes and assignments past week 4.

[daeus]

Vision Statement: We are working with Software Engineers and Data Scientists to build an open source auto-captioning tool to blind people so that they can use “see” the images on the internet. It's also enabling developers easier way to integrate auto-captioning as a part of web accessibility.

[daeus]

Open Canvas: https://docs.google.com/presentation/d/1te3tF-5N03oAU44CFYu1yTQvfXLomi1mVIOG4SZXgU8/edit?usp=sharing

[daeus]

Project Mission & Summary

Soapie is the most simple image auto-captioning model and tool for dev to generate caption for blind people as a part of accessibility goal. Although the model isn't new in the academic field, it is still not popular in actualising it as a daily tool that most people can access to. We want to build a very simple one as an intiation.

Roadmap

In order to achieve the goal, we have defined the following milestones to achieve.

Milestone 1: Architecture Design

• [ ] Design the architecture of the project.
• [ ] Define the ML model we are going to use.
• [ ] Design how tool would be on top of the model

Milestone 2: Kickstarting

• [ ] Create GitHub repository
• [ ] Add Readme
• [ ] Add Code of Conduct
• [ ] Add contributor guidelines
• [ ] Add product code into the repo (non-workable/non-sense is okay for this milestone)

Milestone 3: Model Development

• [ ] Add more code
• [ ] Add more and more code
• [ ] Add more and more and more code
• [ ] Develop a simple workable ML model
• [ ] Add pre-trained data into repo

Milestone 3: Tooling Development

• [ ] Review how it can be wrapped it up as a tool
• [ ] Start adding tooling related code
• [ ] Add more code
• [ ] Add more and more code
• [ ] Add more and more and more code
• [ ] Add simple workable tooling code

Milestone 4: Involve More Contributors

• [ ] Discuss a plan and add further milestone

[annefou]

Great initiative! I really interested to understand more on how this will be achieved.

[victordiaz]

Very interested on this! I would love to get a hint on how you plan to do the auto-captioning and if there is a way to create some context aware captioning. With this I mean, rather than analyzing the image and put some tags like "dog and man wearing blue", it could link it with the content text. So if the article is talking about "John loves dogs" then the caption would read "John is wearing blue and walking his dog Mozzy".

Other than this, very nice vision statement!

[msoka86]

Great project and great roadmap, I will be following this closely!

[jmtaylor86]

Open Canvas: https://docs.google.com/presentation/d/1te3tF-5N03oAU44CFYu1yTQvfXLomi1mVIOG4SZXgU8/edit?usp=sharing

Your project is incredibly interesting! Is the idea that web page developers will then be able to incorporate this code into their pages?

[Cheukting]

Very interested on this! I would love to get a hint on how you plan to do the auto-captioning and if there is a way to create some context aware captioning. With this I mean, rather than analyzing the image and put some tags like "dog and man wearing blue", it could link it with the content text. So if the article is talking about "John loves dogs" then the caption would read "John is wearing blue and walking his dog Mozzy".

Other than this, very nice vision statement!

Thanks @victordiaz Context aware captioning is exactly what I think we need. However, it seem with the current available technology, it's still far from prefect. I am guessing it will also involve some NLU for understanding the context. It will be great if you have any directions that can point us to.

[Cheukting]

Open Canvas: https://docs.google.com/presentation/d/1te3tF-5N03oAU44CFYu1yTQvfXLomi1mVIOG4SZXgU8/edit?usp=sharing

Your project is incredibly interesting! Is the idea that web page developers will then be able to incorporate this code into their pages?

Thanks @jmtaylor86 Yes hopefully, we are still designing what would be the best to implement this so it will be user friendly for web developers. If you have any ideas, please feel free to let us know.