Next Step Microgrant: Operation Dragonborn

somya-15 commented 1 year ago

1. What is your project, and what problem does it solve? (max 100 words)

✨ In D&D, it is compulsory to create an environment that your character embodies, including an avatar. This isn’t easy for first-timers or people with lack of time or imagination. Our tools will help players with this.

“Operation Dragonborn” is the next step after our Top Quality Chainlink Hackathon project D&D Diffusion. Our sprint is focused on the narrow objective of training an AI art generator to understand the specific branded D&D term “Dragonborn.”

We want to create usable, unique, AI generated visual assets for a game that’s played by 50 million people a year. We are going to bring people together to play games, inspire creativity and save time. Tabletop roleplaying is a multibillion dollar industry and not enough of it is decentralized. Our future platform will connect AI generated art to an NFT marketplace for human artists and allow us to participate in the ongoing conversation about art and media and the role of AI in art creation.

2. Project links

Link to Github repo: https://github.com/DnDnDiffusion

Link to demo or website, if applicable: https://devpost.com/software/dndiffusion

Link to our Notion (this grant application, but prettier): Next-Steps-Grant-Proposal-Operation-Dragonborn

Link to sprint progress:

Next Step Grant Progress

License: MIT

3. How is IPFS, Filecoin, or related technology used in this project? (max 200 words)

Right now in our MVPr, we used NFT.Storage to allow the user to programmatically save their AI generated avatar to decentralized storage, returning the CID after it’s stored to the front-end and enabling the minting of an NFT of that avatar.

The full picture:

NextJS
PDF parser and data transformation (we will be deciding between our two versions, one JS and one Flask)
Stable-diffusion running in a cloud GPU, such as runpod.io
A specially trained model to understand D&D Dragonborn and various classes.
NFT Smart Contract to allow users to save their avatars (and our allowing our front-end to display a marketplace)

Is this project contributing to the upcoming FVM launch? (Yes or No)

NO

4. How will you improve your project with this grant? What steps will you take to meet this objective? (max 200 words)

Imagine you’re playing D&D for the first time, as a Dragonborn Cleric, level one. What do you look like? Step one is to make a character sheet, then you’ll need to find an image or two for your character. Meanwhile, your whole party is waiting to start and saying interesting things about their characters!

During our hackathon, we created all the components required for our use-case but not a complete end-to-end deployment. Operation Dragonborn is a series of sprints to integrate our existing components, and accurately visualize the content of a new player’s character sheet with our custom-trained stable diffusion models. We will deploy a proof of concept for our test group to upload a Fast Character sheet PDF and visualize the race “Dragonborn” from the D&D Open License. Users can save, share or mint it as an NFT.

End-to-end features we will develop in 1-2 sprints for this grant:

Functioning frontend
Parsing PDF character sheets
NFT.Storage full NFT metadata
Tested smart-contract
Cloud GPU running stable diffusion

5. Do you agree to share grant reports upon request, including a final grant report at the end of the three month period?

Yes. We are assembling a team of Technical Sponsors which include Discordian, Dawn Kelly, Developer Ally, Alexandre Nicastro and Jenks Guo. We intend for them to be our first test users, and have already created a group communication channel in the Filecoin Slack.

6. Does your proposal comply with our Community Code of Conduct?

Yes

7. Links and submissions

For each team member(s), please list name, email, Github account, and role in the project.
- Tippi Fifestarr: HR, Product & DevRel
  - fifestarr21@gmail.com
  - https://github.com/tippi-fifestarr
- Somya Tomar: Developer (R&D, full-stack, ML), Grant Liaison
  - somyatomar15@gmail.com
  - https://github.com/somya-15
- Danjo the 0x: Chief Technology Officer
  - danjo@danj0x.com
  - https://github.com/danj-O
- Jason Demma: Creative Director & Chief Humanity Officer
  - creatjve@gmail.com
  - jasondemma.com
- Aire: Project Manager and Development Intern
  - aire.forwork@gmail.com
  - https://github.com/airecr
- Cromewar: Just a regular guy
  - vasiliy@cromewar.com
  - https://github.com/cromewar
How did you learn about our microgrant program?
- Tippi introduced our team to this grant.
If your project was created as part of an event or hackathon:
- Chainlink Fall 2022: https://devpost.com/software/dndiffusion

Additional questions:

What DnD race should we do next to train our AI model?

ErinOCon commented 1 year ago

Hi @somya-15, thank you for your proposal! This grant has been approved. We have sent an email to confirm payment details.