dwyl / image-classifier

🖼️ Classify images and extract data from or describe their contents using machine learning
GNU General Public License v2.0
20 stars 3 forks source link

[PR] Creating basic image classifier with Elixir #2

Closed LuchoTurtle closed 10 months ago

LuchoTurtle commented 11 months ago

closes #1

This creates a simple image classifier with Bumblebee and performs image captioning tasks that return a descriptive text of the image (not to be confused with image classification, which yields a weighted list of predictions).

LuchoTurtle commented 11 months ago

I'm having immense trouble previewing the image and copying it to the DOM before consuming it. In order to actually use the image to classify it, I need to consume it. However, this effectively removes the image entry from the temporary folder and I lose access to it to show to the person.

I've tried using the image_live_preview function but it only works if the entry is not consumed, which is not possible in our scenario.

I'm trying to circumvent this by using an additional <input>, copying the file contents and displaying it in the DOM. But this, for some odd reason, makes it so that handle_progress/3 is not called, which in turn doesn't consume the entry.

allow_upload(:image_list,
       accept: ~w(image/*),
       auto_upload: true,
       progress: &handle_progress/3,
       max_entries: 1,
       chunk_size: 4_000,
       max_file_size: 64_000
     )}

I've tried many ways of doing this but I can't seem to figure it out. I could copy the file path to an upload directory, but that would mean that I'd need to manage all of the uploaded images, their correct deletion (even if the person crashes the LiveView), which is something I really don't want to "waste" time on.

nelsonic commented 11 months ago

@LuchoTurtle maybe a quick sketch (paper notebook flow diagram) of what you're trying to do will help others on the team understand where you're stuck. 💭

LuchoTurtle commented 11 months ago

I've added virtually all of the documentation and the application is now functional. I'm just having some trouble with having the tests covering 100% of the application. Because I'm using Task.async, I could technically add a sleep method for it to wait for the model to predict a given image. However, I feel like that's a "hack"/dirty copout.

I've been researching on how to get this to work, namely form:

But haven't had good results on this.

I'm trying to implement a supervisor to do this (from https://elixirforum.com/t/best-way-of-doing-an-integration-test-that-spans-a-task-start/39429/2) but I'm struggling to get it to work.

Other than that, the app works (although it does not show the image after submitting, which is something I wanted to get to work as well, but it's not "essential")

codecov[bot] commented 11 months ago

Welcome to Codecov :tada:

Once merged to your default branch, Codecov will compare your coverage reports and display the results in this comment.

Thanks for integrating Codecov - We've got you covered :open_umbrella:

LuchoTurtle commented 11 months ago

This should be mergeable 👌

nelsonic commented 11 months ago

@LuchoTurtle demo on Standup this morning was superb! 😍 PR is still assigned to you. Happy to review it later today. 👌 Meanwhile there are a couple of tasks you can crack on with: #3 & #4 😉

LuchoTurtle commented 11 months ago

I've reassigned it to myself because I want to make some additional changes. I'll hand it over to you after those.

nelsonic commented 11 months ago

cool-cool-cool

LuchoTurtle commented 11 months ago

Reassigning back to you 👌

nelsonic commented 10 months ago

Gonna pick this up (run + review) as soon as I get a decent chunk of alone time.... 🤞 ⏳