search

bgorman3 / ITSC-3990-AI-Art-Authentication

New Research Project for ITSC 3990 Using AI to Authenticate Art
1 stars 0 forks source link

Research Roadblocks #2

Closed bgorman3 closed 2 weeks ago

bgorman3 commented 1 month ago

Research Roadblock #1: No access to Gemini. With Google AI image generation, I can use DALL-E 3 with Copilot, but I was looking to use Midjourney, Stability AI, and other paid platforms.

bgorman3 commented 1 month ago

I would need these platforms in order to obtain a well-rounded dataset that encapsulates AI-generated images to test with my binary classifier.

btdobbs commented 1 month ago

How did your meeting with Tyler go? Is funding available through 3990? If not, I might have some funds for help.

btdobbs commented 1 month ago

Please see CCI instructions for tools that may help you below. Can you please verify if these work for your project?

Microsoft Copilot with Data Protection was enabled for campus users last month, and Google Gemini with Data Protection should be enabled soon. Here are links to knowledge base (KB) articles for Copilot login and other related topics. When you log into copilot.microsoft.com using your username@charlotte.edu you should see a green shield in the upper right corner of the browser.

https://services.help.charlotte.edu/TDClient/33/Portal/KB/ArticleDet?ID=4030 https://services.help.charlotte.edu/TDClient/33/Portal/KB/?CategoryID=572

OneIT maintains an Artificial Intelligence website that includes an AI Software Guidance web page. https://oneit.charlotte.edu/artificial-intelligence https://oneit.charlotte.edu/ai-software-guidance

btdobbs commented 1 month ago

Also shared claude-monet.zip via google drive because it was too large to upload to GitHub

bgorman3 commented 1 month ago

I got the google drive link, and both tools seem to be working well I haven't looked into automating the image generation process just yet but having Gemini pictures to use is helpful.

btdobbs commented 1 month ago

That's great news! Let's discuss more during our next meeting.

bgorman3 commented 2 weeks ago

New Roadblock: When training a one-class binary classification model, my model becomes overfitted, and guesses everything is a Monet painting rather than learning the actual content. There is an extreme class imbalance.

bgorman3 commented 2 weeks ago

One solution I have been working on is to use an outlier dataset while keeping my problem framed as binary classification, even with severe class imbalance. Essentially, the goal is to treat the non-Monet paintings (the minority class or outliers) as examples of the second class, while still leveraging binary classification techniques.

bgorman3 commented 2 weeks ago

If that doesn't work then I might change to a One-Class SVM (Support Vector Machine) which will train on only one class and flag outliers as non-Monet paintings.

btdobbs commented 2 weeks ago

New Roadblock: When training a one-class binary classification model, my model becomes overfitted, and guesses everything is a Monet painting rather than learning the actual content. There is an extreme class imbalance.

Per our discussion today, I wanted to clarify that only Monet paintings should be in the folder classified as Monet. The same number of paintings should be in the not Monet folder and all of those paintings should not be Monet. You would then train your binary classifier model using both folders. You mentioned that you are modeling using multi-classification code from Kaggle so you may need to tweak the code a little :)

If your code doesn't perform the testing step, you can use the generated model with your test data (equal parts monet/non-monet paintings) and manually calculated metrics (i.e. accuracy, F1, etc).

Perhaps this is what you were saying today, but it sounded like you were mixing both Monet and Non Monet files in the Monet folder.

btdobbs commented 2 weeks ago

One solution I have been working on is to use an outlier dataset while keeping my problem framed as binary classification, even with severe class imbalance. Essentially, the goal is to treat the non-Monet paintings (the minority class or outliers) as examples of the second class, while still leveraging binary classification techniques.

After we clarify what your code is doing, I think this will not be an issue.

btdobbs commented 2 weeks ago

If that doesn't work then I might change to a One-Class SVM (Support Vector Machine) which will train on only one class and flag outliers as non-Monet paintings.

It's probably too late to switch to SVM because you would need to determine which image features to use with SVM.

btdobbs commented 2 weeks ago

Also, adding a separate issue per question will facilitate managing different thoughts. I will close existing issues to promote that.