search

pamaxie / --OLD--Pamaxie

Worker "clients" for analyzing media content. Pamaxie is a Natural Learning API for accessing several Neural Networks and hand-crafted algorithms for moderating content on platforms.
https://pamaxie.com
Apache License 2.0
2 stars 2 forks source link

Rewrite Pamaxies Tooling #39

Closed ghost closed 3 years ago

ghost commented 3 years ago

Pamaxies tooling should be rewritten to be easier to use for people, preferably containing a UI. This way people can train custom datasets more easily and quickly.

ghost commented 3 years ago

Pamaxie.ImagePreperation seems to be done so far. I'm doing a testrun converting roughly 700k images to see if everything works with a decent enough stability. If it does a release for that part will be made.

ghost commented 3 years ago

Testrun completed successfully. This means that Pamaxie.ImagePreperation has been implemented in the new API. We now need to move: Pamaxie.Leecher (seperate issue is: #41 ) - Image Crawler for the web to automatically crawl images from websites to make data collection easier. Pamaxie.Test (doesn't exist yet) - Validation Software for testing if our neural networks work reliable against datasets of roughly 1 million Images, preferably this implements our cluster managment system from #40 because otherwise the re-validation of datasets with a ton of images can take forever (This is to give statistics on reliability & false positives). Pamaxie.ImageSorting - Validates if images are doupes and copies a ton of files from one location to the other. THIS IS A BIT MORE COMPLICATED SINCE IT NEEDS TO BE REALLY FAST (our datasets exceed 1 million images easily). Pamaxie.ImageRenaming - Not needed anymore. Could be included into Pamaxie.ImagePreperation module of hte new UI application.

All the above except Pamaxie.Leecher should be moved into the new Project called "Pamaxie.ImageTooling". This new project will be a UI application that includes all tooling required for images. Pamaxie.Leecher should be in a new application called Pamaxie.Crawler which will include all kinds of media crawlers we will use to collect data over the internet. Pamaxie.Leecher should be done seperately and will need a ton of rewrites.

ghost commented 3 years ago

Rewrite for Pamaxie is so far complete. We will add more functionality later on. If anything is missing please let me know.