Open AlonDan opened 3 years ago
NNNNAI
commented
3 years ago Thanks for your advice. I am implementing Replace'SPECIFIC' Multiple Faces, and I will release the relevant code recently. Then about the GUI, because I am a bit busy lately, I will discuss with my team about the GUI. If there is a need for you, I will ask you for help at that time. Thank you for your support. If this rep is of great help to you, please star it~. This helps us a lot.
NNNNAI
commented
3 years ago We have released the related code for "Replace 'SPECIFIC' Multiple Faces at once",please go to Inference for image or video face swapping for details.
AlonDan
commented
3 years ago We have released the related code for "Replace 'SPECIFIC' Multiple Faces at once",please go to Inference for image or video face swapping for details.
I'm trying now the multi-specific for the first time, had some issues probably because I use wrong paths (still testing) I'll make it work probably sooner or later.
The Solution / Fix: It seems like: ◼target◼ and ◼source◼ directory names are opposite from what they should, so please just switch them. thanks ahead! 🙏
💡 Suggestion: I also found out that we may not need to separate between: ◼Replace 1 SPECIFIC Face◼ and ◼Replace Multi SPECIFIC Faces◼ we can use only the MULTI since it works on 1 face as well💙
📌 Based on my tests: For example if you only use 03.jpg (on target + source) and delete 01.jpg and 02.jpg or any number. as long as you have parallel on both target and source, it will just IGNORES it and still works exactly as "Replace 1 Specific" with no errors.
So I think we (users) can use the Replace Multi Specifics also as Replace 1 Specific Face, we just need to put one image for target and source if we don't want multi.
Anyway, maybe you can make it a bit simple naming / less-directories etc..
💡 More Suggestions: (let's make things much more User-Friendly and EASY to use for all) Please consider I'm not a programmer, but I'm a designer so I like to make things EASIER and simple to use, that's why some things may look weird for normal users.
1️⃣ instead of different directories can we just use: SRC_01.jpg (or PNG) and DST_01.jpg (or png) that way we'll just drop everything in same directory structure.
2️⃣ I'm not sure if it's the reason I couldn't make it work, but I looked at the file name of the python:
test_video_swap_mutilspecific instead of "multi" (not a big deal, I know.. just noticed it)
❓ Also I picked inside the file with notepad, which is gibberish for me (since I'm not a programmer) but I did found the same tiny typo, that is repeating few times, it's not a big deal but you may want to fix it.
👉 I see some "mutil" instead of "multi" maybe you'd like to update as well.
The way I make things much easier, for example I have this structure: Inside my MASTER directory: SimSwap (from master zip) I created these directories:
🔹 MAIN directory.
Clean directory that contain all the batch commands for SimSwap (1 click to run SimSwap for the specific task needed, if it's 1 face, multiple, etc..)
🔹 _ SOURCE _ directory
(YES! with the underscore so it's always on most top naming-wise) inside I put all images + video files
And once it's done it will render the result files to:
🔹 _ RESULT _ directory
As you can see it's very easy to use, not only that, I've created some .bat files to do "1 click rename" so it will rename any NEW image I drop in the "SOURCE" directory to image.jpg and any video file to "video.avi" (or mp4, I have separate batch files sitting there) That's the closest way I could make it easier (for myself) to test around with no GUI, just dropping the files, run the code with batch file and it's doing the magic.
The above directories structure is an example for my temporary use until (or if) we'll get GUI:
The reason that I named the files with xyz and numbers is to organize them from TOP to BUTTOM as I like to view by default, again it's for ease of use. 😉
Obviously on every update if there is a new feature / command, I edit my batch files.
My Contribution (as a non-programmer! nothing much I can help with) but: I don't mind sharing the batch files for ease-of-use BUT! it will be nice to ship them with the zipped master code so it will contain the current directories and code structure. This will sure be VERY EASY to use as soon as a user install it via anaconda (in my case, windows + anaconda), just drop your images + video, run the specific batch file, let SimSwap do it's magic... and BOOM! it's done.
It will be best if first I'll wait for the next updates / fixes / tweaks etc..
I hope that my suggestions are helpful to some of you.
NNNNAI
commented
3 years ago I see, it seems a bug for "Multi specific". Let me fix it.
NNNNAI
commented
3 years ago The problem of "multispecific" should be fixed. Currently, the multispecific dir is SRC and DST style. And the misspell of "multi" is solved.Make sure you get the newest code version,and go to Inference for image or video face swapping for details. Many thanks for you advise, and i will try bat files later~.
ExponentialML
commented
3 years ago These are good ideas. I've actually made a GUI for this personally, but don't know if I'll be releasing it as I didn't code it for production usage. It automatically creates masks, downscales or upscales the videos, and I'll probably implement upscaling the face as well (FYI, you get the best results on low videos with a height below 256, so I may not implement this).
@AlonDan
If you want, you can do what I did, and it's extremely easy even for beginning programmers. Simply use PyQT5, and use PyQT5's QProcess to run the scripts. It's similar to the bash/bat script approach you're using, but you instead execute the Python scripts inside your GUI script. An example would be something like:
from PyQt5 import *
process = QProcess()
def run_swap(_IMAGE, _VIDEO):
commands = [
"test_video_swapsingle.py",
"--isTrain false",
"--name",
"people",
"--Arc_path",
"/arcface_checkpoint.tar",
"--pic_a_path",
_IMAGE,
"--video_path",
_VIDEO,
"--output_path",
f"./output/{video}_out.mp4",
"--temp_path",
"./temp_results"
]
process.start('python', commands)
process.finished.connect(lambda: print("Finished swapping!"))
button_used_to_run_script = QPushButton("Run the script")
button_used_to_run_script.clicked.connect(lambda: run_swap("./my_cool_image.jpg", "./my_video.mp4"))
@ftaker887 Thanks for sharing, I appreciate it. Unfortunately, I'm not even a beginning programmer... I have no idea what are the words and terms you've mentioned (sorry) all I have is my experience with design and some logic via visual scripting but nothing more than basics, nothing related to actual lines of code (and I'm also an Animator, but that doesn't help in this case).
It took me time to understand the all process of how to install Anaconda + each requirement and such for some GitHub projects until I got things to work on my Windows OS, so I only know the basics (that's another reason for a GUI to be much more user-friendly of course for others).
Anyway, this sounds interesting maybe it will inspire others on the community to help on the GUI, as I mention I'll be happy to help with the design (not just Graphics, but the actual way it should function as a GUI looking at it from a beginner user view).
(FYI, you get the best results on low videos with a height below 256, so I may not implement this).
Interesting's! I didn't know that as I'm just a beginner but it's good to know! Probably that's why I got bad results from my tests, I still need to experiment of course as the project progress and updates.
I guess that's because of the pre-trained model we're using with the current version. I hope we'll get better higher-resolution in the future, or if it will be easy (probably not for beginners) train our own models... but I don't think that's for beginners like myself, still will be nice of course.
ExponentialML
commented
3 years ago @AlonDan No problem 🙂. I still think you should take some time to learn a bit of Python. Things become a lot more fun when you can build what you want!
Interesting's! I didn't know that as I'm just a beginner but it's good to know! Probably that's why I got bad results from my tests, I still need to experiment of course as the project progress and updates.
It's actually a very interesting solution. The model was trained on images with a resolution of 224x224. When you downscale your videos first, you don't have to stretch the pixels out, leading to a cleaner swap. Not only that, funnily the compression artifacts from low resolution video hides some of the abnormalities from the swap making it more believable. You can then use a video upscaler to scale it back up.
If you wish to use high resolution video, @woctezuma made a solution using GPEN which works very well when it wants to.
Other than that, you're right, you have to train on a higher resolution dataset. I think the dataset they used to train SimSwap's model doesn't exist anymore, so you'll have to gather millions of images yourself, or look for one online.
AlonDan
commented
3 years ago Thanks for the advice and encouragement! I sure tried to learn programming... I guess I'm not "THAT" smart because once something is not visual to me, my brain can't get it when things gets more complex (more and more line of code etc..) I may try it in the future but yeah, so many tries.. I rather focus on design at the moment but thanks! I appreciate your good vibes! :)
I actually use GPEN for some source images since it's REALLY GOOD (after trying DFDnet which is also nice but limited to 512x512 I think also it's very slow on local machine) unfortunately (for me) GPEN only runs on colab, and I'm a BIG FAN of anything that runs locally on my PC even if it's not super fast as colab (can't compare of course). but I like the way I use for now batch files to run whatever project for quick tests, GUI will be MUCH better for anything in my opinion but... again, I can't really create a GUI only design it and probably some graphics if needed.
The thing about GPEN or DFDnet is.. that their only making high-res of the faces, but not the rest of the video, so I'm not sure what the result should be, but it's interesting to try for sure.
For example: I did some tests with SimSwap combinations of GPEN for the source image, but still the output video looks blury, like it's reducing to the original size as you mentioned and than scale it up so... the GPEN didn't help my results much (I probably doing it wrong) I also tried GPEN + SimSwap + Topaz Video Enhancement AI... nothing much really.
I saw some other GitHub solutions for High-Res video, but couldn't install them locally and I'm not a big fan of google colab so I didn't try some of these.
I just shared my tests and experiments so far but I think that SimSwap will get better at least starting with basic GUI and of course, more futures and user-friendly design in general.
Training on higher resolution will be very interesting to try, probably will not be simple but I hope it will be possible to do on my local machine and not only via google colab. (I know it's much faster and all, there is no doubt hehe).
Yishaqel
commented
3 years ago @AlonDan
Can you share these .bat files? It would be very helpful for me. Thank you!
AlonDan
commented
3 years ago Sure here you go @Yishaqel please make sure you read this first:
Just to be clear: These batch files are just a "shortcut" for myself since we don't have any GUI for SimSwap
In order for these file to work, make sure to EDIT each one of them and CHANGE the directories to whatever your drive and path to SimSwap's root directory.
Starting with the path to your installed SimSwap Anaconda's Environment, in my case: Z:\anaconda3\envs\simswap
I also created extra directories so it will be easier to EDIT / CHANGE source, and watch the latest RESULTS.
At the moment, in my case the directories are like this: Z:\SimSwap_SOURCE\ Z:\SimSwap_RESULT\
If the batch files won't won't work for you it may be for many reasons:
Once you edited the bat files to YOUR paths, it should work with the LATEST VERSION consider to this date, I'm not sure if it will work on future updates while directories and model paths may change just like there is a NEW directory and model now: parsing_model for --no_mask new feature. so if it won't work in future releases, that's probably why.
What I do is I keep UPDATING my batch files when I download a new SimSwap version. if needed of course.
I hope this helps, here are the batch files (attached). SimSwap SHORTCUT by - Alon Dan.zip
Yishaqel
commented
3 years ago Thank you very much for sharing it, I will try it as soon as I can!
Yishaqel
commented
3 years ago Sure here you go @Yishaqel please make sure you read this first:
Just to be clear: These batch files are just a "shortcut" for myself since we don't have any GUI for SimSwap
In order for these file to work, make sure to EDIT each one of them and CHANGE the directories to whatever your drive and path to SimSwap's root directory.
Starting with the path to your installed SimSwap Anaconda's Environment, in my case: Z:\anaconda3\envs\simswap
I also created extra directories so it will be easier to EDIT / CHANGE source, and watch the latest RESULTS.
At the moment, in my case the directories are like this: Z:\SimSwapSOURCE Z:\SimSwapRESULT\
If the batch files won't won't work for you it may be for many reasons:
- Update all batch files for your Anaconda's SimSwap Environment path
- Update your 'SOURCE' and 'RESULT' directories.
Once you edited the bat files to YOUR paths, it should work with the LATEST VERSION consider to this date, I'm not sure if it will work on future updates while directories and model paths may change just like there is a NEW directory and model now: parsing_model for --no_mask new feature. so if it won't work in future releases, that's probably why.
What I do is I keep UPDATING my batch files when I download a new SimSwap version. if needed of course.
I hope this helps, here are the batch files (attached). SimSwap SHORTCUT by - Alon Dan.zip
One thing, is the activate.bat file something that is created automatically? Since it doesn't recognize it when I open it (I think I have the directories set correctly).
""C:\Users\yisha\anaconda3\envs\simswap\activate.bat"" is not recognized as an internal or external command,
AlonDan
commented
3 years ago Unfortunately I'm not sure if I can help here, but I'll try: all my files starts with "xyz - number" so it will be organized by name easy to locate.
I didn't mess with anything in my ...env\simswap\ you shouldn't as well. You can run my batch files from anywhere since it's running command prompt seeking for the path.
If something is not working probably, either something you didn't install / update related to latest version or models or anything originally from SimSwap or the paths are wrong.
I highly recommend to run SimSwap normally using Anaconda commands from the USAGE page first to see that it works. then tweak the batch files. none of them work if SimSwap isn't installed properly with all it's requirements and models.
Yishaqel
commented
3 years ago Apparently, I don't have the file called activate.bat in the folder (which should be there) so it doesn't recognize anything when I try to paste an image.. I'll keep trying, I'm sure I'm doing something wrong.
AlonDan
commented
3 years ago Apparently, I don't have the file called activate.bat in the folder (which should be there) so it doesn't recognize anything when I try to paste an image.. I'll keep trying, I'm sure I'm doing something wrong.
I don't have this file you mention as well, I just looked in my envs\simswap\ which I shouldn't even touch it...
Just read what I mentioned above again, you probably didn't install everything so something is wrong, then just run SimSwap with ANY of the USAGE examples to see if it works...
None of my batch files will help you if SimSwap isn't working for you, so you must follow the instructions again and try make it work before you can use my batch files, as they are just shortcuts nothing more.
Yishaqel
commented
3 years ago Forgive my ignorance, I'm not too good with English either, the activate.bat file was inside the main Anaconda3 folder and that's why it didn't work, is there any way to be able to paste any image inside the .bat to make it work? Having to be modifying to the same name every file is costly.
instant-high
commented
3 years ago I've roughly integrated SimSwap to my First Order Motion Model GUI as you can see here. When I find the time I can code a very simple separate GUI for SimSwap and share it. So that it at least copies the commandline to the clipboard or writes it to the Simswap directory as a batch file.
AlonDan
commented
3 years ago I've roughly integrated SimSwap to my First Order Motion Model GUI as you can see here. When I find the time I can code a very simple separate GUI for SimSwap and share it. So that it at least copies the commandline to the clipboard or writes it to the Simswap directory as a batch file.
Great job! it looks really nice. Please update if you'll release a version that we can use or somehow works based on the SimSwap root directory of the installed PC. that will be cool.
I like the way you show the different color coded areas of the mask, will be interesting to play with that, especially on a visual GUI :)
instant-high
commented
3 years ago The color coded mask is just for part swap/motion co-segmentation you can also find on GitHub
antongonz
commented
3 years ago The color coded mask is just for part swap/motion co-segmentation you can also find on GitHub
Did you finish your GUI yet? Can you add your email to your profile?
instant-high
commented
3 years ago I've made some changes so far. Which part of it is your interest? At the moment I'm trying to insert seamless cloning to first order and partswap as color correction..
bbecausereasonss
commented
1 year ago I've roughly integrated SimSwap to my First Order Motion Model GUI as you can see here. When I find the time I can code a very simple separate GUI for SimSwap and share it. So that it at least copies the commandline to the clipboard or writes it to the Simswap directory as a batch file.
Is this available to download somewhere? :)
instant-high
commented
1 year ago @bbecausereasonss Search my YouTube channel. You find a TKinter version of SimSwap and a VB6 for the old and the recent SimSwap repo.
First of all, THANK YOU for working on this wonderful project! ❤ I found this project few days ago, I'm pretty much fall in love with it right away.
After trying the latest 2 features added:
It inspired me as a simple user, how can I help the project grow with my own ideas & skills.
Suggestion / Ideas:
1. If it's possible to go a bit farther with the "Replace Specific Face" to multiple? For example, good reference is something like ReFace App: First, it analyze the video find ALL the faces (I guess it snapshot and crop them as best as it can consider if face is close to edge or something it gives extra space on the other sides), Then it allow the user to choose / select which one of the founded faces to swap: just a specific face or every single one of them. (total freedom of selection to the user)
2. Related to above suggestion, if possible it will be AMAZING : Can you make a super-simplistic (basic as possible for start) GUI so we run 1 command it will open the GUI, this will help the user with some preview and controls such as:
The most helpful thing will be to SEE the recognized face of the current video / image so the users can CLICK the thumbnail of FACE-1 and swap to (either drag and drop or just load from our PC) the face we want, same for FACE-2, FACE-6 or Remove their selection, so it's easy to test results in one place which is the main GUI.
In case of multiple faces, if you don't swap one of the SPECIFIC faces, it will just leave them as the original video / image and do it's swapping magic.
❤ I can help with the GUI + GUI Design: I'm aware a GUI could be different, but it's a very inviting user-friendly for sure.
If you need help with the GUI design + buttons, I'll be happy to help using my skills: I can design the basic GUI mockup for reference and the buttons (if needed), maybe you just want to draw whatever default buttons uses for minimalistic GUI which is fine and maybe faster to produce. again, I'm not a programmer I'm an animator + game/software designer so I look at it from other angle.
THE MAIN IDEA Is to keep it simple for the user, but also start re-build upcoming features based on a specific Base-GUI so the user may see immediately new buttons or options when update. If you will like my idea I'll be happy to help with the GUI graphics design: make a rough Mockup + Design the buttons.
Thanks ahead and sorry about my bad English, I hope that you can understand most of what I wrote.