SaptarshiSarkar12 / Drifty

Drifty is an Open-Source Interactive File Downloader System built with Java
https://saptarshisarkar12.github.io/Drifty/
Apache License 2.0
171 stars 121 forks source link

[BUG] Broken Image Link #225

Closed EasyG0ing1 closed 1 year ago

EasyG0ing1 commented 1 year ago

Describe the bug

This link has a broken image: https://saptarshisarkar12.github.io/Drifty/about

See screen shot

Steps To Reproduce

Click on link ... observe broken image link

Expected Behavior

Not a broken image

Operating System

MacOS

Browser

Google Chrome

Screenshots

Screenshot 2023-07-14 at 8 19 10 AM

Additional information

No response

github-actions[bot] commented 1 year ago

Hello 👋! Thank you very much for raising an issue 🙌! The maintainers will get back to you soon for discussion over the issue! 🚀

Meanwhile you can also discuss about the project in our Discord Server 😀

SaptarshiSarkar12 commented 1 year ago

Hey Michael (@EasyG0ing1), the image becomes like this due to a problem in the deployment/maintenance of the github-readme-stats project and not on our's end. So, we can not resolve it. If you have any other idea like to remove the image if unavailable, then you might create a PR for that. Thank you for raising the issue :smile: !

EasyG0ing1 commented 1 year ago

@SaptarshiSarkar12 - OK, that makes sense. Incidentally, I spent all weekend writing a JavaFX GUI for Drifty, including a form dedicated to batch-processing downloads. I also figured out how to use yt-dlp to extract the filename from links which has been working well. Though I can't for the life of me access the file download progress given by yt-dlp ... it must be happening on a different thread than what the Process object has access to because when I try the standard methods of reading the output, it just blows by that code before the download even starts. Would be nice to have progress bars, but that looks elusive for now.

EasyG0ing1 commented 1 year ago

@SaptarshiSarkar12

Here are some screen shots of the FX update ... I'll post a PR soon.

Screenshot 2023-07-18 at 12 48 45 PM Screenshot 2023-07-18 at 12 51 08 PM
SaptarshiSarkar12 commented 1 year ago

@SaptarshiSarkar12 - OK, that makes sense. Incidentally, I spent all weekend writing a JavaFX GUI for Drifty, including a form dedicated to batch-processing downloads. I also figured out how to use yt-dlp to extract the filename from links which has been working well. Though I can't for the life of me access the file download progress given by yt-dlp ... it must be happening on a different thread than what the Process object has access to because when I try the standard methods of reading the output, it just blows by that code before the download even starts. Would be nice to have progress bars, but that looks elusive for now.

That's great @EasyG0ing1! Please remember to create a issue (feature) before opening the PR.

SaptarshiSarkar12 commented 1 year ago

@SaptarshiSarkar12

Here are some screen shots of the FX update ... I'll post a PR soon.

Screenshot 2023-07-18 at 12 48 45 PM Screenshot 2023-07-18 at 12 51 08 PM

@EasyG0ing1 The new UI of the Drifty GUI looks pretty awesome :star_struck: , thank you for your time and consideration!

Please open an issue before opening a PR for the same.

SaptarshiSarkar12 commented 1 year ago

Closing this issue as the bug is not on our end.

EasyG0ing1 commented 1 year ago

@SaptarshiSarkar12 - OK, I'll open an issue first.

SaptarshiSarkar12 commented 1 year ago

@EasyG0ing1 When are you planning to open the issue and the corresponding pull request?

EasyG0ing1 commented 1 year ago

@SaptarshiSarkar12 - I have been trying to get the progressbar working using different libraries to see if I can get access to that download progress but have not had much success. I'll post a PR today and work on that problem at a later time.

SaptarshiSarkar12 commented 1 year ago

@SaptarshiSarkar12 - I have been trying to get the progressbar working using different libraries to see if I can get access to that download progress but have not had much success. I'll post a PR today and work on that problem at a later time.

Yeah @EasyG0ing1 , we can have that issue resolved later on. It's great :star_struck: that you are opening a PR today!

EasyG0ing1 commented 1 year ago

@SaptarshiSarkar12 So I kind of had this working with a few issues but as I started to try and resolve those I started having other issues with this yt-dlp program not running properly in a Java Process. So I decided to download the original code from the repository and package it and just get a feel for the program in general but even that doesn't run properly on MacOS without some tweaking of the code. I've gone beyond my frustration levels with this code so I'm gonna set this project aside for now.

EasyG0ing1 commented 1 year ago

@SaptarshiSarkar12 I did want to ask about this method:

    public void copyToTemp() throws IOException{
        String yt_dlpFileName = FileDownloader.getYt_dlpProgramName();
        try {
            Files.copy(Path.of("./src/main/resources/" + yt_dlpFileName), Path.of(getTempDir() + yt_dlpFileName));
        } catch (FileAlreadyExistsException e){
            messageBroker.sendMessage("Skipping copying yt-dlp to " + getTempDir() + " folder as it is already present!", LOGGER_INFO, "download");
        }
    }

Has this been working at all? Because that's not how you copy files out of a Resources folder ... at least I've never seen it done that way and it won't work from the packaged code ... in an IDE it works, but not packaged code.

When I changed it to this and re-packaged, it worked:

    public void copyToTemp() throws IOException{
        String path = getTempDir();
        String yt_dlpFileName = FileDownloader.getYt_dlpProgramName();
        Path destination = Paths.get(path, yt_dlpFileName);
        if (!destination.toFile().exists()) {
            try (InputStream inputStream = copyYt_dlp.class.getClassLoader().getResourceAsStream(yt_dlpFileName);
                 OutputStream outputStream = Files.newOutputStream(destination)) {
                if (inputStream != null) {
                    byte[] buffer = new byte[4096];
                    int bytesRead;
                    while((bytesRead = inputStream.read(buffer)) != -1) {
                        outputStream.write(buffer, 0, bytesRead);
                    }
                }
                else {
                    System.out.println("Resource not found: " + yt_dlpFileName);
                }
            }
        }
        else {
            messageBroker.sendMessage("Skipping copying yt-dlp to " + path + " folder as it is already present!", LOGGER_INFO, "download");
        }
    }

Also in the youtube download method, for some reason, the command formatted with these options:

./yt-dlp_macos --quiet --progress -P /Users/michael/Downloads/Video/ https://www.youtube.com/watch?v=ujysdH5FDHomY -o 'Video.mp4' -f [height<=1120][width<=1792]

The -f option is what throws me ... when I try to run that in a bash shell, I get this error:

-bash: =1120][width: No such file or directory

SaptarshiSarkar12 commented 1 year ago

Yeah @EasyG0ing1, the copyToTemp method is having issues and it has been resolved in another branch. You can try the below code if this works or not.

public void copyToTemp(InputStream inputStream) throws IOException{
        String yt_dlpFileName = FileDownloader.getYt_dlpProgramName();
        try (InputStream stream = inputStream) {
            Path yt_dlpTempFilePath = Paths.get(getTempDir() + yt_dlpFileName);
            // convert stream to file
            Files.copy(stream, yt_dlpTempFilePath);
            if (!Files.isExecutable(yt_dlpTempFilePath)){
                ProcessBuilder makeExecutable = new ProcessBuilder("chmod", "+x", yt_dlpTempFilePath.toString());
                makeExecutable.inheritIO();
                Process yt_dlp = makeExecutable.start();
                yt_dlp.waitFor();
            }
        } catch (FileAlreadyExistsException e){
            messageBroker.sendMessage("Skipping copying yt-dlp to " + getTempDir() + " folder as it is already present!", LOGGER_WARN, "only log");
        } catch (InterruptedException e) {
            messageBroker.sendMessage("Failed to make executable the yt-dlp file in temporary directory. " + e.getMessage(), LOGGER_ERROR, "only log");
        }
    }

And use the below code for downloadFromYouTube method in FileDownloader class.

public static void downloadFromYouTube(String dirOfYt_dlp) throws InterruptedException, IOException {
        String outputFileName;
        outputFileName = Objects.requireNonNullElse(fileName, "%(title)s.%(ext)s");
        String fileDownloadMessagePart;
        if (outputFileName.equals("%(title)s.%(ext)s")){
            fileDownloadMessagePart = "the YouTube Video";
        } else {
            fileDownloadMessagePart = outputFileName;
        }
        ProcessBuilder processBuilder;
        messageBroker.sendMessage("Trying to download " + fileDownloadMessagePart + " ...", LOGGER_INFO, "download");
        if ((dir.isEmpty()) || (dir.equalsIgnoreCase("."))){
            processBuilder = new ProcessBuilder(dirOfYt_dlp + yt_dlpProgramName, "--quiet", "--progress", link, "-o", outputFileName);
        } else {
            processBuilder = new ProcessBuilder(dirOfYt_dlp + yt_dlpProgramName, "--quiet", "--progress", "-P", dir, link, "-o", outputFileName);
        }
        processBuilder.inheritIO();
        messageBroker.sendMessage(DOWNLOADING + fileDownloadMessagePart + " ...", LOGGER_INFO, "download");
        Process yt_dlp = processBuilder.start();
        yt_dlp.waitFor();
        int exitValueOfYt_Dlp = yt_dlp.exitValue();
        if (exitValueOfYt_Dlp == 0) {
            messageBroker.sendMessage(SUCCESSFULLY_DOWNLOADED + fileDownloadMessagePart + " !", LOGGER_INFO, "download");
        } else if (exitValueOfYt_Dlp == 1) {
            messageBroker.sendMessage(FAILED_TO_DOWNLOAD + fileDownloadMessagePart + " !", LOGGER_ERROR, "download");
        }
    }

I have removed the height and width as they are no longer required. You also need to change the code for the run() function in FileDownloader class to the below code :

    @Override
    public void run() {
        link = link.replace('\\', '/');
        if (!(link.startsWith("http://") || link.startsWith("https://"))) {
            link = "http://" + link;
        }
        if (link.startsWith("https://github.com/") || (link.startsWith("http://github.com/"))) {
            if (!(link.endsWith("?raw=true"))) {
                link = link + "?raw=true";
            }
        }
        if (dir.isEmpty()) {
            dir = System.getProperty("user.dir");
        }
        if (!(dir.endsWith("\\"))) {
            dir = dir + System.getProperty("file.separator");
        }
        try {
            boolean isInstagramLink = Drifty_CLI.getIsInstagramLink();
            boolean isInstagramImage = false;
            if (isInstagramLink) {
                isInstagramImage = Drifty_CLI.getIsInstagramImage();
            }
            // If link is of an YouTube or Instagram video, then the following block of code will execute.
            if (isYoutubeLink(link) || (!isInstagramImage && isInstagramLink)) {
                try {
                    String directoryOfYt_dlp = "./src/main/resources/";
                    messageBroker.sendMessage("Checking for component (yt-dlp) update ...", LOGGER_INFO, "download");
                    ProcessBuilder yt_dlp_update = new ProcessBuilder(directoryOfYt_dlp + yt_dlpProgramName, "-U");
                    yt_dlp_update.inheritIO();
                    Process yt_dlp = yt_dlp_update.start();
                    yt_dlp.waitFor();
                    if (isYoutubeLink(link)) {
                        downloadFromYouTube(directoryOfYt_dlp);
                    } else if (!isInstagramImage) {
                        downloadFromInstagram(directoryOfYt_dlp);
                    }
                } catch (IOException e) {
                    try {
                        messageBroker.sendMessage(GETTING_READY_TO_DOWNLOAD_FILE, LOGGER_INFO, "download");
                        InputStream yt_dlpProgramStream = ClassLoader.getSystemResourceAsStream(yt_dlpProgramName);
                        copyYt_dlp cy = new copyYt_dlp();
                        cy.copyToTemp(yt_dlpProgramStream);
                        try {
                            String tempDir = copyYt_dlp.getTempDir();
                            messageBroker.sendMessage("Checking for component (yt-dlp) update ...", LOGGER_INFO, "download");
                            ProcessBuilder yt_dlp_update = new ProcessBuilder(tempDir + yt_dlpProgramName, "-U");
                            yt_dlp_update.inheritIO();
                            Process yt_dlp = yt_dlp_update.start();
                            yt_dlp.waitFor();
                            if (isYoutubeLink(link)) {
                                downloadFromYouTube(tempDir);
                            } else if (!isInstagramImage) {
                                downloadFromInstagram(tempDir);
                            }
                        } catch (InterruptedException ie) {
                            messageBroker.sendMessage(USER_INTERRUPTION, LOGGER_ERROR, "download");
                        } catch (Exception e1) {
                            messageBroker.sendMessage(FAILED_TO_DOWNLOAD_YOUTUBE_VIDEO, LOGGER_ERROR, "download");
                            String message = e1.getMessage();
                            String[] messageArray = message.split(",");
                            if (messageArray.length >= 1 && messageArray[0].toLowerCase().trim().replaceAll(" ", "").contains("cannotrunprogram")) { // If yt-dlp program is not marked as executable
                                messageBroker.sendMessage(DRIFTY_COMPONENT_NOT_EXECUTABLE, LOGGER_ERROR, "download");
                            } else if (messageArray.length >= 1 && messageArray[1].toLowerCase().trim().replaceAll(" ", "").equals("permissiondenied")) { // If a private YouTube video is asked to be downloaded
                                messageBroker.sendMessage(PERMISSION_DENIED_YOUTUBE_VIDEO, LOGGER_ERROR, "download");
                            } else if (messageArray[0].toLowerCase().trim().replaceAll(" ", "").equals("videounavailable")) { // If YouTube Video is unavailable
                                messageBroker.sendMessage(YOUTUBE_VIDEO_UNAVAILABLE, LOGGER_ERROR, "download");
                            } else {
                                messageBroker.sendMessage(e.getMessage(), LOGGER_ERROR, "download");
                            }
                        }
                    } catch (IOException io) {
                        messageBroker.sendMessage(FAILED_TO_INITIALISE_YOUTUBE_VIDEO_DOWNLOADER, LOGGER_ERROR, "download");
                    }
                } catch (InterruptedException e) {
                    messageBroker.sendMessage(USER_INTERRUPTION, LOGGER_ERROR, "download");
                }
            } else {
                if (isInstagramImage){
                    if (link.endsWith("/") || link.endsWith("\\")){
                        link += "media";
                    } else {
                        link += "/media";
                    }
                }
                url = new URI(link).toURL();
                URLConnection openConnection = url.openConnection();
                openConnection.connect();
                totalSize = openConnection.getHeaderFieldLong("Content-Length", -1);

                String acceptRange = openConnection.getHeaderField("Accept-Ranges");
                FileDownloader.supportsMultithreading = (totalSize > threadingThreshold) && (acceptRange != null) && (acceptRange.equalsIgnoreCase("bytes"));

                if (fileName.isEmpty()) {
                    String[] webPaths = url.getFile().trim().split("/");
                    fileName = webPaths[webPaths.length - 1];
                }
                messageBroker.sendMessage(TRYING_TO_DOWNLOAD_FILE, LOGGER_INFO, "download");
                downloadFile();
            }
        } catch (MalformedURLException | URISyntaxException e) {
            messageBroker.sendMessage(INVALID_LINK, LOGGER_ERROR, "link");
        } catch (IOException e) {
            messageBroker.sendMessage(FAILED_TO_CONNECT_TO_URL + url + " !", LOGGER_ERROR, "download");
        }
    }