search

GoogleChromeLabs / ProjectVisBug

FireBug for designers › Edit any webpage, in any state https://a.nerdy.dev/gimme-visbug
https://visbug.web.app
Apache License 2.0
5.42k stars 285 forks source link

Download all images on the page #480

Open argyleink opened 3 years ago

argyleink commented 3 years ago

/download-images command in the search bar runs the following plugin script

const run = async () => {
  const dirHandle = await window.showDirectoryPicker();
  const imgs = document.querySelectorAll("img");
  let i = 0;

  imgs.forEach(async (img) => {
    const url = img.src;
    const name = `img-${i}.png`;
    i++;

    try {
      console.log(`Fetching ${url}`);
      const response = await fetch(url);

      console.log(`Saving to ${name}`);

      const file = await dirHandle.getFileHandle(name, { create: true });
      const writable = await file.createWritable();
      await response.body.pipeTo(writable);
    } catch (err) {
      console.log(err);
    }
  });
};

run();

https://paul.kinlan.me/bookmarklet-to-download-all-images-on-a-page-with-the-file-system-api/

argyleink commented 3 years ago

@aashu0148 here's another one that needs finished if you're lookin for another plugin to hack on!? the #481 has the work started but needs a few tweaks before it's ready. 🙂

aashu0148 commented 3 years ago

@aashu0148 here's another one that needs finished if you're lookin for another plugin to hack on!? the #481 has the work started but needs a few tweaks before it's ready. 🙂

I'll check for sure. But now i have a function in my family 🥳🥳 I'll check later

aashu0148 commented 3 years ago

@argyleink I'll be happy to work on this issue but can you please tell me where the javascript file is located from which i can access the value entered in the search box.

argyleink commented 3 years ago

sure! PR #481 branch download-images-plugin file with function invoked by slash command https://github.com/GoogleChromeLabs/ProjectVisBug/pull/481/files#diff-594f34f26bc63ff8ca673fe36c1acded478fe92a04306aa9783f8ab50fe4d3e8

aashu0148 commented 3 years ago

Hey @argyleink I'm really sorry for this delay but first i was busy in my family functions and then i got my exams but i thought i can easily make a good percentage in my exams but after giving my first exam i realized that this will not going until i dedicatedly study for it... so I decided to study for my exams and forgot about informing you..... yesterday was my last exam. I'll be working on this issue from now itself.

Again So sorry for accepting this issue and not active for such a long period :(

aashu0148 commented 3 years ago

hey @argyleink I run it on my machine and it is downloading all the images. What's for me in it ?

argyleink commented 3 years ago

yes, sorry, I see I never listed the missing aspects of the feature!

Currently, the user picks a folder to save the images to, and all the images download, but there's many that are broken and the original names of the images is lost. A file named logo.svg should save as logo.svg, where right now it saves as img-2.png. Both the name and src type are not brought along in the download.

Thanks for the help!

argyleink commented 3 years ago

also, would be cool if it downloaded all images that were referenced via CSS too! background images, etc. want to investigate level of effort to adding that too?

aashu0148 commented 3 years ago

okay @argyleink we will move to CSS images also later on but now let me work on javascript to complete the above mentioned tasks :)

aashu0148 commented 3 years ago
  • / Screenshot (40)

hey @argyleink first task is completed i think. If not please let me know;

aashu0148 commented 3 years ago

hey @argyleink to check for showDirectoryPicker can we put it into try and add a alert in catch. Will this work ?Because i dont know how to test this as my machine support showDirectoryPicker

argyleink commented 3 years ago

nice work on the names and file extensions 👍🏻

you could try catch for feature testing yeah. i just tested this and it works too:

if (window.showDirectoryPicker === undefined)
  console.log('no directory support')
argyleink commented 3 years ago

here's some JS that grabs the external urls from stylesheets on the page:

let css_urls = [...document.styleSheets]
  .filter(sheet => {
    try { return sheet.cssRules }
    catch {}
  })
  .flatMap(sheet => Array.from(sheet.cssRules))
  .filter(rule => rule.style)
  .filter(rule => rule.style.backgroundImage !== '')
  .filter(rule => rule.style.backgroundImage !== 'initial')
  .filter(rule => rule.style.backgroundImage.includes("url"))
  .reduce((urls, {style}) => {
    urls.push(style.backgroundImage);
    return urls;
  }, []);

console.log(css_urls);

output example:

["url("../media/foo.jpg")", "url("../media/bar.svg")", "url("../media/baz.png")"]

seems doable with this concept proof yeah!?

aashu0148 commented 3 years ago

here's some JS that grabs the external urls from stylesheets on the page:

let css_urls = [...document.styleSheets]
  .filter(sheet => {
    try { return sheet.cssRules }
    catch {}
  })
  .flatMap(sheet => Array.from(sheet.cssRules))
  .filter(rule => rule.style)
  .filter(rule => rule.style.backgroundImage !== '')
  .filter(rule => rule.style.backgroundImage !== 'initial')
  .filter(rule => rule.style.backgroundImage.includes("url"))
  .reduce((urls, {style}) => {
    urls.push(style.backgroundImage);
    return urls;
  }, []);

console.log(css_urls);

output example:

["url("../media/foo.jpg")", "url("../media/bar.svg")", "url("../media/baz.png")"]

seems doable with this concept proof yeah!?

ya it seems you already done most of my work. Thanks :) I'll update you soon

aashu0148 commented 3 years ago

hey @argyleink I'm stuck with an awkward issue :/ can't say why is this showing. The issue is that i have changed the JS file and in the console it is still showing the console logs and not implementing my changes. I have closed vs code 3-4 times and re run the localhost dozen of times but still it is not showing the console according to the current code. I'm attaching the screenshot of that please help me with that. check there in the console it says at line no 48,53,54 but in my code there is no console logs on that lines.. there are some before but i had deleted them but it is not showing the updated result.

can you please take a look at it. I'll also continue to find the problem and update you if it is solved.

Screenshot (41)

aashu0148 commented 3 years ago

hey @argyleink I'm stuck with an awkward issue :/ can't say why is this showing. The issue is that i have changed the JS file and in the console it is still showing the console logs and not implementing my changes. I have closed vs code 3-4 times and re run the localhost dozen of times but still it is not showing the console according to the current code. I'm attaching the screenshot of that please help me with that. check there in the console it says at line no 48,53,54 but in my code there is no console logs on that lines.. there are some before but i had deleted them but it is not showing the updated result.

can you please take a look at it. I'll also continue to find the problem and update you if it is solved.

Screenshot (41)

Also in the source tab of dev tools the code is not what i have in my vs code.

Screenshot (42)

argyleink commented 3 years ago

hm! I can try to help 🙂

aashu0148 commented 3 years ago

hm! I can try to help 🙂

  • do you have devtools local overrides turned on? perhaps you did some work inside of the sources tab on that file and hit save / ctrl + s? if so, then devtools will reload a modified file that it maintains, instead of the source file from your server. to me, this is the most likely given the screenshots and clues you've provided so far
  • when you run npm start in the visbug project, are there js errors? the site may continue to work and serve an older, previously built file and isnt producing a new one due to a syntax issue somewhere. tldr; it may be that a new js file isnt getting bundled, and therefore not showing your new lines of code

Thanks @argyleink there are some errors while running npm start now it is working.

but can you tell me what to put in the name when we download css images because we fetch them from a url and hence can't find the name.

so should i name them like css-1,css-2,... ?

aashu0148 commented 3 years ago

@argyleink ??🤔

argyleink commented 3 years ago

Can you give me examples of what you mean? URLs with no names?

aashu0148 commented 3 years ago

Can you give me examples of what you mean? URLs with no names?

yaa sure like if a image was fetched from this url https://images.unsplash.com/photo-1608806947629-da2ae50be954?ixid=MXwxMjA3fDB8MHxlZGl0b3JpYWwtZmVlZHwyfHx8ZW58MHx8fA%3D%3D&ixlib=rb-1.2.1&auto=format&fit=crop&w=400&q=60

then we don't have a name for this image then how can i name such images... I have currently named them as css-1 , css-2 , css-3 ....

aashu0148 commented 3 years ago

@argyleink 🤔🤔🤔

aashu0148 commented 3 years ago

hey @argyleink where are you ??

argyleink commented 3 years ago

Vacation 😁

aashu0148 commented 3 years ago

Vacation 😁

oh :) when will it end ??

argyleink commented 3 years ago

early january! but i can give some feedback real quick 👍🏻

some url rules?

that way really long ones can be cropped, and if they're still too long, they can be maxed out then saved. thoughts?

aashu0148 commented 3 years ago

early january! but i can give some feedback real quick 👍🏻

some url rules?

  • remove all params after the ? on urls, as they're parameters and not filename values
  • limit filenames to 40 characters

that way really long ones can be cropped, and if they're still too long, they can be maxed out then saved. thoughts?

ya its great i'll implement it and update you

aashu0148 commented 3 years ago

Its done @argyleink Please review it :)

Screenshot (43) Screenshot (44)

argyleink commented 3 years ago

Looks ready to share! Submit a pr?

aashu0148 commented 3 years ago

Looks ready to share! Submit a pr?

already made a pr. please review it

aashu0148 commented 3 years ago

hey @argyleink can you please tell me how can i use this project locally with any website? like the chrome extension has only features which are developed and as we are developing it then how can i see this project working on other websites ?

argyleink commented 3 years ago

Here's a wiki page with some instruction: https://github.com/GoogleChromeLabs/ProjectVisBug/wiki/Local-Extension-Dev-Testing

your github repo extension/ folder will allow you to load the plugin from your computer, it'll be called DevBug in Chrome. Visit any webpage, then launch DevBug and you'll be working with your local branch and plugin against any webpage. Depending on what code changes, you may need to rebundle the extension.

Let me know if you need more help 🙂

aashu0148 commented 3 years ago

ya @argyleink I need some more help!

First of all i really like this project idea and i want to contribute more to it until i have time from my studies and all stuff. and here at github comment section is too much slow like i usually work at night i when i see your message i reply to it at night and then i'll get your reply next night and this process goes on like it takes 2-3 days to even understand this issue. Frankly speaking i'm new to github and don't know if github have another method of communicating rather then commenting on the issue. It would be great if you tell me where can we communicate faster then what we are currently doing.

Also i saw the link and in the download section of it i saw this error page. Screenshot (52)

argyleink commented 3 years ago

join me on Slack? https://join.slack.com/t/visbugworkspace/shared_invite/zt-l4xg8yci-QpQAv7GXl6vsm5F1uJ60qg