Add ability to read file of urls + help page info

[ghost]

Reads list of URLs from file.lst (extension hardcoded)

[ghost]

Yes, this is just the gui. The added code only affects a .lst file uri entered in the starting url box, it has no effect on a .lst file encountered while crawling.

I don't understand what you mean by a 'separate field'.

Staff here are using the Linkchecker GUI because it has a more familiar interface style than the CLI version and does not need familiarity with starting a DOS box and so on. We use a list of URLs in a file rather than pasting a comma-separated list of URLs into the URL box because we may have lists of thousands of URLs to check.

[anarcat]

Yes, this is just the gui. The added code only affects a .lst file uri entered in the starting url box, it has no effect on a .lst file encountered while crawling.

Sure, that's fine while crawling - but what if the first URL you want to inspect actually happens to end with .list?

I don't understand what you mean by a 'separate field'.

I haven't used the GUI, so maybe it makes no sense at all. But I was assuming you would put a URL in a field, then hit enter or press a "go" button or something and the crawling would begin. Instead of reusing the URL field to turn it into something that looks for especially-named .list files, why not have a "Browse" button that would allow loading a file list through the file manager?

I'm fine with supporting such a feature - i just feel it's a hack to use an arbitrary suffix to designate URL lists. There should be a distinct and clear UI for this, otherwise only you and your team will know that this feature exists! There should be some discoverability here...

[ghost]

There are a number of separate things:

1. Using a file extension like this is of course a kludge
2. Entering a file:// URI in a box instead of a http:// URI doesn't seem that bad to me, especially as it is how the GUI already works: you can already enter a local html file or a remote URL in this same box. However, if using this kludge there should be a check not just on the extension but also that it actually IS a file URI. Given that, the likelihood of someone wanting to run a check on a local html file which happens to end in .list seems vanishingly small to me. I need to add this check.
3. It would be much better done as you suggest with a file selector. However, it's not straightforward. The UI was designed using qt-creator, and this generates the python from the design in xml form (the .ui files). qt-creator doesn't include a file selector widget, so in my understanding you'd need to manually edit the generated python to add one, creating more maintenance problems. Maybe someone who knows qt-creator could correct me, but I think this is probably why the existing GUI doesn't have a file selector.
4. I need this feature in daily use, so if necessary we could close this PR, add it as an enhancement issue and I'll just run a local fork for now
5. The GUI as is should be easily installable if you want to see it run: providing you have linkchecker already installed, setup build and install linkchecker-gui and then run it. It shouldn't be broken, though there may be minor display issues with error messages and I'm not sure the configuration editor is reliable.

[anarcat]

On 2017-02-07 09:33:07, Graham Seaman wrote:

There are a number of separate things:

2. Entering a file:// URI in a box instead of a http:// URI doesn't seem that bad to me, especially as it is how the GUI already works: you can already enter a local html file or a remote URL in this same box. However, if using this kludge there should be a check not just on the extension but also that it actually IS a file URI. Given that, the likelihood of someone wanting to run a check on a local html file which happens to end in .list seems vanishingly small to me. I need to add this check.

Understood. It is quite unlikely, but yes, let's make that explicit that it works only with file:// URLs.

3. It would be much better done as you suggest with a file selector. However, it's not straightforward. The UI was designed using qt-creator, and this generates the python from the design in xml form (the .ui files). qt-creator doesn't include a file selector widget, so in my understanding you'd need to manually edit the generated python to add one, creating more maintenance problems. Maybe someone who knows qt-creator could correct me, but I think this is probably why the existing GUI doesn't have a file selector.

I'm surprised that qt-creator doesn't do that, but oh well. :)

4. I need this feature in daily use, so if necessary we could close this PR, add it as an enhancement issue and I'll just run a local fork for now

I'd rather you not fork! ;) Let's figure this out, I'm sure we can do it.

If it applies only to file:// URLs, i remove my objection. I still think there should be a note somewhere to tell people about the feature so it's not lost in the mists of time. This can be a bug report to remind us to do this later, however.

Thanks for your patience!

[ghost]

It is quite unlikely, but yes, let's make that explicit that it works only with file:// URLs.

Not quite so simple: the GUI allows partial URIs without any scheme and tries to guess the scheme intended. But I've added a check that either the user entered a file:// URI or this is a really-existing file

I still think there should be a note somewhere to tell people about the feature so it's not lost in the mists of time. This can be a bug report to remind us to do this later, however.

I'd already added a note in the help file

[anarcat]

okay then, sorry for the nitpicking. :)

[ghost]

Not nit-picking, and you wouldn't be doing your job if you weren't ;-)

From: anarcat [mailto:notifications@github.com] Sent: 08 February 2017 14:34 To: linkcheck/linkchecker-gui Cc: Seaman, Graham; Author Subject: Re: [linkcheck/linkchecker-gui] Add ability to read file of urls + help page info (#2)

okay then, sorry for the nitpicking. :)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/linkcheck/linkchecker-gui/pull/2#issuecomment-278344450, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AEW9hjt8cyCkATO3IA-Sj2746jdOoWafks5radJJgaJpZM4L2gm8.

This email was received from the INTERNET and scanned by the Government Secure Intranet anti-virus service supplied by Cable&Wireless Worldwide in partnership with MessageLabs. (CCTM Certificate Number 2009/09/0052.) In case of problems, please call your organisation’s IT Helpdesk. Communications via the GSi may be automatically logged, monitored and/or recorded for legal purposes. Please don't print this e-mail unless you really need to.

---

National Archives Disclaimer

This email and any files transmitted with it are intended solely for the use of the individual(s) to whom they are addressed. If you are not the intended recipient and have received this email in error, please notify the sender and delete the email. Opinions, conclusions and other information in this message and attachments that do not relate to the official business of The National Archives are neither given nor endorsed by it.

---