Thanks and (lots of) feedback

[t214c]

Thanks a lot for HTTP Downloader :) . It is fast, straightforward, handles what more established downloaders can't, and seems very promising.

Some feedback:

Adding URLs:

• Editable download location field (i.e. allows paste).
• Show the better location selection window (where you can access location history from the address bar) like here and here.
• Option to show drop window from tray menu.
• Option to automatically paste clipboard contents when clicking add URLs.
• Bug: crashed when I dragged a txt file with links to it.

• Option to automatically encode non-ASCII characters in imported/pasted URLs.

  Saving:

• If there is a "File IO Error", you can rename the file but the restart option (and related options) is grayed out.
• Rename patterns see DownThemAll (which was better than most standalone download managers). This will also help witch duplicate filenames.
• Append an extension to the incomplete file part during download, or download to a temp folder then move to location.
• If two downloads have the same filename (File.zip, file.zip), add option to rename automatically instead of I/O Error.
• If a link has a URL-encoded non-ASCII characters (like here or here or here), (in these examples, each file is saved with a different name. See image below), I believe the file should be saved in Unicode.

List:

• Confirmation policy for different task (e.g. for me, warning when deleting an unfinished entry, but not needed when clearing completed downloads).
• Bug: Select entry in download list. Move to a different window. Click HTTP Downloader window title. Press down or up keys. Won't scroll although file is highlighted.
• Del key should be to remove entry not the file I just downloaded. A shortcut should be for the more common action, I think. You could provide hotkey options.
• Bug: When you click "delete" on a running download, the program continues to download.

• Some options defaults should be revised, I think. For example, prevent sleep while downloading.

  Behavior:

• Better dealing with unexpected termination (force quit, power failures, etc). It returns to the last saved state and you have to re-add interrupted files. I think it should save every couple of seconds or something to that effect.
• Don't resume automatically on startup. Or offer an option for this.
• Download verification: IDM does something like this (which I think you should consider): On resuming, redownload the last few bytes/kilobytes of the finished part(s) to make sure they are the same thus decreasing the likelihood of file corruption. Maybe also check modification date and time of the server file.

• Set concurrent downloads/connections/parts/etc per domain.

  Settings:

• Why save file settings as a binary? This hinders export, modification, etc.

  Export:

• Option to export just the URL Links (this can be done currently by editing the exported CSV file but I think it should be easier.)

  Extension:

• Get extension on addons.mozilla.org

  General:

• Better (unique) name for the future :) ?

Thanks again!

P.S. Please tell me if you want separate issues :)

[erickutcher]

There's a lot to mention here.

Adding URLs:

The download directory text box is read only to prevent the user from inputting some weird directory that'll cause the file save to fail. It's meant primarily for informational purposes. As far as a better folder browser goes, the two images that you linked are using the IFileDialog interface which is available for Windows Vista and above. Since I'd like to support XP I wouldn't be able to implement it without using a fallback solution and that would increase the size of the program unnecessarily.

When you say show drop window from tray menu, do you mean the ability to enable/disable (show/hide) it from a menu entry? If you don't want it to always be on top of other windows, you can middle click it to toggle on/off its always on top property.

Clipboard monitoring is not something I'm keen on doing. I'll give it some thought if I can find an efficient and less impactful way to process the clipboard's content.

Could you send the txt file that caused the crash when you dragged it into the program? Where also did you drag it into?

The URL(s) text box should support Unicode characters. If you copy this (not as a link, but as text) https://en.wiktionary.org/wiki/￦ then it should end up saving a file with that Unicode character. My program is decoding the percent encoded portion of the URL, but it's not converting any UTF-8 characters to their wide character equivalent. I'll have to look into a decent solution.

Saving:

I've changed downloads that have the File IO Error status to be able to start and restart.

When there's duplicate files, the program should prompt you to either overwrite, rename, or skip the file. If you select rename it'll append a number to the end of the filename like how Windows behaves. There's also a checkbox in the prompt to allow the program to automatically rename when there's a conflict.

I'll look into adding a file extension for incomplete downloads.

List:

I like having a warning message when removing and or deleting items from the list. It's just added safety for accidental clicks.

I fixed the list not gaining focus when the main windows is switched back.

I used Del(ete) for file deletions so that it's consistent with Windows Explorer. All of the shortcut keys are listed in the main menu.

The download still going after delete is definitely problematic. I'll have it fixed for the next release.

When you say option defaults revised, do you mean reworded? I tried to make them unambiguous and consistent with Windows wording.

Behavior:

The program saves when shut down properly or after all active downloads have completed. If I were to save every couple of seconds there would be a lot of bytes written to the disk. The entire list gets saved every time it saves. A compromise would be to auto save every 5 minutes. I'll think about it.

Paused downloads will stop if the program is shut down and restarted. Only active downloads are continued when shut down and restarted. I kind of like that behavior. I think it would be easy enough to just pause or stop them after the program's been started up again, or before it's shut down.

Checking for a changed modified time would probably be a better solution. I prefer that to redownloading and comparing bytes.

Per-domain settings would take a lot of work. I'm not sure if the payoff is worth it.

Settings:

Binary settings are the quickest to save and load. I could use XML, but then I'd have to write a ton of parsing code for that since there's no API for XP and newer that'll let me work with it. I also don't want to rely on external resources that aren't native to Windows systems. The only exclusion I made was for the zlib compression library.

Export:

You can select all of the items in the main window, right click on one of them, and select Copy URL(s). Then just paste it in a file to save.

Extension:

I don't like extension stores. There's too many hoops to jump through and so many awful comments that I just don't want to deal with. It's easier for me to put everything up on Github.

General:

HTTP Downloader is a boring name, but it gets to the point. Maybe I'll have something for the future. We'll see.

I appreciate all the input. I'm not sure when I'll have a new release for these points. I have other priorities at the moment, but I'll try my best.

[smaragdus]

@erickutcher

The program saves when shut down properly or after all active downloads have completed.

For me this is the perfect behaviour.

If I were to save every couple of seconds there would be a lot of bytes written to the disk. The entire list gets saved every time it saves. A compromise would be to auto save every 5 minutes. I'll think about it.

I do not like the idea of auto-saving every 5 minutes. If you choose to implement this behaviour please make it optional (with check-box).

HTTP Downloader is a boring name, but it gets to the point. Maybe I'll have something for the future. We'll see.

The name may be boring but it shows perfectly what the program does. HTTP Downloader is already an established name- the program is listed in several sites (MajorGeeks, SnapFiles, The Portable Freeware) and changing the name would only cause confusion. Please keep the current "boring" name.

[t214c]

Thanks a lot for the response. I appreciate it.

I also appreciate that you are giving us a valuable tool without asking for something in return. I don't want to seem presumptuous or demanding.

To give you some context of one of my use cases: I was doing a personal project with the REST API of MediaWiki which necessitated the following to be able to get all files correctly:

• Handling any printable Unicode character (or even zero width joiner).
• Handling files with the same name (case insensitive) like here and here.
• Handling hundreds of thousands of links at a time (which HTTP downloader did well unlike other download managers I have tried. Thanks!

---

The download directory text box is read only to prevent the user from inputting some weird directory that'll cause the file save to fail. It's meant primarily for informational purposes. As far as a better folder browser goes, the two images that you linked are using the IFileDialog interface which is available for Windows Vista and above. Since I'd like to support XP I wouldn't be able to implement it without using a fallback solution and that would increase the size of the program unnecessarily.

I understand.

When you say show drop window from tray menu, do you mean the ability to enable/disable (show/hide) it from a menu entry? If you don't want it to always be on top of other windows, you can middle click it to toggle on/off its always on top property.

I meant enable/disable. If I chose not to show it on top (because always on top can be annoying), I'd have to fish it from underneath all the open windows to be able to use it (or CTRL+D > show any window > minimize it to show the drop window). I think a tray menu toggle would be easier.

Clipboard monitoring is not something I'm keen on doing. I'll give it some thought if I can find an efficient and less impactful way to process the clipboard's content.

Minor issue anyway. Thanks.

Could you send the txt file that caused the crash when you dragged it into the program? Where also did you drag it into?

Any text file. I can't upload text files, it seems. Try a one-liner (encoding doesn't matter): https://github.com/erickutcher/httpdownloader/releases/download/v1.0.1.8/HTTP_Downloader_64.zip

The URL(s) text box should support Unicode characters. If you copy this (not as a link, but as text) https://en.wiktionary.org/wiki/￦ then it should end up saving a file with that Unicode character. My program is decoding the percent encoded portion of the URL, but it's not converting any UTF-8 characters to their wide character equivalent. I'll have to look into a decent solution.

OK, Thanks.

When there's duplicate files, the program should prompt you to either overwrite, rename, or skip the file. If you select rename it'll append a number to the end of the filename like how Windows behaves. There's also a checkbox in the prompt to allow the program to automatically rename when there's a conflict.

Some times the downloads are concurrent (added and downloading at the same time). See the examples at the top. The REST API allows up to 200 connections/second!

I like having a warning message when removing and or deleting items from the list. It's just added safety for accidental clicks.

Clicking OK/Confirm becomes an automatic habit by human nature which can lead to problems if you clicked the wrong list item (e.g. remove and delete instead of remove completed). I understand that you may use it differently so an option would be nice though.

I used Del(ete) for file deletions so that it's consistent with Windows Explorer. All of the shortcut keys are listed in the main menu.

True. But how often would you want to delete the files themselves vs. clearing the list of completed files? I think this is another argument in favor of hotkey options :)

When you say option defaults revised, do you mean reworded? I tried to make them unambiguous and consistent with Windows wording.

They are clear (Thanks). I meant their status (On by default):

• Set date and time of file from server response
• Prevent system standby while downloads are active

The program saves when shut down properly or after all active downloads have completed. If I were to save every couple of seconds there would be a lot of bytes written to the disk. The entire list gets saved every time it saves. A compromise would be to auto save every 5 minutes. I'll think about it.

OK :) How about also when there is an event change as a completed download or a cleared list etc? You can set a rate limit of maybe 10 seconds to avoid constant writing in case of a lot of events happening in a short period of time.

Paused downloads will stop if the program is shut down and restarted. Only active downloads are continued when shut down and restarted. I kind of like that behavior. I think it would be easy enough to just pause or stop them after the program's been started up again, or before it's shut down.

True, it is easy. But consider servers with limitations (downloads or connections/hour or unresumable downloads for example). If the program unexpectedly closes, it would hit the server limit trying to redownload files already downloaded but not saved to the database/download list before the crash. It is not a common case to happen in general, but can be frustrating when dealing with download projects. I think an option wouldn't harm. If you fear featuritis, you can add them under "Advanced" with a big red warning :)

Per-domain settings would take a lot of work. I'm not sure if the payoff is worth it.

This can wait. I can make do with the Add URL dialog's advanced settings. Maybe keep it at the bottom of your to-do list. :)

Binary settings are the quickest to save and load. I could use XML, but then I'd have to write a ton of parsing code for that since there's no API for XP and newer that'll let me work with it. I also don't want to rely on external resources that aren't native to Windows systems. The only exclusion I made was for the zlib compression library.

How about export and import settings? Is there a way we can make this work somehow?

You can select all of the items in the main window, right click on one of them, and select Copy URL(s). Then just paste it in a file to save.

OK. Thanks!

I don't like extension stores. There's too many hoops to jump through and so many awful comments that I just don't want to deal with. It's easier for me to put everything up on Github.

Fair enough.

HTTP Downloader is a boring name, but it gets to the point. Maybe I'll have something for the future. We'll see.

It was difficult to reach from a search engine. See here.

I've changed downloads that have the File IO Error status to be able to start and restart.

I'll look into adding a file extension for incomplete downloads.

I fixed the list not gaining focus when the main windows is switched back.

The download still going after delete is definitely problematic. I'll have it fixed for the next release.

Checking for a changed modified time would probably be a better solution. I prefer that to redownloading and comparing bytes.

I appreciate all the input. I'm not sure when I'll have a new release for these points. I have other priorities at the moment, but I'll try my best.

Thank you!

---

One more thing :) :

• Don't show a notification when file has been stopped manually.

[erickutcher]

I fixed the drag and drop crash. I could only reproduce the crash with the 64-bit version. The issue was that memory was being freed twice.

The Wiktionary links should now be named with a proper Unicode filename. I discovered an issue with relative paths not being set right as well. That's now fixed.

Downloads that are deleted while still active are now stopped and then deleted.

[t214c]

Thanks!

[erickutcher]

In order to implement the last modified verification check, I had to introduce an extra value in the download history files. As a result, the current history files won't be loaded in the next release. I did create a program (HD History Editor) that can view and modify those history files. It'll also upgrade the history to the latest version. Export the history from the old HTTP Downloader, open it in the editor, save it in the editor, and then import the saved history back into the new HTTP Downloader.

You can use the editor to do multi-host-multi-part downloads like I mentioned here: https://github.com/erickutcher/httpdownloader/issues/44#issuecomment-462081722 The range info has to be changed in order to achieve that. Start, End, and Offset values all begin with an offset of 0. Length values begin with an offset of 1. I can go into more detail if anyone is interested.

I've opted not to implement a file extension for incomplete downloads. I don't like any of the solutions for handling incomplete downloads. Moving files is no good because if the file has to be moved to another partition or drive, then every byte has to be copied and I can't stand unnecessary disk access. Temporary file extensions and names look messy and make it a hassle when dealing with really long file paths. Ideally everything should be managed through the program. Just make sure the download is Completed and you should be good.

I'll have a new version of HTTP Downloader released by the end of the week.

[t214c]

Thanks a lot! You really went all the way with the history editor :)

I kinda wanna press the issue a bit with the clear marking of unfinished downloads. There are multiple reasons for this and the feature is implemented in basically every download manager I've tried. Maybe there are other ways to do it without moving files or long paths (although I think a ".p" after the real extension won't really have that much effect). Maybe we can try something similar to the Firefox way: reserve the real filename and download to a second file with a short "DOS-like" name-path if length is an issue.

Anyway, thanks a lot for your response and responsiveness!

[erickutcher]

A new version is up. Be sure to export your download history from the old version and update it with the history editor before running the new version.

I think a temporary folder would be a more appropriate solution than renaming files. It would be an option though. I'll consider it for another release, but for now I'm taking a break unless there's a showstopping bug.

[smaragdus]

@erickutcher

A new version is up. Be sure to export your download history from the old version and update it with the history editor before running the new version.

It seems that version 1.0.1.9 reads download history from version 1.0.1.8 just fine, or am I missing something?

[erickutcher]

1.0.1.9 shouldn't read any history files that were created by earlier versions. There's a magic ID at the beginning of the history file that would prevent the program from reading an incompatible version. If you saved an earlier history file with the history editor, then it will have been saved as the latest compatible version and you'll be able to load that file in the program (1.0.1.9).

The history editor, however, can load 1.0.1.8 and 1.0.1.9 history files without issue.

[smaragdus]

@erickutcher

You are right, I forgot that I opened the download history file with the editor- it seems that just opening and then closing the history file the editor automatically saves it to the new history format. I tried to open history file created by an older version of HTTP Downloader and version 1.0.1.9 really did not load it. Thanks for the clarification.

However I did not see how this feature works in version 1.0.1.9 (under Windows 8):

Added progress bar in the taskbar for supported systems (Windows 7 and above).

I minimize HTTP Downloader to task bar and close it to system tray- while downloading files when minimized to task bar I did not see any progress bar, perhaps I do not get how this feature is supposed to work.

[erickutcher]

The taskbar progress bar should look something like this:

If you're running the program as an administrator, then it won't work. Neither will drag and drop. UAC prevents that unfortunately.

If all downloads have finished, and you activate the main window (minimize, maximize, restore, gain focus) then the progress will be cleared.

[smaragdus]

@erickutcher On my machine UAC is disabled and I don't see something like that.

[erickutcher]

I noticed, when testing this on Windows 10, that after the download completes, it clears the progress. That doesn't happen on my Windows 7 system. I'm not sure how 8 or 8.1 handle things. I'll have to debug that. In the mean time, try doing a simulated download with this: http://www.ovh.net/files/1Gio.dat It should be large enough to show some sort of progress.

[smaragdus]

@erickutcher

Perhaps it is my system (Windows 8 x64, UAC disabled) but this is what I see in the task bar:

There is no progress bar whatsoever.

Anyway, this doesn't really matter for me because almost always HTTP Downloader is closed to system tray.

[erickutcher]

I ran a copy of Windows 8 Pro x64 and it showed the progress bar when UAC was enabled and disabled. It didn't show it when the program ran as administrator. Do you happen to have "Enable quick file allocation" set in the options? That will run the program as an administrator and prevent the progress from showing. Since you don't have UAC enabled, it wouldn't have alerted you that it was being run under administrative privileges. That's the only thing I can think of.

[smaragdus]

@erickutcher

'Enable quick file allocation' is not enabled. Perhaps the problem is within my system. I have not restarted my machine for quite some time, when I restart it I will make additional tests. Anyway, since HTTP Downloader is usually closed to system tray I do not really need progress bar in task bar. However I thought of something else but I never dared suggest it because I had requested so many things already:

• Progress bar in URL drop window;
• Tray icon changing colour while there are active downloads;

What do you think? Will these features requite too much effort?

[erickutcher]

I can create a progress bar for both the URL drop window and the system tray icon. I'll see what I can come up with.

Here's what I did: When it's paused, it'll turn yellow. If there's an error, it'll turn red. Both progress bars are options in the Options window.

[erickutcher]

I implemented the new style folder browser and the old one is a fallback. I was able to refactor the code enough to where there's no impact in terms of program size.

I also implemented the temporary download folder as an option.

[smaragdus]

@erickutcher

My mistake- the progress bar in task bar works just fine in both versions 1.0.1.9 and 1.0.2.0:

I was switching back and forth between versions 1.0.1.8 and 1.0.1.9 (because of the new download history format) and somehow I missed that I kept version 1.0.1.8 thinking it was version 1.0.1.9. Please excuse me for my omission.

The progress bar in system tray and in URL drop window is nicely implemented- thank you very much!

I wonder- after a download is finished shouldn't the progress bar in system tray and in URL drop window be automatically cleared? Now it is cleared after main window is maximized and then minimized again.

[erickutcher]

I have it keep the progress until the user interacts with any of the windows so that it acts as an acknowledgment that the downloads have finished. If there's an error, for example, the progress will turn red when everything is finished. Suppose you start a large download and walk away until it completes. If you get back and see the bars cleared, you might think the download never started. And you wouldn't know if any error occurred. That was my justification.

[smaragdus]

@erickutcher

Your justification is logical and I accept it. Off-topic- I could not find what is new in the latest versions of history editor (1.0.0.2) and chrome extension (1.0.0.5).

[erickutcher]

For the history editor, I added a list box for flag values. You can select multiple values at the same time. Although, not all combinations are valid. The Downloading status flag can be paired with Queued or Paused, for example.

As for the extensions, I created a tabbed section in the advanced section that contains the Cookies, Headers, and POST Data text boxes. I did the same with the program as well. Keeps things more compact.

[smaragdus]

@erickutcher

I experienced a tiny issues- a couple of times the progress indicator in both URL drop window and in system tray icon was not cleared after all the downloads had been completed and the main window was restored- I did close and restore it several times but to no avail. The workaround was exiting and restarting the program. Can you confirm that? Or the problem might be system specific?

[erickutcher]

I haven't noticed that myself, but I had to change the code a bit to fix a bug that could cause what you're experiencing.

[smaragdus]

@erickutcher

I haven't noticed that myself, but I had to change the code a bit to fix a bug that could cause what you're experiencing.

This happens not so seldom. For the time being the only cure is restarting the program.

I have noticed something similar- a wrong notification- double notification:

• Completed: 1
• Timed Out: 1

Screen:

• Completed: 1
• Timed Out: 2

Screen:

although all downloads have been completed and there is not a single incomplete/timed out download. This happens mostly when I download APK files from F-Droid, a website notorious for its connectivity problems.

[erickutcher]

It's counting the retry attempts when it times out/fails. This will be fixed in the next release.

[erickutcher]

The latest version is up. I went through the code that clears the completed progress bars when a window is activated and I can't see anything that would prevent it from working. The minor changes I made relating to it might have fixed it, but I don't know.

[smaragdus]

@erickutcher

I have already updated to version 1.0.2.1. I hope that the problem with not clearing progress bar after opening main window is resolved. I will check if it would appear again. The only way I can prove the problem is taking a full screen screen-shot which would contain the main window, the tray icon and the URL drop window. Thank you for the new version.

[smaragdus]

@erickutcher

I have been using HTTP Downloader version 1.0.2.1 for 2 days and so far the problem has disappeared- the progress bar is always cleared both in URL drop window and in system tray icon after opening the main window. This was not the case with the previous version (1.0.2.0).

The minor changes I made relating to it might have fixed it, but I don't know.

It seems these changes really fixed the problem, thank you.

[smaragdus]

@erickutcher

The problem with progress bar not cleared after main window is maximized still appears, but rarely.

[erickutcher]

If the main window is in focus when all of the downloads finish and you maximize it, then the progress bars won't clear. It's only when the program loses or gains focus that it will clear the progress.

[smaragdus]

@erickutcher

If the main window is in focus when all of the downloads finish and you maximize it, then the progress bars won't clear. It's only when the program loses or gains focus that it will clear the progress.

I believe this was not the case:

• I add a new download;
• The program is closed to tray;
• The download completes;
• I open the main window;
• The progress bars (url drop window and tray icon) do not get cleared;
• I restart the program to get the progress bars cleared;

I download lots of files and HTTP Downloader is always running so I will keep on testing. If the problem appears again I will try to provide more details.

[erickutcher]

The next time it happens see if the progress bars get cleared when you start or restart a download. Also, take a screenshot of the main window for me.

There's a variable that gets set when all downloads finish. It only gets reset when downloading again, or a window activation causes a focus change.

Does the taskbar progress also get cleared, or is it just the tray icon and drop window?

[smaragdus]

@erickutcher

This happened again. I cannot recount any special circumstances, this is what I did:

• I just downloaded 4 files in a row from Gajim;
• During the download the program was minimized to tray;
• After all the downloads were completed I maximized the main window;
• The progress bar in task bar icon was cleared;
• The progress bar in URL drop window was not cleared;
• The progress bar in system tray icon was not cleared;
• I restarted one of the completed downloads (the last one) and both progress bars in URL drop window and in system tray icon were cleared;

screen:

If needed I can also provide a full screen shot which shows that the progress bar in task bar icon was cleared.

The next time it happens see if the progress bars get cleared when you start or restart a download.

Does the taskbar progress also get cleared, or is it just the tray icon and drop window?

The progress bar in task bar icon is always cleared, the progress bars in URL drop window and in system tray icon are these which do not get cleared sometimes. I cannot see anything extraordinary- just normal downloads completed in a normal way. Most of the time I add new downloads, in fact- almost always, HTTP Downloader resides in system tray.

If I can provide any other information needed please ask.

edit If this happens only to me I presume the problem might be withing my system, I do not know.

[erickutcher]

Here's the lines of code where the progress values get reset: 3261 to 3281

If the program is clearing the taskbar progress, then there should be no reason why the other two aren't. And if it only happens some of the time, that leaves me stumped.

[smaragdus]

@erickutcher

full screen image

The raw, unedited image shows that the progress bar in task bar icon is cleared but the other two progress bars are not.

Anyway, this is not of any importance and it happens randomly.

[t214c]

Can you please please please make an option to clear the progress bars automatically when they finish successfully (not yellow or red)?

Leaving it at green runs against the convention of Windows and every other software I know of.

And honestly it makes me anxious. Like a sneeze that won't come :|

Thank you!

[erickutcher]

I'll put it on my to-do list.

[t214c]

Thanks a lot! Appreciate it!

[smaragdus]

@erickutcher

It seems that version 1.0.2.6 finally resolves the problem with tray icon and URL drop window progress bars sometimes not cleared when the downloads are completed, thanks.

[erickutcher]

I had some progress values inside a code block that wouldn't have been reached under certain conditions. That's probably what fixed it.