781flyingdutchman / background_downloader

Flutter plugin for file downloads and uploads
Other
158 stars 73 forks source link

Query all Completed Download #4

Closed basnetjiten closed 1 year ago

basnetjiten commented 1 year ago

Hi thanks for this awesome alternative package.🙏 I am thinking to switch from flutter downloader, as this package meets all of my requirements.

I have a question though which bits confuses me. Our app is complete offline app after file downloads and mostly used images and audio and very less video file.

In the app we have the list of task to be downloaded and hence it's exact length. I just want to Query if all the task has been downloaded (don't need progress for individual task) , but check for all file download completes.

So that later I can test the list of original download task and the completed Task have <= length of original list.l, that confirm that although some task failed but 90% of task are completed and I want to route to another page.

Thank-you and hope to get your response

781flyingdutchman commented 1 year ago

Hi, thanks for reaching out and great to hear the package meets your needs! If I understand it correctly, you want to download a list of files, and then check if all downloads succeeded. If you know the list of files you want to download, then I think the easiest way to do this is using downloadBatch: create a List of BackgroundDownloadTask objects, one for each file, in variable tasks, then download that list of files in a batch, like so:

  FileDownloader.initialize();  // initialize only once
  final result = await FileDownloader.downloadBatch(tasks);

This method won't return until all tasks in the tasks list have been completed (either successfully, or failed). The return value (in result) is a DownloadBatch object that you can test - for example, you can use something like this:

if (result.numFailed > 0) {
  print('${result.numFailed} tasks failed');
  print(Failed tasks: '${result.failed.toList()}');  // in practice you would for example retry the .failed tasks
} else {
  print('All tasks downloaded successfully');
}

Alternatively, if you don't have the full list of tasks ready in advance, you can add each separately using enqueue, but you'll not have the convenience of the batch download - for example, you then need to listen to FileDownloader.updates to keep track of each task completion individually.

Either way, you may also want to manage what may happen when the app goes into the background/suspended, and then resumes. Status updates for tasks completed while in the background may not have been properly tracked by your app (this is a feature of the Android and iOS platforms, and is unavoidable), so when your app resumes you may want to check what tasks have completed or failed while the app was in the background. There are many possible strategies here. A comprehensive one is to store your list of tasks in 'permanent' storage (e.g. in preferences, or in a simple database), so it survives app shutdowns and restarts. For convenience, you can call the .toJsonMap() method on a BackgroundDownloadTask to get a JSON representation that you can store as a String, and .fromJsonMap to create the object again.
Upon resume, you then check for existence of each file in that list, and if the file exists you remove the task from that list (because the download completed successfully while in the background, otherwise the file wouldn't be there). If your remaining list is empty, you are done. If not, you need to figure out if those remaining tasks are still running, or if they failed while in the background: query FileDownloader.allTaskIds to get a list of running download tasks, then iterate over that list and call Filedownloader.taskForId(taskId) to get the actual BackgroundDownloadTask object for each taskId, and compare that with your list of remaining downloads to make sure that all files you still need to download are indeed running. Those not running are the downloads that failed, and you can choose to enqueue those again, or handle failures differently.
A much simpler strategy is to store the timestamp of when you started your batch download in preferences, and upon resume IF not all files are downloaded AND substantial time has passed, THEN simply download the remaining files again. The right approach depends a lot on your application.

I hope this helps - let me know if you get it to work, and/or if you have suggestions to improve the package.

781flyingdutchman commented 1 year ago

Actually, looking at your question again, I think you're asking for something simpler. If you just want to know what download tasks are still running, you can call FileDownloader.allTaskIds, which gets you a list of tasks that are still in progress. When that list is empty (or if that list doesn't contain any taskIds that are in your list of submitted taskIds) then all downloads have finished. I would still suggest that you also check that each file exists, because a download can finish with a failure, and failed tasks will not show up in the list of taskIds returned by FileDownloader.allTaskIds.

781flyingdutchman commented 1 year ago

@basnetjiten in the most recent version (1.5.0) you can call FileDownloader.allTasks to get a list of tasks that are still running. Per your suggestion, if that list is empty (or does not contain any of the tasks that are in your list of submitted tasks) then all downloads have finished. I would still suggest that you also check that each file exists, because a download can finish with a failure, and failed tasks will not show up in the list of taskIds returned by FileDownloader.allTasks.

markst commented 1 year ago

@781flyingdutchman is it possible to retrieve all completed downloads (between app launches) similar to how flutter_downloader works:

final tasks = await FlutterDownloader.loadTasks();

I suppose this would require persistence layer which perhaps is something you wanted to avoid when creating background_downloader

markst commented 1 year ago

Seems simple enough to use hive or similar to persist the downloads upon completion. Then have some migration from existing flutter_downloader usage.

781flyingdutchman commented 1 year ago

Hi @markst thanks for checking this out. That persistence layer is indeed what I've wanted to avoid, as it adds a ton of complexity and dependencies, and that is one reason why flutter_downloader is rather buggy, IMHO. I use background_downloader to download thousands of files, and it does that fine even as the app moves to the background on both iOS and Android. There is always the possibility that the app is fully stopped, or the queued tasks are unceremoniously deleted by the OS, so the approach I take is that upon app resume I check a) what files do I need, b) what files exist (take those off the list of needed files), and c) what files are scheduled (take those off also). Whatever is left on your list, you enqueue again, as they got lost somewhere.

The native platforms are quite aggressive in managing background tasks, so there is no 100% reliable way to keep track of state - it won't work with a database either, as flutter_downloader demonstrates, because you simply will not be notified by the platform in all cases about a change in task state, and then your database does not reflect state accurately either. The only option really is to do a check upon startup/resume and enqueue any remaining files needed for download.

In practice, I have found that my app continues downloads in the background for quite a while, even after the user has moved onto another app or the device goes to sleep, so I have really not found this to be an issue.

Hope this helps - let me know what you think.

781flyingdutchman commented 1 year ago

Sorry, forgot to add that there is some persistence in the plugin: the native side stores a list of tasks in progress in 'preferences', which persists through app shutdowns. If the app wakes up and the OS sends it a lot of task completion messages (that happened in the background while the app was asleep) then as long as you have registered your callbacks immediately upon startup, you will get all those status update messages sent to you for processing as well.

781flyingdutchman commented 1 year ago

Finally, @markst I saw you reference this issue that I opened in 2020 to address the relative savedDir issue. I submitted a pull request to fix it, acknowledging that it breaks the API and requires a major version change for flutter_downloader, but it never was adopted. As a result, you now end up with very platform-specific code to work with that plugin, and that is really not the spirit of plugins, which is why I decided to create an alternative :)

781flyingdutchman commented 1 year ago

@basnetjiten the latest version now has an optional persistent database that holds the state for each download. Perhaps this enables the use case you were asking about originally?