Expose done/queue list as JSON endpoint

[arabcoders]

Hello,

I have exposed the done/queue list as JSON endpoint which helps in aiding of writing scripts to capture live streams as sometimes the added item or cronjob will miss events. Some channels i follow after the live stream ends they make the video private thus the next time the cronjob runs, it will be presented with a private video message.

I have also fixed unrelated bug that mostly happens probably in docker if you sometimes suddenly restart docker container when the container was downloading video, the queue get corrupted. This fix simply checks if the entry is actually set instead of assume it is and crashing. Example of the error this bug fixes

AttributeError: 'NoneType' object has no attribute 'get'
etype = entry.get('_type') or 'video'
File "/app/app/ytdl.py", line 246, in __add_entry
return await self.__add_entry(entry, quality, format, folder, custom_name_prefix, already)
File "/app/app/ytdl.py", line 293, in add
await self.add(v.url, v.quality, v.format, v.folder, v.custom_name_prefix)
File "/app/app/ytdl.py", line 207, in __import_queue
Traceback (most recent call last):
uture: <Task finished name='Task-3' coro=<DownloadQueue.__import_queue() done, defined at /app/app/ytdl.py:205> exception=AttributeError("'NoneType' object has no attribute 'get'")>
ERROR:asyncio:Task exception was never retrieved
INFO:ytdl:adding https://www.youtube.com/shorts/sample: quality='best' format='mp4' already=None folder='sample' custom_name_prefix=''
INFO:ytdl:waiting for item to download

I have changed the url and folder to prevent copystrikes

[arabcoders]

@alexta69 I've implemented the suggested fixes.

[arabcoders]

@alexta69 i started with using the same serializer however it was complaining about DownloadInfo cannot be JSON encoded

[alexta69]

@alexta69 i started with using the same serializer however it was complaining about DownloadInfo cannot be JSON encoded

So now with the latest changes it works OK?

[arabcoders]

@alexta69 i started with using the same serializer however it was complaining about DownloadInfo cannot be JSON encoded

So now with the latest changes it works OK?

From my limited testing with sample data yeah.

[alexta69]

Perfect. Thanks for your contribution!

[arabcoders]

Perfect. Thanks for your contribution!

Thank you. Now i wish to tackle multi download. As i download mostly live streams from youtube and usually the streamers start at or close to each other. Do you have an interest in that?

[alexta69]

I would merge such a PR, sure. A configuration variable would need to be added that limits the number of concurrent downloads, and I think the safest would be to default to 1, and let users set it if needed, because I'm afraid that concurrent downloads opened by default might get people throttled (maybe not on youtube but on other platforms, who knows).