54ac / stream-detector

A Firefox addon for keeping track of manifests used by various streaming protocols and downloading media files.
https://addons.mozilla.org/en-US/firefox/addon/hls-stream-detector/
Mozilla Public License 2.0
643 stars 97 forks source link

Request: Add support for checking Content-Type in headers for all URLs on fly #128

Closed Dean-Corso closed 2 years ago

Dean-Corso commented 2 years ago

Hi rowrawer,

so today I have some kind of special request and I'am not sure whether you will like it as I do or whether you would add the new idea / feature to this stream-detector extension/addon.

I would like to check all loaded URLs (see network tab) content-type headers for specific types when you load a webpage in browser. So now you will say hey this feature is already added which is correct so far but this only works when a URL also get accessed / loaded like to press on a player button and then you catch the data for a video etc you know. So in my case now I would like that your extension does scan all URLs on fly and just checking the content-type in header (for types I have added to look for) and when it does match then I want your extension to show also that URLs in your URL list. I think so far you did understand me what I mean right? Now you will ask why? I'am not only looking for streams / videos etc itself so I'am also looking for specific content like JSON files where offten many helpfully info are stored and I would like to prevent checking the network tab itself to find them etc you know. I want your extension to do the scan and list the found URLs in your popup URL window. Next point is that I want to open that JSON file in a new tab in browser what is also doable with your extension (right mouse / open new tab or middle mouse click) but here you need to add a new button to open the file in new tab and you extension must use the same original request parameters like you can see in the network tab. In some cases you need to request with many paramters cookies / origin / referer etc to get a valid successfully response. What do you think about that? Could you add such feature in your stream-detector extension or would that be already too far away for you? If you think so then it would be maybe possible to create a new extension what can do this and you could use your stream-detector as template etc.

Example: I do add the content-type "application/json" into your extension. Now it should scan all URLs headers for this content-type when I load any webpage. Lets say your extension did the job and found 4 URLs what using the content-type in header and now your extension should add all 4 URLs into your URL list. Now I can see that your extension did found some JSON URLs and I can click on it with left mouse to copy it into clipboard or I can use right mouse / middle click to open URL in new tab etc. But to open it in new tab only works when no extra request parameters are needed to get status 200 back. I want that your extension can execute the URL orignal with all request paramters (optional / just add a another small button or something in your popup window which is called like open original or so). Something like this you know. So what do you think about it? Is that doable for you and if yes would you add to this stream-detector extension or better create another extension etc? So the goal in this special request is it to log specific URLs with specific content-types in their response headers to get those URLs listed in your extension popup window and to have a quick access on them without to do those manually steps. But the important thing is the option to request the URL with all used original paramters (status 200) or otherwise I need to check network tab itself again to read JSON content there but I don't like that detour and would prefer a more quicker & easier way like your extension does already with the streams when they getting loaded / executed. Its almost same only a little more specific. Just think about it and send your opinion. Thanks.

54ac commented 2 years ago

Hi, thanks for your incredibly detailed request.

This is already implemented to its possible extent ("detect additional Content-Type" option). Unfortunately, requests cannot be perfectly replicated most of the time, as that simply is not allowed by the browser, especially Chrome, for safety reasons. That being said, as long as you use the same browser, any relevant cookies should still be available, even when you make a manual request.

Dean-Corso commented 2 years ago

Thanks for your answer rowrawer. Sounds not very good so far. There mostly many files they need to use cookies & Co parameters to make a valid request. Sure I could check the network tab to see the content there but its just laborious each time to do that you know. Really a pity that this is not doable. 😢 Anyway, thank you again rowrawer.

54ac commented 2 years ago

I'll change the direct download option to also work on custom entries - that way you'll be able to download any custom content types/extensions through the addon. It might help.

Dean-Corso commented 2 years ago

Sounds good rowrawer, so lets give it a try on your next release. 😃

54ac commented 2 years ago

Direct download is now available for custom-detected entries as of v2.10.9.