search

wkeeling / selenium-wire

Extends Selenium's Python bindings to give you the ability to inspect requests made by the browser.
MIT License
1.9k stars 254 forks source link

Can we enable forced caching ?[#feature_request] #11

Closed appunni-m closed 2 years ago

appunni-m commented 5 years ago

As selenium wire is a proxy, it is the best place to enable caching. And from what I understand selenium wire is handling SSL Termination. So might as well add an option to force cache successful requests for a specified period of time.

wkeeling commented 5 years ago

Hi @appunni-dishq - thanks for this. I like this idea. Caching could improve the performance of Selenium tests fairly significantly. Implementation shouldn't be too difficult. Responses that are returned from the cache could be marked as such with a cached attribute or similar. The cache expiry could be set as an option with a reasonable default.

I'll look into this in more detail in the next few days.

appunni-m commented 5 years ago

Application cache at web driver happens according to the Cache-Control headers so it's not selenium-wire's headache to manage that. Selenium-wire can instead provide a switch or a flag, which control cache up to a certain time.

wkeeling commented 5 years ago

Ok I misunderstood. So you're suggesting that selenium-wire could insert the relevant Cache-Control headers into the request to control caching, rather than performing the caching itself. This should be far more straightforward.

appunni-m commented 5 years ago

@wkeeling I didn't think that way, but I know that the storage is cleared everytime I restart selenium-wire. So it makes sense to use user-data-dir for caching. Otherwise, we would need a completely new implementation of storage.

Miando commented 3 years ago

Hi guys. I am also want to use cache, because it is to slow use selenium wire in the same website

appunni-m commented 3 years ago

I moved to pyppeteer. It was around 10 times faster the selenium + selenium wire.

wkeeling commented 2 years ago

Using Selenium Wire's storage as a temporary cache is out of scope of the project at the current time.

lucanello commented 2 years ago

I've got a question regarding this issue. Does the missing caching feature mean that I am not able to read the response body if cache control is set to either no-cache, no-store or must-revalidate?

wkeeling commented 2 years ago

@lucanello the response body should be available for no-store since this effectively means "don't cache". The other two directives however don't guarantee that the response won't be cached. In these cases you may find that the response body isn't always available.