mosharaf13 / fleet-searcher

Upload a CSV file containing keywords and view google search stats
1 stars 0 forks source link

[Feature] Asynchronous processing of keywords #31

Open olivierobert opened 1 year ago

olivierobert commented 1 year ago

Issue

Upon uploading keywords, the scraping will be performed synchronously in the controller:

https://github.com/mosharaf13/fleet-searcher/blob/dc70169cf85dd164c55b9b25b355c34390aa04f2/app/Http/Controllers/SearcherController.php#L41-L43

https://github.com/mosharaf13/fleet-searcher/blob/dc70169cf85dd164c55b9b25b355c34390aa04f2/app/Http/Controllers/SearcherController.php#L55-L56

Even if it is performed in batches, the request can still take a long time to complete (and potentially could exceed the HTTP request timeout). In addition, if any errors occur for any keyword, the processing of keywords will stop.

Note Using Laravel Events to decouple the database operations from the Google search is acknowledged as a nice-to-have improvement.

Expected

The benefits are:

mosharaf13 commented 1 year ago

Hi,

Created pull request to add this feature. Also, this PR resolves #36 and #37. So, I have closed my previously created two PR's. Please provide suggestions if there's any changes that should be made to upgrade the quality of user experience or code base.