The submission_worker and remote_submission_worker have a lot in common currently. Naturally, if a change happens in the future, code in both files will have to be edited.
In addition to that, many functions are very much exactly the same.
Moreover, within each file, there is repetition of code, e.g.: the download_and_extract_file and download_and_extract_zip_file use the same code for the downloading part.
Expected:
The code should be modularized and helper methods / utils should be created accessible to both the workers. This will:
allow easy updates in the future
make the code more readable
tests can be made more effective / independent of the specific modules used. e.g., for testing if a file has been asked for downloading, a util function download_file can be mocked rather than having to mock requests.get which may change in the future if we switch the library used.
Objective of resolving PR:
Analyze the code, find reused / utility code.
Make utils accessible to both
Keep required changes in the main code to a minimum
Try to keep required edits in the corresponding tests to a minimum.
@RishabhJain2018 @Ram81 This is my suggestion, please let me know if a PR for this is required. (I am also a GCI participant, so it would be great to have this as a task :D)
Current:
The
submission_worker
andremote_submission_worker
have a lot in common currently. Naturally, if a change happens in the future, code in both files will have to be edited. In addition to that, many functions are very much exactly the same. Moreover, within each file, there is repetition of code, e.g.: thedownload_and_extract_file
anddownload_and_extract_zip_file
use the same code for the downloading part.Expected:
The code should be modularized and helper methods / utils should be created accessible to both the workers. This will:
download_file
can be mocked rather than having to mockrequests.get
which may change in the future if we switch the library used.Objective of resolving PR:
@RishabhJain2018 @Ram81 This is my suggestion, please let me know if a PR for this is required. (I am also a GCI participant, so it would be great to have this as a task :D)