feat(page): recent scans add page jumper

MobSF / Mobile-Security-Framework-MobSF

Mobile Security Framework (MobSF) is an automated, all-in-one mobile application (Android/iOS/Windows) pen-testing, malware analysis and security assessment framework capable of performing static and dynamic analysis.

https://opensecurity.in

GNU General Public License v3.0

17.18k stars 3.22k forks source link

feat(page): recent scans add page jumper #2348

Closed miaoyc666 closed 4 months ago

miaoyc666 commented 7 months ago

Describe the Pull Request

When MobSF has a large amount of sample data in the system, the recent scans api speed has been very slow because it will retrieve full table data from the database.

To fix this, recent scans api add page jump, default display 10 items per page

Renderings

Checklist for PR

[x] Run MobSF unit tests and lint tox -e lint,test
[x] Tested Working on Linux, Mac, Windows, and Docker
[x] Add unit test for any new Web API (Refer: StaticAnalyzer/tests.py)
[x] Make sure tests are passing on your PR

Additional Comments (if any)

DESCRIBE HERE

ajinabraham commented 6 months ago

Can you also include the changes in UI as well?

miaoyc666 commented 6 months ago

Can you also include the changes in UI as well?

Do you mean adding paging parameters in the fe as well? Of course, it's just that it takes time. 😂

miaoyc666 commented 6 months ago

Can you also include the changes in UI as well?

@ajinabraham ui done!

ohyeah521 commented 6 months ago

@miaoyc666 Do you have any suggestions for optimizing the database? I found that when the amount of data exceeds 30,000, data query will become extremely slow, even if the page display is still very slow

miaoyc666 commented 6 months ago

@miaoyc666 Do you have any suggestions for optimizing the database? I found that when the amount of data exceeds 30,000, data query will become extremely slow, even if the page display is still very slow

@ohyeah521

When you query a database, the first thing to control is the number of rows you query. You need to add offset and limit parameters to your SQL statement for pagination queries, limiting the number of rows returned. Secondly, you need to select the fields that need to be displayed for select queries. Do not search for useless fields and cannot use select * because it will query all fields in a record, which will consume additional network bandwidth and CPU computing resources. You can take a look at the code I commited, perform pagination queries in django, and only query the fields displayed on the page in the recent list of samples.

ohyeah521 commented 6 months ago

@miaoyc666 thanks

ajinabraham commented 4 months ago

@miaoyc666 Thanks for taking your time and working on the UI bits. I will review this soon and get back to you.

ajinabraham commented 4 months ago

Initial testing looks good. I will see if I can improve up on the UI based on https://realpython.com/django-pagination/