OWASP Top 10 Claim

[digininja]

Your readme states:

Akto offers tests for all OWASP top 10

Can you explain how you test for these two issues:

A04:2021-Insecure Design A09:2021-Security Logging and Monitoring Failures

[Ankita28g]

Hi @digininja

Thanks for the question.

1. We offer tests for OWASP Top 10 for API security.
2. Couple of categories are in beta. Our roadmap is public. We will have it available for all very soon.

Your suggestions are welcome.

[digininja]

Can you give a quick summary of how your beta version covers these two issues:

API9:2019 Improper Assets Management

API10:2019 Insufficient Logging & Monitoring

[ankush-jain-akto]

Hi @digininja

API9:2019 Improper Assets Management - Old API versions: In many cases, we detect old API versions. For example, if an API looks like GET /api/v3/users, we try to check vulnerabilities for GET /api/v2/users and GET /api/v1/users as well. This is solved through a test case Old API Versions Test. As a future implementation, we are also considering to get list of APIs from the source code as well. This implementation is currently under very early stages of development. [Edit: Akto has a provision to capture data from application servers directly through Traffic mirroring. If you have any unpatched systems, as long as they are receiving traffic, Akto can log all the API endpoints. Users can check for old api endpoints on Akto dashboard (say /api/v1 or /api/v2) and request the dev team to retire or patch these servers]

API10:2019 Insufficient Logging & Monitoring - Akto captures all the API traffic that comes to the application servers. User can choose to save all this traffic. This way, we can completely log all the requests. When we detect any misconfiguration (for example, endpoint sharing any sensitive info), a Slack alert also goes to the team.

[digininja]

So for 10, you are logging your requests, not testing if they are logging anything, is that right?

And if you find an issue, you are doing the alerting, you aren't testing if they are sending off any kind of alerts internally.

On Sat, 11 Feb 2023, 09:07 Ankush Jain, @.***> wrote:

Hi @digininja https://github.com/digininja

API9:2019 Improper Assets Management - Old API versions: In many cases, we detect old API versions. For example, if an API looks like GET /api/v3/users, we try to check vulnerabilities for GET /api/v2/users and GET /api/v1/users as well. This is solved through a test case Old API Versions Test https://github.com/akto-api-security/tests-library/blob/master/BOLA/business-logic/OldApiVersionTest.java. As a future implementation, we are also considering to get list of APIs from the source code as well. This implementation is currently under very early stages of development.

API10:2019 Insufficient Logging & Monitoring - Akto captures all the API traffic that comes to the application servers. User can choose to save all this traffic. This way, we can completely log all the requests. When we detect any misconfiguration (for example, endpoint sharing any sensitive info), a Slack alert also goes to the team.

— Reply to this email directly, view it on GitHub https://github.com/akto-api-security/community-edition/issues/14#issuecomment-1426668031, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWMWHJVIPWEMTI3XQFTWW5JE7ANCNFSM6AAAAAAUTXEANY . You are receiving this because you were mentioned.Message ID: @.***>

[ankush-jain-akto]

Yes. We are logging it ourselves. User can send to any logging tool as well (eg DataDog, Newrelic etc.) We are not testing if they are being logged already.

We are not testing if requests are logged on a diff tool or if alerts are set on a diff tool. Users can choose Akto to log & alert or send this data to their favourite tool and set logging/alerting from there itself. [Edit: This reply is for the current master branch version. @digininja, do you have any suggestions how we can test for Logging/Monitoring? We do have a few ideas there. Would love to learn from you if you have tried to solve this through testing.]

[digininja]

So you aren't testing either of these top 10 issues. I think you need to look at the wording of your readme as it is misleading at the moment.

On Sat, 11 Feb 2023, 10:03 Ankush Jain, @.***> wrote:

Yes. We are logging it ourselves. User can send to any logging tool as well (eg DataDog, Newrelic etc.) We are not testing if they are being logged already.

We are not testing if requests are logged on a diff tool or if alerts are set on a diff tool. Users can choose Akto to log & alert or send this data to their favourite tool and set logging/alerting from there itself.

— Reply to this email directly, view it on GitHub https://github.com/akto-api-security/community-edition/issues/14#issuecomment-1426683115, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWN7UPIPSXFT7H5EOPDWW5PXBANCNFSM6AAAAAAUTXEANY . You are receiving this because you were mentioned.Message ID: @.***>

[ankush-jain-akto]

Hi @digininja - we do solve for API9:2019 Improper Assets Management through a test case Old API Versions Test as I mentioned above.

I will update the Readme file accordingly. I will make a PR linked to this issue too.

[digininja]

Looking to see if there is a v2 when you are testing v3 isn't really doing any kind of thorough testing. If a tester told me that that is all they did to look for that area then I'd send them back to do a lot more work.

On Sat, 11 Feb 2023 at 10:24, Ankush Jain @.***> wrote:

Hi @digininja https://github.com/digininja - we do solve for API9:2019 Improper Assets Management through a test case Old API Versions Test https://github.com/akto-api-security/tests-library/blob/master/BOLA/business-logic/OldApiVersionTest.java as I mentioned above.

I will update the Readme file accordingly. I will make a PR linked to this issue too.

— Reply to this email directly, view it on GitHub https://github.com/akto-api-security/community-edition/issues/14#issuecomment-1426688043, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWOY7ZWSIRQ66SU7I5LWW5SG7ANCNFSM6AAAAAAUTXEANY . You are receiving this because you were mentioned.Message ID: @.***>

[digininja]

You still say you cover all the top 10 issues which you don't do.

On Wed, 15 Feb 2023, 13:38 aktoboy, @.***> wrote:

Closed #14 https://github.com/akto-api-security/akto/issues/14 as completed via #20 https://github.com/akto-api-security/akto/pull/20.

— Reply to this email directly, view it on GitHub https://github.com/akto-api-security/akto/issues/14#event-8526163043, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWMVXOKGU43J6ICWPFLWXTL6HANCNFSM6AAAAAAUTXEANY . You are receiving this because you were mentioned.Message ID: @.***>

[ankush-jain-akto]

Hi @digininja, These discussions are quite useful to us. Really appreciate you helping us fill the understanding gaps. We want to be very transparent & provide complete info about what we exactly do. Thanks from our entire team for this! I am happy to say we are yet to cover a particular category, if that is really the case. I want to be thorough to understand if this is a documentation problem or actually insignificant coverage problem. I am assuming API-9- Improper Assets Management is the subject of concern here. Taking a section from OWASP website about this issue, I am adding how Akto helps in each of these -

1. The purpose of an API host is unclear, and there are no explicit answers to the following questions:
   1. Which environment is the API running in (e.g., production, staging, test, development)? Akto provides this out of the box. If you install Akto using Traffic Mirroring in your VPC, this happens automatically in real time.
   2. Who should have network access to the API (e.g., public, internal, partners)? For each API on Akto dashboard, you can see a column called "Access Type" which says Public/Internal.
   3. Which API version is running? If API version is a part of the URL, this again is logged in Akto dashboard and users can tag the APIs containing "v1" as #version-1. They can also set alerts - if a new v1 API is added (no additions should be done to old API versions)
   4. What data is gathered and processed by the API (e.g., PII)? Akto provides this too out of the box. If some PII patterns are very business specifc, users can add their custom PII patterns too.
   5. What's the data flow? This isn't covered yet.
2. There is no documentation, or the existing documentation is not updated - Akto prepares an inventory for all APIs across all services. It also provides a way for users to download it as an OpenAPI spec file or export it to their Postman workspace. Users can do it any number of times. In fact, they can prepare a cron job to extract OpenAPI spec file from Akto and save it in their repo.
3. There is no retirement plan for each API version - This is more of a process fix rather than a product fix. But Akto also mentions APIs that are not being used anymore. Users can take these APIs and test themselves if they are running. They can also export it as a CSV and send it to dev teams to confirm if they are shut down from backend also.
4. Hosts inventory is missing or outdated - Akto can fill this too. Because it captures all the traffic, all the hosts are also automatically registered along with all their APIs too.
5. Integrated services inventory, either first- or third-party, is missing or outdated - Akto doesn't provide this out of the box. This requires an integration. Third party services are typically called via https. We get this traffic too for analysis, but nothing useful comes out because its encrypted. However, devs can use our SDKs to send a copy of the HTTP call to Akto and then Akto dashboard can show the 3rd party calls as well. Too much work - meant for desperate users only I would say.
6. Old or previous API versions are running unpatched - If old versions are running, they will be logged by Akto and tested too for all vuln.

You would see a lot of dependency on traffic here - which is true for Akto. Akto's source of information is traffic data. As long as there is even 1 API hit, Akto will log it and analyze it for vulnerabilities. Akto will also document it and users can check for its traffic usage, download openapi-spec, set alerts etc. If an API is never hit, it won't be logged by Akto. In such cases, we do try to find these inactive APIs by changing version numbers. In future we want to launch an endpoint discovery module too - through fuzzing (sitting outside the application) or code analysis (sitting inside the application). Does this help @digininja? Please let me know if we are missing major cases. Also, putting in a user-research plug here. Personally, I know you are a legend when it comes to this space. Would love to know more from you on how you envision some of the API security vuln be exposed through testing. Your suggestions will be quite valuable to us & I promise I will put them in our product.

[digininja]

My problem is with this statement:

Akto offers coverage for all OWASP top 10 and HackerOne Top 10 categories

You don't test for API:10, you can't do it in an automated tool and so your statement is wrong.

For API:9, you may cover it, but I would argue that it can't be automated, you may be able to give some clues to things which may be there when they shouldn't, but you can't state there are any issues as the tool doesn't know enough about an environment to make that call. It may be that you find versions 2 and 3 but they are both designed to be live.

Asset management is also a lot more than just a version number.

Have you considered mapping yourself to the https://owasp.org/www-project-web-security-testing-guide/ instead as that is designed to be testable whereas the Top 10 is just a list of common issues, it isn't designed to be something you can automate or say that you cover.

[craig-shony]

I wonder if the appropriate wording would be "Akto offers coverage for all applicable OWASP Top 10 (API Security)" because it's not possible for a DAST-like tool such as Akto to provide coverage for all 10. As OP mentioned, some of the Top 10 are not suitable for automated testing.

[digininja]

That would be more appropriate.

[Ankita28g]

hi @digininja @craig-shony

Acknowledged. We will be keeping this issue open and get more feedback. Meanwhile, we will also be getting more first hand opinions from folks in our network.

will be update the wording here once we collect feedback.

thanks!