WebFuzzing / EvoMaster

The first open-source AI-driven tool for automatically generating system-level test cases (also known as fuzzing) for web/enterprise applications. Currently targeting whitebox and blackbox testing of Web APIs, like REST, GraphQL and RPC (e.g., gRPC and Thrift).
GNU Lesser General Public License v3.0
524 stars 85 forks source link

Phg/run bb e2e py #1042

Closed Pgarrett closed 3 months ago

Pgarrett commented 3 months ago
arcuri82 commented 3 months ago

@Pgarrett thx ;) it looks like there is some issue with GitHub, that is not running the jobs :( i ll merge once all green (i ll try to re-run the jobs). at the moment, unclear if issue is temporary at github, or something we introduced

arcuri82 commented 3 months ago

@Pgarrett looks like there are some issues when running on Windows (eg my machine). i m looking into them now

Pgarrett commented 3 months ago

Hmm, I guess it'll be related to how to call python. I had looked into it and found that it was called the same way

arcuri82 commented 3 months ago

@Pgarrett i saw at least 2 issues, which hopefully should be fixed now: 1) write-rights (used --user, hopefully it does not have side effects) 2) timeouts don't work on Windows :( so they are disabled there now

arcuri82 commented 3 months ago

in another branch, i should see to make sure that bb tests are run on Windows (and Mac) as well. by default we don't run tests there, as those OS on GitHub have issues with Docker (at least last time i tried)

Pgarrett commented 3 months ago

Windows docker images require them to be executed in a Windows host, so yep, that won't run in CI. Surprised by the fact that timeouts don't work in Windows, I'll dig into it (I don't have a Windows PC but I'll see what I can find)

Pgarrett commented 3 months ago

@arcuri82 I've been looking at the test results and it seems changes in this commit have broken the python implementation. The test written is now something like:

requests \
    .post(self.baseUrlOfSut + "/api/logintoken/login")
    headers = {}
    headers["content-type"] = "application/json"
    body = " { " + \
        " \"username\": \"foo\", " + \
        " \"password\": \"123\" " + \
        " } "
    cookies_foo = requests \, 
                headers=headers, data=body.cookies

Which will naturally break since there's indentation issues and misplaced calls. It should be something like:

headers = {}
headers["content-type"] = "application/json"
body = " { " + \
    " \"username\": \"foo\", " + \
    " \"password\": \"123\" " + \
    " } "
cookies_foo = requests \
    .post(self.baseUrlOfSut + "/api/logintoken/login", 
        headers=headers, data=body).cookies

So as to be in sync, do you want me to fix it? It goes beyond OS issues, it's python writing, so I feel I should fix it. Just let me know and I'll tackle it!

arcuri82 commented 3 months ago

@Pgarrett yes, please fix it, as it is related to python writing

Pgarrett commented 3 months ago

Fixed! @arcuri82