HAR converter batches dependent requests

[thorstenkampe]

I recorded a web browser session to http://www.microsoft.com, saved the session as HAR file and converted the file to JavaScript for replaying. A look into the generated code shows that the recording and the conversion worked well (see below): I open http://www.microsoft.com/ and I'm redirected to https://www.microsoft.com/de-de/

But the HTTP debug output shows the requests out of order - first "GET /de-de/", then "GET /". Obviously that would make any real debugging via `http-debug impossible.

          /\      |‾‾|  /‾‾/  /‾/
     /\  /  \     |  |_/  /  / /
    /  \/    \    |      |  /  ‾‾\
   /          \   |  |‾\  \ | (_) |
  / __________ \  |__|  \__\ \___/ .io

  execution: local-
     output: -
     script: .\session.js

    duration: -,  iterations: 1
         vus: 1, max: 1

Request: [----------------------------------------------------------] starting
GET /de-de/ HTTP/1.1
Host: www.microsoft.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:64.0) Gecko/20100101 Firefox/64.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: en
Connection: keep-alive
Upgrade-Insecure-Requests: 1

Request:
GET / HTTP/1.1
Host: www.microsoft.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:64.0) Gecko/20100101 Firefox/64.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: en
Connection: keep-alive
Upgrade-Insecure-Requests: 1

RedirectResponse:
HTTP/1.1 302 Moved Temporarily
Connection: keep-alive
Date: Sun, 14 Oct 2018 08:50:14 GMT
Location: https://www.microsoft.com/de-de/
X-Rtag: ARRPrd
Content-Length: 0

import { group, sleep } from 'k6';
import http from 'k6/http';

// Version: 1.1
// Creator: Firefox
// Browser: Firefox

export let options = {
    maxRedirects: 0,
};

export default function() {

    group("page_1 - Neuer Tab", function() {
        let req, res;
        req = [{
            "method": "get",
            "url": "http://www.microsoft.com/",
            "params": {
                "headers": {
                    "Host": "www.microsoft.com",
                    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:64.0) Gecko/20100101 Firefox/64.0",
                    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
                    "Accept-Language": "en",
                    "Accept-Encoding": "gzip, deflate",
                    "Connection": "keep-alive",
                    "Upgrade-Insecure-Requests": "1"
                }
            }
        },{
            "method": "get",
            "url": "https://www.microsoft.com/de-de/",
            "params": {
                "headers": {
                    "Host": "www.microsoft.com",
                    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:64.0) Gecko/20100101 Firefox/64.0",
                    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
                    "Accept-Language": "en",
                    "Accept-Encoding": "gzip, deflate, br",
                    "Connection": "keep-alive",
                    "Upgrade-Insecure-Requests": "1"
                }
            }
        }];

[na--]

Ah, I'd say that the generated script from the HAR file is wrong in this case, not the k6 execution or --http-debug. Looking at the script snippet in your message, I assume that an http.batch() call is present immediately after the part you posted, which will execute the 2 requests in req simultaneously.

So it's somewhat normal for the second request in a batch call to be executed slightly ahead of the first one, that's just due to the concurrent nature of the execution. The bug is in the k6 converter that didn't separate the two...

[na--]

You can manually edit the script so that the first request is a separate http.get() or you can use the --no-batch option for k6 convert (though it would un-batch all requests in the script, which is also an issue), but long-term we should definitely make the k6 HAR converter smarter...It should be possible to detect which requests are separate and dependent on one another (e.g. loading one page redirects you to another) and which are simultaneous (e.g. loading all static resources from a single page).

Connected issue: https://github.com/loadimpact/k6/issues/762

[codebien]

The k6 convert command has been deprecated and will be removed in the future https://github.com/grafana/k6/pull/3365.

Use https://github.com/grafana/har-to-k6 as a better alternative. The linked issue already tracks the improvements required to address this issue. Feel free to open a new issue there if you want to provide more details or specific proposals.