feat: allow dumping response buffer to some file

[eggachecat]

@bogdanfinn Hi this is a PR for enabling saving in-process bytes to given-path-file.

With this feature one can easily implement some callback functions from their app (with other languages they just need to watch the file).

May you help to review this ?

[eggachecat]

background: I am trying to use acheong08/ChatGPT which depends on tls-client. And UI/UX is unacceptably slow because of no-stream-response-support

[bogdanfinn]

Thank you for contribution! @eggachecat

[acheong08]

So it streams normally or does it require file?

[eggachecat]

It requires files. Clients need to poll to update I believe

[eggachecat]

def parse_file(file_name):
    try:
        with open(file_name, "rb") as fp:
            now = fp.read().decode()
            lines = [line for line in now.split('\n\n') if len(line) > 0]
            if len(lines) == 0:
                return None, False
            if len(lines) == 1:
                return lines[-1], False

            latest_resp_0 = lines[-1]
            latest_resp_1 = lines[-2]

            if "DONE" in latest_resp_0:
                print("detect done")
                return latest_resp_1, True

            latest_resp = latest_resp_0 if len(latest_resp_0) >= len(latest_resp_1) else latest_resp_1
            return latest_resp, False
    except Exception as e:
        print(traceback.format_exc())
        return None, False
    return None, False

def watch_resp_stream(file_name, callback, max_same_wait=10, sleep_ms=1):
    prev = None
    same_ctr = 0
    time_ctr = 0
    while same_ctr < max_same_wait or time_ctr > 300:
        latest, done = parse_file(file_name)
        if latest is not None:
            if done:
                callback(latest, True)
                break
            if latest == prev:
                same_ctr += 1
            else:
                callback(latest)
            prev = latest
        time.sleep(sleep_ms)
        time_ctr += 1
         def make_callback(scope):
            batch_text_size = 100
            lock = threading.Lock()
            has_been_last = False

            def _f(raw_text, is_last=False):
                print('processing with', raw_text)
                nonlocal scope, lock, has_been_last
                while lock.locked():
                    print('waiting lock...')
                    time.sleep(1)
                lock.acquire()
                print("scope['n']", scope['n'])
                try:
                    if "data:" in raw_text:
                        text = json.loads(raw_text.split("data:")[1])['message']['content']['parts'][0]
                    else:
                        text = raw_text

                    print("len(text)", len(text))
                    print("is_last", is_last)
                    print("has_been_last", has_been_last)
                    if len(text) > scope['n'] + batch_text_size or (is_last and not has_been_last):
                        msg = text[scope['n']:]
                        if scope['n'] != 0:
                            msg = "..." + msg
                        if not is_last:
                            msg += "..."
                        if is_last:
                            has_been_last = True
                        self.wb.send_txt_msg(to=to, content=msg)
                        scope['n'] = len(text)
                except Exception as e:
                    print(traceback.format_exc())
                lock.release()

            return _f

scope = {
     'n': 0,
}

 stream_callback  = make_callback(scope)
 t = threading.Thread(target=watch_resp_stream, args=(file_name, stream_callback, 5,))
 t.start()

Here's code for ChatGPT I am using.

[acheong08]

My solution was to write a proxy in Go and use that via normal requests from Python

[bogdanfinn]

My solution was to write a proxy in Go and use that via normal requests from Python

@acheong08 Check out this repository maybe: https://github.com/bogdanfinn/tls-client-api

[GunGunGun]

Could I get an example of how to use this feature, in my case I want to use Python to read reponse from tls-client by chunks, for example 8192 bytes per while loop.