Noticed while traveling to Mexico that while using the tool from some networks the response to the first prompt consistently took > 15 seconds, even though subsequent followups were still returning in ~1-3 seconds. My hunch is that something is slowing down the initial http client setup / first request, and then reusing the http client for followups makes it quicker. Ideally this tool should work quickly from any network.
Task
Investigate why first request is so slow, propose potential solutions, discuss and implement.
Some ideas:
Use a daemon to maximize http client reuse.
Cache some intermediate connection state on disk and load it at runtime(?)
Context
Noticed while traveling to Mexico that while using the tool from some networks the response to the first prompt consistently took > 15 seconds, even though subsequent followups were still returning in ~1-3 seconds. My hunch is that something is slowing down the initial http client setup / first request, and then reusing the http client for followups makes it quicker. Ideally this tool should work quickly from any network.
Task
Investigate why first request is so slow, propose potential solutions, discuss and implement.
Some ideas: