TIL:
Link: https://github.com/simonw/til/blob/main/httpx/openai-log-requests-responses.md
📌 Logging OpenAI API requests and responses using HTTPX
My LLM tool has a feature where you can set a LLM_OPENAI_SHOW_RESPONSES environment variable to see full debug level details of any HTTP requests it makes to the OpenAI APIs.
This broke when I upgraded the OpenAI Python it depends on, because OpenAI switched from using requests to using HTTPX and my previous mechanism had hooked into.
I figured out how to do this against HTTPX, using the event hooks mechanism in that library.
Update: See the bottom of this for a better solution.
✏️ Passing a custom HTTPX client to OpenAI
Using the OpenAI library starts with a client object created like this:
import openai
client = openai.OpenAI(api_key="...")
# Then to run a prompt:
chat_completion = client.chat.completions.create(
messages=[{
"role": "user",
"content": "Say hello",
}],
model="gpt-3.5-turbo",
)
print(chat_completion.choices[0].message.content)
The document describes the parameter to that OpenAI() constructor, which can be used to pass in a custom HTTPX client object.
Here's a basic illustration of that, using event hooks:
import httpx
client = openai.OpenAI(
http_client=httpx.Client(
event_hooks={
"request": [print],
"response": [print]
}
)
)
This sets the print() function as a hook for both the request and the response. Running a prompt shows the following:
<Request('POST', 'https://api.openai.com/v1/chat/completions')>
<Response [200 OK]>
✏️ Showing detailed request and response data
I iterated my way to the following as a response hook. Note that you can access the original request using response.request so its possible to output details of both the request and the response in the same response hook:
import textwrap, json
def log_response(response: httpx.Response):
request = response.request
print(f"Request: {request.method} {request.url}")
print(" Headers:")
for key, value in request.headers.items():
if key.lower() == "authorization":
value = "[...]"
if key.lower() == "cookie":
value = value.split("=")[0] + "=..."
print(f" {key}: {value}")
print(" Body:")
try:
request_body = json.loads(request.content)
print(
textwrap.indent(
json.dumps(request_body, indent=2), " "
)
)
except json.JSONDecodeError:
print(textwrap.indent(request.content.decode(), " "))
print(f"Response: status_code={response.status_code}")
print(" Headers:")
for key, value in response.headers.items():
if key.lower() == "set-cookie":
value = value.split("=")[0] + "=..."
print(f" {key}: {value}")
client = openai.OpenAI(
http_client=httpx.Client(
event_hooks={
"response": [log_response]
}
)
)
Running a prompt through this client produces the following output:
Request: POST https://api.openai.com/v1/chat/completions
Headers:
host: api.openai.com
accept-encoding: gzip, deflate, br
connection: keep-alive
accept: application/json
content-type: application/json
user-agent: OpenAI/Python 1.0.0
x-stainless-lang: python
x-stainless-package-version: 1.0.0
x-stainless-os: MacOS
x-stainless-arch: arm64
x-
Link: https://github.com/simonw/til/blob/main/httpx/openai-log-requests-responses.md
📌 Logging OpenAI API requests and responses using HTTPX
My LLM tool has a feature where you can set a LLM_OPENAI_SHOW_RESPONSES environment variable to see full debug level details of any HTTP requests it makes to the OpenAI APIs.
This broke when I upgraded the OpenAI Python it depends on, because OpenAI switched from using requests to using HTTPX and my previous mechanism had hooked into.
I figured out how to do this against HTTPX, using the event hooks mechanism in that library.
Update: See the bottom of this for a better solution.
✏️ Passing a custom HTTPX client to OpenAI
Using the OpenAI library starts with a client object created like this:
import openai
client = openai.OpenAI(api_key="...")
# Then to run a prompt:
chat_completion = client.chat.completions.create(
messages=[{
"role": "user",
"content": "Say hello",
}],
model="gpt-3.5-turbo",
)
print(chat_completion.choices[0].message.content)
The document describes the parameter to that OpenAI() constructor, which can be used to pass in a custom HTTPX client object.
Here's a basic illustration of that, using event hooks:
import httpx
client = openai.OpenAI(
http_client=httpx.Client(
event_hooks={
"request": [print],
"response": [print]
}
)
)
This sets the print() function as a hook for both the request and the response. Running a prompt shows the following:
<Request('POST', 'https://api.openai.com/v1/chat/completions')>
<Response [200 OK]>
✏️ Showing detailed request and response data
I iterated my way to the following as a response hook. Note that you can access the original request using response.request so its possible to output details of both the request and the response in the same response hook:
import textwrap, json
def log_response(response: httpx.Response):
request = response.request
print(f"Request: {request.method} {request.url}")
print(" Headers:")
for key, value in request.headers.items():
if key.lower() == "authorization":
value = "[...]"
if key.lower() == "cookie":
value = value.split("=")[0] + "=..."
print(f" {key}: {value}")
print(" Body:")
try:
request_body = json.loads(request.content)
print(
textwrap.indent(
json.dumps(request_body, indent=2), " "
)
)
except json.JSONDecodeError:
print(textwrap.indent(request.content.decode(), " "))
print(f"Response: status_code={response.status_code}")
print(" Headers:")
for key, value in response.headers.items():
if key.lower() == "set-cookie":
value = value.split("=")[0] + "=..."
print(f" {key}: {value}")
client = openai.OpenAI(
http_client=httpx.Client(
event_hooks={
"response": [log_response]
}
)
)
Running a prompt through this client produces the following output:
Request: POST https://api.openai.com/v1/chat/completions
Headers:
host: api.openai.com
accept-encoding: gzip, deflate, br
connection: keep-alive
accept: application/json
content-type: application/json
user-agent: OpenAI/Python 1.0.0
x-stainless-lang: python
x-stainless-package-version: 1.0.0
x-stainless-os: MacOS
x-stainless-arch: arm64
x-