TIL that the Python requests library is able to dump request/response details to the debug log - quite useful for development and debugging of http clients.
Switch it on with something like this:
import requests
import logging
try: # for Python 3
from http.client import HTTPConnection
except ImportError:
from httplib import HTTPConnection
HTTPConnection.debuglevel = 1
logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
req_log = logging.getLogger('requests.packages.urllib3')
req_log.setLevel(logging.DEBUG)
req_log.propagate = True
This will log requests (including headers and body if any) and the response to the console. It's not complete, but a useful quick overview of the conversation.
When coding on a networked client usually tcpdump and/or wireshark are my tools of choice to inspect what gets sent over the wire. Tcpdump and wireshark are great, but for just getting the gist of a http conversation they're a bit unwieldy. Feels like unpacking a full complement of precision tools to just hammer in a nail - you'd surely get that nail in, but there are quicker ways.