ヒトリ歩き

愚痴とかいろいろ書きます

HTTPクライアントはrequestsだけじゃない!! httpxという選択肢

非同期のHTTPクライアントは、aiohttpがメジャーかなと思っていたが、httpxを使用している人が社内に いたのでhttpxに触れてみることにする。

httpxとは

HTTPXとは、Python3向けのHTTPクライアント。同期、非同期に対応しており、HTTP/1.1とHTTP/2をサポートしている。

www.python-httpx.org

セットアップ

httpxライブラリをインストールするだけ。

pip install httpx

pip install 'httpx[cli]' でコマンドラインで実行できる。

使ってみる

実際にGET/POSTをやってみる。

GET/POST

GETリクエスト

GETリクエストは以下のような実装で実行できる。 requestsモジュールとほぼ同様の記述。

def get_request():

    r = httpx.get("http://localhost:8000/item")
    if r.status_code == 200:
        print("Operate Request Success")
        r_json = r.json()
        print(f"Receive body: {r_json}")

出力結果

DEBUG [2024-07-02 07:46:55] httpcore.http11 - send_request_body.started request=<Request [b'GET']>
DEBUG [2024-07-02 07:46:55] httpcore.http11 - send_request_body.complete
DEBUG [2024-07-02 07:46:55] httpcore.http11 - receive_response_headers.started request=<Request [b'GET']>
DEBUG [2024-07-02 07:46:55] httpcore.http11 - receive_response_headers.complete return_value=(b'HTTP/1.1', 200, b'OK', [(b'date', b'Mon, 01 Jul 2024 22:46:54 GMT'), (b'server', b'uvicorn'), (b'content-length', b'15'), (b'content-type', b'application/json')])
INFO [2024-07-02 07:46:55] httpx - HTTP Request: GET http://localhost:8000/item "HTTP/1.1 200 OK"
DEBUG [2024-07-02 07:46:55] httpcore.http11 - receive_response_body.started request=<Request [b'GET']>
DEBUG [2024-07-02 07:46:55] httpcore.http11 - receive_response_body.complete
DEBUG [2024-07-02 07:46:55] httpcore.http11 - response_closed.started
DEBUG [2024-07-02 07:46:55] httpcore.http11 - response_closed.complete
DEBUG [2024-07-02 07:46:55] httpcore.connection - close.started
DEBUG [2024-07-02 07:46:55] httpcore.connection - close.complete
Operate Request Success
Receive body: {'status': 'OK'}

Postリクエスト

POSTリクエストは以下のような記述で可能。こちらもrequestsモジュールとほぼ同じような感じ。

def post_request():
    body = {"mode": "post", "data": "aaa"}
    r = httpx.post("http://localhost:8000/item", json=body)
    print(r)
    if r.status_code == 200:
        print("Post Request Success")
        print(f"Response body: {r.json()}")

結果

# クライアント側
DEBUG [2024-07-02 08:01:26] httpcore.http11 - send_request_headers.started request=<Request [b'POST']>
DEBUG [2024-07-02 08:01:26] httpcore.http11 - send_request_headers.complete
DEBUG [2024-07-02 08:01:26] httpcore.http11 - send_request_body.started request=<Request [b'POST']>
DEBUG [2024-07-02 08:01:26] httpcore.http11 - send_request_body.complete
DEBUG [2024-07-02 08:01:26] httpcore.http11 - receive_response_headers.started request=<Request [b'POST']>
DEBUG [2024-07-02 08:01:26] httpcore.http11 - receive_response_headers.complete return_value=(b'HTTP/1.1', 200, b'OK', [(b'date', b'Mon, 01 Jul 2024 23:01:26 GMT'), (b'server', b'uvicorn'), (b'content-length', b'15'), (b'content-type', b'application/json')])
INFO [2024-07-02 08:01:26] httpx - HTTP Request: POST http://localhost:8000/item "HTTP/1.1 200 OK"
DEBUG [2024-07-02 08:01:26] httpcore.http11 - receive_response_body.started request=<Request [b'POST']>
DEBUG [2024-07-02 08:01:26] httpcore.http11 - receive_response_body.complete
DEBUG [2024-07-02 08:01:26] httpcore.http11 - response_closed.started
DEBUG [2024-07-02 08:01:26] httpcore.http11 - response_closed.complete
DEBUG [2024-07-02 08:01:26] httpcore.connection - close.started
DEBUG [2024-07-02 08:01:26] httpcore.connection - close.complete
<Response [200 OK]>
Post Request Success
Response body: {'status': 'OK'}

# サーバー側
INFO:     Receive POST Request.
INFO:     body : {'mode': 'post', 'data': 'aaa'}
INFO:     127.0.0.1:55026 - "POST /item HTTP/1.1" 200 OK

Async Support

httpxは、同期リクエストだけではなく、非同期リクエストもサポートしている。 requestsモジュールは同期のみのため、両方をサポートしていると別のモジュールを入れる必要がない。

www.python-httpx.org

GET

GETモジュールは以下のように記述できる。 aiohttpだとレスポンスの情報を取得するときもawaitが必要だが、httpxでは不要。

async def async_get_request():

    async with httpx.AsyncClient() as client:
        r = await client.get("http://localhost:8000/item")

        if r.status_code == 200:
            print(f"Recevice body: {r.json()}")

DEBUG [2024-07-03 06:52:02] asyncio - Using selector: KqueueSelector
DEBUG [2024-07-03 06:52:02] httpx - load_ssl_context verify=True cert=None trust_env=True http2=False
DEBUG [2024-07-03 06:52:02] httpx - load_verify_locations cafile='/Users/kotaro/.pyenv/versions/3.12.1/lib/python3.12/site-packages/certifi/cacert.pem'
DEBUG [2024-07-03 06:52:02] httpcore.connection - connect_tcp.started host='localhost' port=8000 local_address=None timeout=5.0 socket_options=None
DEBUG [2024-07-03 06:52:02] httpcore.connection - connect_tcp.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0x1057f59a0>
DEBUG [2024-07-03 06:52:02] httpcore.http11 - send_request_headers.started request=<Request [b'GET']>
DEBUG [2024-07-03 06:52:02] httpcore.http11 - send_request_headers.complete
DEBUG [2024-07-03 06:52:02] httpcore.http11 - send_request_body.started request=<Request [b'GET']>
DEBUG [2024-07-03 06:52:02] httpcore.http11 - send_request_body.complete
DEBUG [2024-07-03 06:52:02] httpcore.http11 - receive_response_headers.started request=<Request [b'GET']>
DEBUG [2024-07-03 06:52:02] httpcore.http11 - receive_response_headers.complete return_value=(b'HTTP/1.1', 200, b'OK', [(b'date', b'Tue, 02 Jul 2024 21:52:02 GMT'), (b'server', b'uvicorn'), (b'content-length', b'15'), (b'content-type', b'application/json')])
INFO [2024-07-03 06:52:02] httpx - HTTP Request: GET http://localhost:8000/item "HTTP/1.1 200 OK"
DEBUG [2024-07-03 06:52:02] httpcore.http11 - receive_response_body.started request=<Request [b'GET']>
DEBUG [2024-07-03 06:52:02] httpcore.http11 - receive_response_body.complete
DEBUG [2024-07-03 06:52:02] httpcore.http11 - response_closed.started
DEBUG [2024-07-03 06:52:02] httpcore.http11 - response_closed.complete

Recevice body: {'status': 'OK'}

POST

POSTリクエストの場合は以下のように記述する。

async def async_post_request():

    async with httpx.AsyncClient() as client:
        body = {"mode": "async", "key": "hogehoge"}
        r = await client.post("http://localhost:8000/item", json=body)
        if r.status_code == 200:
            print(f"Recevice body: {r.json()}")

DEBUG [2024-07-03 06:56:57] asyncio - Using selector: KqueueSelector
DEBUG [2024-07-03 06:56:57] httpx - load_ssl_context verify=True cert=None trust_env=True http2=False
DEBUG [2024-07-03 06:56:57] httpx - load_verify_locations cafile='/Users/kotaro/.pyenv/versions/3.12.1/lib/python3.12/site-packages/certifi/cacert.pem'
DEBUG [2024-07-03 06:56:57] httpcore.connection - connect_tcp.started host='localhost' port=8000 local_address=None timeout=5.0 socket_options=None
DEBUG [2024-07-03 06:56:57] httpcore.connection - connect_tcp.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0x109a8a2a0>
DEBUG [2024-07-03 06:56:57] httpcore.http11 - send_request_headers.started request=<Request [b'POST']>
DEBUG [2024-07-03 06:56:57] httpcore.http11 - send_request_headers.complete
DEBUG [2024-07-03 06:56:57] httpcore.http11 - send_request_body.started request=<Request [b'POST']>
DEBUG [2024-07-03 06:56:57] httpcore.http11 - send_request_body.complete
DEBUG [2024-07-03 06:56:57] httpcore.http11 - receive_response_headers.started request=<Request [b'POST']>
DEBUG [2024-07-03 06:56:57] httpcore.http11 - receive_response_headers.complete return_value=(b'HTTP/1.1', 200, b'OK', [(b'date', b'Tue, 02 Jul 2024 21:56:56 GMT'), (b'server', b'uvicorn'), (b'content-length', b'15'), (b'content-type', b'application/json')])
INFO [2024-07-03 06:56:57] httpx - HTTP Request: POST http://localhost:8000/item "HTTP/1.1 200 OK"
DEBUG [2024-07-03 06:56:57] httpcore.http11 - receive_response_body.started request=<Request [b'POST']>
DEBUG [2024-07-03 06:56:57] httpcore.http11 - receive_response_body.complete
DEBUG [2024-07-03 06:56:57] httpcore.http11 - response_closed.started
DEBUG [2024-07-03 06:56:57] httpcore.http11 - response_closed.complete
Recevice body: {'status': 'OK'}
DEBUG [2024-07-03 06:56:57] httpcore.connection - close.started
DEBUG [2024-07-03 06:56:57] httpcore.connection - close.complete

HTTP/2 Support

HTTP/2もサポートしている。

www.python-httpx.org

raise_for_status

気になったのがHTTPステータスコードが2XXではない場合、例外を挙げてくれるメソッドがあるということ。

raise_for_statusメソッドを使うと、ステータスコードが2XX以外の場合は、HTTPStatusErrorを あげてくれる。わざわざステータスコードの判定分岐が不要になるので便利。

www.python-httpx.org

def get_request_except():

    r = httpx.get("http://localhost:8000/item2")
    try:
        r.raise_for_status()
        print("Operate Request Success")
        r_json = r.json()
        print(f"Receive body: {r_json}")
    except httpx.HTTPStatusError as ex:
        print(f"Error Response Status Code: {ex.response.status_code}")

後から確認してみるとrequestsモジュールにはないと思っていたら、同じものがあった。

docs.python-requests.org

Event Hooks

www.python-httpx.org

特定のイベント時に指定した関数を呼び出す機能がある。

  • request リクエストの送信の準備が完了し、レクエストを送信する前に指定した関数を呼び出す。 requestインスタンスが渡される。

  • response レスポンスを受信し、リクエストの送信元に応答を返す前に呼び出す responseインスタンスが渡される。

EventHookを使用する場合は、httpx.Clientでclientインスタンスの生成が必要。 インスタンス生成時にevent_hooksパラメータに指定関数を渡す。

def log_request(request):
    print(f"Request Event Hook request={request}")


def log_response(response):
    print(f"Response Event Hook response={response}")


def get_request_log():

    client = httpx.Client(
        event_hooks={
            "request": [log_request],
            "response": [log_response],
        }
    )
    r = client.get("http://localhost:8000/item")
    if r.status_code == 200:
        print("Operate Request Success")
        r_json = r.json()
        print(f"Receive body: {r_json}")

Request Event Hook request=<Request('GET', 'http://localhost:8000/item')>
Response Event Hook response=<Response [200 OK]>
Operate Request Success
Receive body: {'status': 'OK'}

EventHooksは複数のメソッドを指定可能。 raise_for_statusメソッドもこのEventHooksに指定するメソッド内で呼べば、HttpStatusErrorをあげてくれる。

def log_request(request):
    print(f"Request Event Hook request={request}")


def log_response(response):
    print(f"Response Event Hook response={response}")

def log_response2(response):
    print(f"Status Code Check {response.status_code}")

def log_response3(response):
    print("last event hook function")


def get_request_log():

    client = httpx.Client(
        event_hooks={
            "request": [log_request],
            "response": [log_response, log_response2, log_response3],
        }
    )
    r = client.get("http://localhost:8000/item")
    if r.status_code == 200:
        print("Operate Request Success")
        r_json = r.json()
        print(f"Receive body: {r_json}")

Request Event Hook request=<Request('GET', 'http://localhost:8000/item')>

Response Event Hook response=<Response [200 OK]>
Status Code Check 200
last event hook function

Operate Request Success
Receive body: {'status': 'OK'}

requestsモジュールにもEventHooksあった。

requests.readthedocs.io

最後に

httpxを使ってみて、使い方はaiohttpにもrequestsの両方に近い。　　 ただ、同期／非同期に対応しているので、複数のモジュールのインストールが不要なところはいい点だ。 機会があれば、httpxを使う方向で再度調査して使ってみようと思う。