http_client

On this page:

spider.http_client: The HTTP client: get, post, put, patch, delete, head, or a Client that keeps connections.
spider.http_client_mtls: HTTP requests that present a client certificate (mutual TLS).

`spider.http_client`

The HTTP client (spider.http_client): one function per method for a single request, and Client for many requests to the same server over kept-alive connections. The whole response body is read into memory; redirects are not followed.

`spider.http_client.Body`

zig

pub const Body = union(enum)

What a request sends. Each kind sets a Content-Type unless FetchOptions.headers (or, for a Client, its own headers) already has one. Only POST, PUT and PATCH take a body (see FetchOptions.body).

Fields

raw: []const u8: Bytes sent as they are. Content-Type: application/octet-stream.
json: []const u8: JSON text, already serialized (it is not checked). Content-Type: application/json.
form: []const [2][]const u8: Name/value pairs, sent URL-encoded (a space goes as %20, not +). Content-Type: application/x-www-form-urlencoded.

`spider.http_client.jsonBody`

zig

pub fn jsonBody(serialized: []const u8) Body

.{ .json = serialized }: a body from JSON text that is already serialized.

`spider.http_client.Headers`

zig

pub const Headers = struct

The headers a server answered with. Names and values live in the response: they are gone after Response.deinit().

Fields

items: []http.Header: Every header, in the order received; a name sent twice is here twice. A header with an empty value is left out.

`http_client.Headers.get`

zig

pub fn get(self: Headers, name: []const u8) ?[]const u8

The value of the first header called name (case does not matter), or null.

`spider.http_client.Response`

zig

pub const Response = struct

An HTTP answer, read to the end. It owns everything it points to (headers, body, what json parses): call deinit() once, and copy what must outlive it.

Fields

status: http.Status: The status code: res.status == .ok, @intFromEnum(res.status).
headers: Headers: The response headers: res.headers.get("content-type").
arena: *std.heap.ArenaAllocator
body_text: []const u8: The body, decompressed. Same as text().
http_client: *HttpClient
owns_http_client: bool

`http_client.Response.deinit`

zig

pub fn deinit(self: *Response) void

Frees the response. After a single request (get, post, ...) this also closes its connection; a Client's connections stay open.

`http_client.Response.text`

zig

pub fn text(self: *Response) []const u8

The body as bytes. Valid until deinit().

`http_client.Response.json`

zig

pub fn json(self: *Response, comptime T: type) !std.json.Parsed(T)

The body parsed as JSON into T; fields of the body that T does not have are ignored. Fails with the JSON parser's error when the body is not JSON or does not fit T. The result lives in the response's memory: it is valid until deinit() of the response, and its own deinit() is optional.

zig

const parsed = try res.json(struct { access_token: []const u8 = "" });
const token = parsed.value.access_token;

`spider.http_client.Client`

zig

pub const Client = struct

A client bound to one base URL, with a pool of kept-alive connections. Create it once and share it; requests may run through it concurrently.

A GET, HEAD, PUT or DELETE that fails on a connection taken from the pool, before any response arrived, is sent again on a new connection (up to twice): servers close idle connections without notice. A POST or a PATCH is never sent again: the call returns the error. There is no head method: use the standalone head.

zig

var client = try spider.http_client.Client.init(io, allocator, .{
    .base_url = "https://api.example.com",
    .headers = &.{.{ .name = "Authorization", .value = bearer }},
});
defer client.deinit();

var res = try client.get("/users", .{ .timeout_ms = 3000 });
defer res.deinit();

Fields

io: Io
allocator: std.mem.Allocator
base_url: []const u8: The base_url given to init.
headers: []const std.http.Header: The headers given to init.
proxy_url: ?[]const u8: The proxy_url given to init.
http_client: HttpClient: Persistent — created once here, reused (with its connection pool) across every .get()/.post()/etc made through this Client. Closed by Client.deinit(), not by individual Response.deinit() calls (see Response.owns_http_client).
proxy_arena: std.heap.ArenaAllocator

`http_client.Client.init`

zig

pub fn init(io: Io, allocator: std.mem.Allocator, opts: struct {
    /// Scheme and host, and a path prefix if any. Each request's `path`
    /// is appended to it as it is, so write one slash between them.
    base_url: []const u8,
    /// Sent with every request (an API key, a user agent). Default: none.
    headers: []const std.http.Header = &.{},
    /// The proxy for every request of this client, fixed here. null: the
    /// proxy environment variables apply (see `FetchOptions.proxy_url`).
    proxy_url: ?[]const u8 = null,
    /// A connection left unused for longer than this is closed rather
    /// than reused. Servers drop idle connections on their side
    /// (Cloudflare within minutes); 30 s stays under the usual limits.
    /// 0: no limit.
    idle_timeout_ms: u32 = 30_000,
    /// false: every request opens its own connection and closes it.
    /// Slower (a handshake per request), never meets a stale one.
    keep_alive: bool = true,
}) !Client

A client for opts.base_url. No connection is opened yet, and base_url is not checked: a bad one fails the first request. Fails when proxy_url is not a valid URL, or out of memory. The strings and the header list of opts are not copied: they must stay valid as long as the Client.

`http_client.Client.deinit`

zig

pub fn deinit(self: *Client) void

Closes the connection pool. Call once, when done with this Client and no request is running through it. Responses already returned stay valid: each is freed by its own deinit().

`http_client.Client.get`

zig

pub fn get(self: *Client, path: []const u8, opts: FetchOptions) !Response

A GET to base_url ++ path. Like the standalone get (same options, same errors), but the connection is reused, and the request carries the headers given to init followed by opts.headers (one of those with the name of a client header replaces it, for this request). opts.method is ignored. error.ProxyMismatch when opts.proxy_url is set to something other than the client's proxy. The response is allocated with the client's allocator: call its deinit().

`http_client.Client.post`

zig

pub fn post(self: *Client, path: []const u8, opts: FetchOptions) !Response

A POST to base_url ++ path with opts.body. See get.

`http_client.Client.put`

zig

pub fn put(self: *Client, path: []const u8, opts: FetchOptions) !Response

A PUT to base_url ++ path with opts.body. See get.

`http_client.Client.patch`

zig

pub fn patch(self: *Client, path: []const u8, opts: FetchOptions) !Response

A PATCH to base_url ++ path with opts.body. See get.

`http_client.Client.delete`

zig

pub fn delete(self: *Client, path: []const u8, opts: FetchOptions) !Response

A DELETE to base_url ++ path. See get.

`spider.http_client.FetchOptions`

zig

pub const FetchOptions = struct

The options of a request. Every field has a default: .{} is a plain request with no deadline.

Fields

method: http.Method = .GET: Set by post, put, patch, delete, head and by Client, whatever is given here. get and request send the method given (GET by default).
headers: []const http.Header = &.{}: Request headers: &.{.{ .name = "Authorization", .value = token }}. A Client sends the headers given to Client.init and then these; one of these with the name of a client header (case does not matter) replaces it for this request.
body: ?Body = null: The request body. null: none (POST, PUT and PATCH then send an empty one, Content-Length: 0). Only for POST, PUT and PATCH: with any other method (GET, DELETE, HEAD) a body is error.BodyNotAllowed and nothing is sent.
query: []const [2][]const u8 = &.{}: Name/value pairs added to the URL as its query string, names and values percent-encoded (after a ?, or a & when the URL already has a query). Added after params were filled in, so the two combine: /users/:id with both gives /users/42?page=2.
params: []const [2][]const u8 = &.{}: Values for :name placeholders in the URL: .{ "id", "42" } turns /users/:id into /users/42. The value is percent-encoded (all but letters, digits and -_.~), so it stays one path segment: "a/b" becomes a%2Fb. A placeholder is a : followed by a letter, then letters, digits and _; one without a value stays as written.
uri: ?std.Uri = null: A URI the caller already parsed, used instead of the URL: the url argument (for a Client, its base_url and the path), query and params are then all ignored.
timeout_ms: u32 = 0: Deadline for the WHOLE request, in milliseconds: connecting, the TLS handshake, sending, waiting for the response and reading its body. When it passes, the request is canceled, its connection is closed (never returned to a pool) and the call fails with error.Timeout. 0 = no deadline: the call waits for as long as the server takes. The request runs as a concurrent task of io: when io cannot start one, the call fails with error.ConcurrencyUnavailable.
max_response_bytes: usize = default_max_response_bytes: Largest response body accepted, in bytes, counted AFTER decompression (a small gzip body can inflate to a huge one). A larger body fails the call with error.ResponseTooLarge; nothing past the limit is kept. Raise it for a known-large download; use std.math.maxInt(usize) for no limit.
proxy_url: ?[]const u8 = null: The proxy for this request: an HTTP(S) proxy ("http://user:pass@host:8080") or a SOCKS5 one ("socks5://host:1080", "socks5h://..."; either way the proxy resolves the host name). It is used whatever no_proxy says. null: the environment decides (http_proxy / https_proxy / all_proxy, lower or upper case, unless no_proxy lists the host); with none of them set, no proxy. A URL with another scheme is ignored: the request goes direct. On a Client the proxy is fixed at init: a value here that differs from it fails with error.ProxyMismatch.

`spider.http_client.default_max_response_bytes`

zig

pub const default_max_response_bytes: usize = 64 * 1024 * 1024

Default for FetchOptions.max_response_bytes: generous enough for a file download, small enough that a misbehaving server can't exhaust the process's memory.

`spider.http_client.request`

zig

pub fn request(io: Io, allocator: std.mem.Allocator, url: []const u8, opts: FetchOptions, existing_client: ?*HttpClient) !Response

Sends one request with the method in opts.method: what get, post and Client call. Errors and the response are those of get.

existing_client null: the request gets a connection of its own, closed by Response.deinit(). Non-null: the http_client field of a Client (&client.http_client), whose pooled connections are used instead. This function never destroys it; it stays the Client's. The Client's base_url and headers are NOT applied here: url and opts.headers are sent as given.

`spider.http_client.asyncGet`

zig

pub fn asyncGet(client: *Client, path: []const u8, opts: FetchOptions) !Response

client.get(path, opts) as a plain function, to hand to io.async. It does not start anything by itself.

`spider.http_client.asyncPost`

zig

pub fn asyncPost(client: *Client, path: []const u8, opts: FetchOptions) !Response

client.post(path, opts) as a plain function, for io.async.

`spider.http_client.asyncPut`

zig

pub fn asyncPut(client: *Client, path: []const u8, opts: FetchOptions) !Response

client.put(path, opts) as a plain function, for io.async.

`spider.http_client.asyncPatch`

zig

pub fn asyncPatch(client: *Client, path: []const u8, opts: FetchOptions) !Response

client.patch(path, opts) as a plain function, for io.async.

`spider.http_client.asyncDelete`

zig

pub fn asyncDelete(client: *Client, path: []const u8, opts: FetchOptions) !Response

client.delete(path, opts) as a plain function, for io.async.

`spider.http_client.get`

zig

pub fn get(io: Io, allocator: std.mem.Allocator, url: []const u8, opts: FetchOptions) !Response

A GET request to url, on a connection of its own (opened for this call, closed by Response.deinit()). A 2xx, 4xx or 5xx answer comes back as a Response. Redirects are not followed, and a redirect is not returned either: a 3xx other than 304 fails the call with error.HttpRedirectLocationOversize (error.HttpRedirectLocationMissing when it has no Location); only head gets it as a Response. Other errors: error.Timeout (timeout_ms passed), error.ResponseTooLarge (max_response_bytes), error.HttpBodyCutShort (the body stopped early), or the error of the connection or of parsing url. The response is allocated from allocator and owns its memory: call deinit(). opts.method, when set, replaces GET.

`spider.http_client.post`

zig

pub fn post(io: Io, allocator: std.mem.Allocator, url: []const u8, opts: FetchOptions) !Response

A POST request to url with opts.body. See get.

`spider.http_client.put`

zig

pub fn put(io: Io, allocator: std.mem.Allocator, url: []const u8, opts: FetchOptions) !Response

A PUT request to url with opts.body. See get.

`spider.http_client.patch`

zig

pub fn patch(io: Io, allocator: std.mem.Allocator, url: []const u8, opts: FetchOptions) !Response

A PATCH request to url with opts.body. See get.

`spider.http_client.delete`

zig

pub fn delete(io: Io, allocator: std.mem.Allocator, url: []const u8, opts: FetchOptions) !Response

A DELETE request to url. See get.

`spider.http_client.head`

zig

pub fn head(io: Io, allocator: std.mem.Allocator, url: []const u8, opts: FetchOptions) !Response

A HEAD request to url: status and headers, an empty body. See get.

`spider.http_client_mtls`

HTTP client with mTLS (client-certificate) support, via shell-out to curl.

std.http.Client (and therefore pacman/spider.http_client, which wraps it directly) has no way to present a client certificate during the TLS handshake — confirmed by reading std.crypto.tls.Client.Options, whose only certificate-related fields are for verifying the server (.host, .ca). There is no client-side certificate/signing hook. This is a known stdlib gap (see ziglang/zig's open mTLS-for-clients issue), not a Spider limitation — so this module deliberately bypasses std.http.Client entirely and shells out to curl, which has supported --cert/--key (including PKCS#12) for years. Needed for integrations that require mTLS with an ICP-Brasil certificate (e.g. Sicoob's Open Finance API).

`spider.http_client_mtls.MtlsCert`

zig

pub const MtlsCert = union(enum)

A client certificate, in one of the two formats real-world mTLS deployments show up in. ICP-Brasil certificates are normally issued as PKCS#12 (.pfx/.p12); .pkcs12 accepts that directly (curl decrypts it itself), so callers don't need an extra manual conversion step to PEM.

Fields

pem: struct { ... }
pkcs12: struct { ... }

`spider.http_client_mtls.MtlsRequest`

zig

pub const MtlsRequest = struct

What to send: the options of request. Every field has a default, so .{} is a GET with no extra headers.

Fields

method: std.http.Method = .GET
headers: []const [2][]const u8 = &.{}: Request headers as name and value pairs, e.g. &.{.{ "Content-Type", "application/json" }}.
body: ?[]const u8 = null: Request body, sent as is. null: no body.
timeout_ms: u32 = 30_000: Longest the whole exchange may take; request then fails with error.MtlsTimeout or error.MtlsTransportFailed. Default 30 s.

`spider.http_client_mtls.MtlsResponse`

zig

pub const MtlsResponse = struct

The answer to request. A 4xx or 5xx status is a response like any other, not an error.

Fields

status: std.http.Status
body: []const u8: The response body, allocated with the allocator given to request.

`http_client_mtls.MtlsResponse.deinit`

zig

pub fn deinit(self: *MtlsResponse, allocator: std.mem.Allocator) void

Frees body. allocator is the one given to request.

`http_client_mtls.MtlsResponse.json`

zig

pub fn json(self: MtlsResponse, comptime T: type, allocator: std.mem.Allocator) !std.json.Parsed(T)

Parses body as JSON into T; fields of the JSON that T does not have are ignored. The caller calls deinit() on the result.

`spider.http_client_mtls.request`

zig

pub fn request(
    gpa: std.mem.Allocator,
    url: []const u8,
    cert: MtlsCert,
    opts: MtlsRequest,
) !MtlsResponse

Makes one HTTP request over an mTLS connection by running curl, and waits for it: the calling thread is blocked until curl ends. It takes no io: it uses a blocking Io of its own, whatever the server's backend. Redirects are not followed.

The certificate, the key and the request body are written to files in /tmp (mode 0600) for the time of the call and deleted before it returns.

The body of the response is allocated with gpa: free it with res.deinit(gpa). Needs curl on the PATH (error.FileNotFound without it). Fails with error.MtlsTimeout when curl does not finish within opts.timeout_ms, and with error.MtlsTransportFailed when the connection or the TLS handshake fails (refused, DNS, certificate rejected). A response body over 64 MiB is an error.

zig

var res = try spider.http_client_mtls.request(gpa, url, cert, .{
    .method = .POST,
    .headers = &.{.{ "Content-Type", "application/json" }},
    .body = body,
});
defer res.deinit(gpa);