TB

MoppleIT Tech Blog

Welcome to my personal blog where I share thoughts, ideas, and experiences.

🌐 Reuse HttpClient for Faster, Safer Web Calls in PowerShell

Published on February 10, 2026 | Category: DevOps | Tags: powershell, httpclient, apis, performance, scripting

You can dramatically speed up API scripts and eliminate socket exhaustion by reusing a single HttpClient instance in PowerShell. Instead of creating and disposing a client for every call, build one client, set sane defaults (timeouts, headers, decompression), send requests with cancellation, check status codes, and only dispose the client when your script or module exits. This pattern improves performance, reduces errors under load, and gives you clearer, more actionable failures.

Build One Reusable HttpClient

The biggest performance and reliability win comes from reusing a single HttpClient across all calls. Per-request clients create and tear down sockets that linger in TIME_WAIT, quickly exhausting ephemeral ports. A single client maintains a connection pool so you get connection reuse, fewer TCP handshakes, lower latency, and fewer transient errors.

Minimal pattern

# Reuse a single HttpClient across calls
if (-not $script:Client) {
  $handler = [System.Net.Http.HttpClientHandler]::new()
  $script:Client = [System.Net.Http.HttpClient]::new($handler)
  $script:Client.Timeout = [TimeSpan]::FromSeconds(10)
  $script:Client.DefaultRequestHeaders.Accept.Clear()
  $null = $script:Client.DefaultRequestHeaders.Accept.Add(
    [System.Net.Http.Headers.MediaTypeWithQualityHeaderValue]::new('application/json')
  )
}

That already gets you most of the benefits. For production-grade control, prefer SocketsHttpHandler (PowerShell 7+ on .NET) so you can tune connection lifetimes, decompression, and concurrency.

Production-ready client with pooling and HTTP/2

# Create once per process (script/module scope)
if (-not $script:Client) {
  $handler = [System.Net.Http.SocketsHttpHandler]::new()
  $handler.AutomaticDecompression = [System.Net.DecompressionMethods]::GZip -bor [System.Net.DecompressionMethods]::Deflate
  $handler.PooledConnectionLifetime = [TimeSpan]::FromMinutes(5)   # Refresh TCP/DNS periodically
  $handler.MaxConnectionsPerServer = 50                            # Concurrency limit per host
  $handler.ConnectTimeout = [TimeSpan]::FromSeconds(5)
  $handler.AllowAutoRedirect = $true

  $script:Client = [System.Net.Http.HttpClient]::new($handler)
  $script:Client.Timeout = [TimeSpan]::FromSeconds(15)              # Overall request ceiling
  $script:Client.DefaultRequestVersion = [System.Net.HttpVersion]::Version20
  $script:Client.DefaultVersionPolicy = [System.Net.Http.HttpVersionPolicy]::RequestVersionOrHigher

  # Default headers (per-request headers can override/append)
  $script:Client.DefaultRequestHeaders.Accept.Clear()
  $null = $script:Client.DefaultRequestHeaders.Accept.Add(
    [System.Net.Http.Headers.MediaTypeWithQualityHeaderValue]::new('application/json')
  )
  $null = $script:Client.DefaultRequestHeaders.UserAgent.ParseAdd('PwshHttpClient/1.0 (+https://example.com)')
}

Notes:

  • PooledConnectionLifetime: periodically recycles connections to avoid stale DNS and long-lived, potentially unhealthy connections.
  • AutomaticDecompression: enables gzip/deflate so you download fewer bytes.
  • MaxConnectionsPerServer: caps parallel connections per host; pair this with your scripts concurrency.
  • HTTP/2: RequestVersionOrHigher lets you use HTTP/2 when available for multiplexing and better throughput.

Resilient Request Pattern: Cancellation, Status Checks, and Parsing

Every request should: build a HttpRequestMessage, optionally add per-request headers (like Authorization), send with a CancellationTokenSource, check status codes, parse the body, and dispose the message objects. Do not dispose the HttpClient per call.

GET with cancellation and status checks

# One request with cancellation and status checks
$cts = [System.Threading.CancellationTokenSource]::new(10000)  # 10s
$request = [System.Net.Http.HttpRequestMessage]::new('GET', 'https://api.example.com/items?limit=3')

# Optional: per-request auth header
if ($env:API_TOKEN) {
  $request.Headers.Authorization = 
    [System.Net.Http.Headers.AuthenticationHeaderValue]::new('Bearer', $env:API_TOKEN)
}

try {
  $response = $script:Client.SendAsync(
    $request,
    [System.Net.Http.HttpCompletionOption]::ResponseHeadersRead, # start processing early
    $cts.Token
  ).GetAwaiter().GetResult()

  if (-not $response.IsSuccessStatusCode) {
    throw ('HTTP {0} {1}' -f [int]$response.StatusCode, $response.ReasonPhrase)
  }

  $body = $response.Content.ReadAsStringAsync().GetAwaiter().GetResult()
  $data = $body | ConvertFrom-Json -Depth 10
  $data | Select-Object -First 3 Name, Id
}
finally {
  if ($null -ne $response) { $response.Dispose() }
  $request.Dispose()
  $cts.Dispose()
}

Why this works well:

  • Cancellation: you control how long the request may run; this avoids hangs and frees connections quickly.
  • Status enforcement: throw on non-2xx and surface a clear error with HTTP code and reason.
  • Disposal: free the HttpRequestMessage/HttpResponseMessage objects so the connection returns to the pool.

POST JSON helper with retries (backoff on 429/5xx)

Transient errors happen. Add a tiny retry helper that backs off on common retryable statuses.

function Invoke-Json {
  param(
    [ValidateSet('GET','POST','PUT','PATCH','DELETE')] [string]$Method,
    [Parameter(Mandatory)] [string]$Uri,
    [hashtable]$Headers,
    $Body,
    [int]$TimeoutMs = 10000,
    [int]$Retries = 2
  )

  $attempt = 0
  do {
    $attempt++
    $cts = [System.Threading.CancellationTokenSource]::new($TimeoutMs)
    $request = [System.Net.Http.HttpRequestMessage]::new($Method, $Uri)

    try {
      if ($Headers) {
        foreach ($k in $Headers.Keys) {
          $null = $request.Headers.TryAddWithoutValidation($k, [string]$Headers[$k])
        }
      }

      if ($PSBoundParameters.ContainsKey('Body')) {
        $json = $Body | ConvertTo-Json -Depth 20
        $request.Content = [System.Net.Http.StringContent]::new($json, [System.Text.Encoding]::UTF8, 'application/json')
      }

      $response = $script:Client.SendAsync($request, $cts.Token).GetAwaiter().GetResult()
      if ($response.IsSuccessStatusCode) {
        $text = $response.Content.ReadAsStringAsync().GetAwaiter().GetResult()
        if (-not [string]::IsNullOrWhiteSpace($text)) { return $text | ConvertFrom-Json -Depth 20 }
        return $null
      }

      $code = [int]$response.StatusCode
      if ($code -in 429,500,502,503,504 -and $attempt -le $Retries) {
        $delay = [math]::Min(16000, 500 * [math]::Pow(2, $attempt - 1))
        Start-Sleep -Milliseconds $delay
        continue
      }

      throw "HTTP $code $($response.ReasonPhrase)"
    }
    finally {
      if ($null -ne $response) { $response.Dispose() }
      $request.Dispose()
      $cts.Dispose()
    }
  } while ($attempt -le $Retries)
}

# Example usage
$resp = Invoke-Json -Method 'POST' -Uri 'https://api.example.com/items' -Body @{ name = 'demo' } -Headers @{ Authorization = "Bearer $env:API_TOKEN" }

Production Hardening and Lifecycle

Concurrency without chaos

  • Throttle parallelism: in PowerShell 7, use ForEach-Object -Parallel -ThrottleLimit to cap worker count.
  • MaxConnectionsPerServer: tune this on SocketsHttpHandler to match your concurrency and server quotas.
$uris = 1..100 | ForEach-Object { "https://api.example.com/items/$_" }
$results = $uris | ForEach-Object -Parallel {
  # $using:Client is accessible if defined at script scope before parallel block
  $req = [System.Net.Http.HttpRequestMessage]::new('GET', $_)
  try {
    $res = $using:Client.SendAsync($req).GetAwaiter().GetResult()
    if ($res.IsSuccessStatusCode) { return $res.Content.ReadAsStringAsync().GetAwaiter().GetResult() }
    throw "HTTP $([int]$res.StatusCode)"
  }
  finally { if ($null -ne $res) { $res.Dispose() }; $req.Dispose() }
} -ThrottleLimit 16

Tip: keep ThrottleLimit at or below MaxConnectionsPerServer unless you deliberately want queuing client-side.

DNS and long-lived processes

  • Rotate pooled connections with PooledConnectionLifetime so long-running jobs pick up DNS changes and close old sockets.
  • Short ConnectTimeout catches unreachable hosts early and fails fast.

Security best practices

  • Leave certificate validation on. Avoid ServerCertificateCustomValidationCallback = { $true } unless you pin a cert thumbprint for a known host.
  • Handle secrets safely: inject tokens via environment variables, Azure Key Vault, or SecretManagement; attach Authorization per request, not as a globally logged default header.
  • Avoid logging full bodies that may contain PII or secrets; log status code, URI path, and correlation IDs.

Dispose on exit, not per call

Shut down cleanly at process end so sockets close promptly without tearing down the pool between calls.

function Stop-HttpClient {
  if ($script:Client) {
    try { $script:Client.Dispose() } finally { Remove-Variable -Scope Script -Name Client -ErrorAction SilentlyContinue }
  }
}

try {
  # ... your script logic that uses $script:Client ...
}
finally {
  Stop-HttpClient
}

Clear, actionable errors

  • Throw messages that include HTTP status and reason phrase.
  • Optionally parse problem details (RFC 7807) when APIs return structured errors.
  • Surface timeouts distinctly so CI/CD logs show whether you hit a remote slowness or a local bug.

Real-world DevOps benefits

  • CI/CD speedups: fewer TCP handshakes per job, faster pipelines.
  • Stability at scale: no more socket exhaustion under parallel test runs.
  • Observable failures: consistent cancellation and error formatting reduce mean-time-to-diagnose.

What you get: faster calls, fewer sockets, stable timeouts, clearer errors.

Build practical networking habits in PowerShell. Further reading: PowerShell Advanced CookBook.

← All Posts Home →