Event-Driven File Watching in PowerShell: Real-Time Processing with FileSystemWatcher

Published on October 20, 2025 | Category: Automation | Tags: PowerShell, FileSystemWatcher, Automation, Events, Scripting

If you still rely on polling loops to detect new files, youre leaving performance and reliability on the table. PowerShell can react to file changes instantly using .NETs System.IO.FileSystemWatcher and PowerShells Register-ObjectEvent, delivering near real-time automation without busy-waiting. In this guide, youll set up an event-driven watcher that filters by extension, safely reads files only after writes complete, and cleans up resources for a tidy exit.

Why go event-driven?

Instant feedback: React to new files as soon as they hit disk; no more sleep intervals.
Lower CPU and I/O: Eliminate tight polling loops that hammer disks and waste cycles.
Fewer race conditions: Watcher events align with file system changes; you can debounce and verify completion.
Cleaner lifecycle: Register/unregister events, dispose watchers, and exit gracefully.

A minimal watcher for new JSON files

The following script listens for newly-created .json files in C:\\drop, processes them on arrival, and exits cleanly on stop. It uses event-driven notifications instead of polling.

$path = 'C:\drop'
$filter = '*.json'

$w = New-Object System.IO.FileSystemWatcher -Property @{
  Path = $path
  Filter = $filter
  IncludeSubdirectories = $false
  EnableRaisingEvents = $true
}

$handler = {
  param($s, $e)
  if ($e.ChangeType -eq 'Created' -and (Test-Path -LiteralPath $e.FullPath)) {
    try {
      $obj = Get-Content -LiteralPath $e.FullPath -Raw | ConvertFrom-Json
      Write-Host ("Processed {0}" -f $e.Name)
    } catch {
      Write-Warning ("Error reading {0}: {1}" -f $e.FullPath, $_.Exception.Message)
    }
  }
}

$sub = Register-ObjectEvent -InputObject $w -EventName Created -Action $handler

try {
  while ($true) { Wait-Event -Timeout 1 | Out-Null }
} finally {
  Unregister-Event -SubscriptionId $sub.Id
  $w.Dispose()
  Get-Event | Remove-Event -ErrorAction SilentlyContinue
}

This works, but production workloads often need a bit more hardening. Writes may be partial, events can fire multiple times, and internal buffers can overflow if many files arrive at once. Lets make it robust.

Production-ready: safe reads, better filters, clean exits

1) Ensure complete writes before processing

A common pitfall is trying to read a file while another process is still writing it. Avoid partial reads by verifying two conditions: the file size is stable, and it can be opened exclusively for read (no writers).

function Wait-FileReady {
  param(
    [Parameter(Mandatory)] [string] $Path,
    [int] $TimeoutSec = 30,
    [int] $StableMillis = 250
  )
  $sw = [System.Diagnostics.Stopwatch]::StartNew()
  $lastLen = -1
  while ($sw.Elapsed.TotalSeconds -lt $TimeoutSec) {
    if (-not (Test-Path -LiteralPath $Path)) { Start-Sleep -Milliseconds 50; continue }
    try {
      $len1 = (Get-Item -LiteralPath $Path -ErrorAction Stop).Length
      Start-Sleep -Milliseconds $StableMillis
      $len2 = (Get-Item -LiteralPath $Path -ErrorAction Stop).Length
      if ($len1 -eq $len2) {
        $fs = [System.IO.File]::Open(
          $Path,
          [System.IO.FileMode]::Open,
          [System.IO.FileAccess]::Read,
          [System.IO.FileShare]::None
        )
        $fs.Dispose()
        return $true
      }
    } catch { }
    Start-Sleep -Milliseconds 100
  }
  return $false
}

This function waits for the file to stop changing and confirms no other process is writing by temporarily opening it with FileShare.None. If it times out, you can decide to skip, retry later, or move the file to a quarantine folder.

2) Hardened watcher with debouncing and multiple events

Depending on the application, some writers trigger Created, then one or more Changed events, and sometimes a Renamed. You can register handlers for all of them and debounce inside your action. Also tune NotifyFilter and internal buffer size to reduce dropped events.

$path = 'C:\drop'
$filter = '*.json'

# Main watcher
$w = [System.IO.FileSystemWatcher]::new($path, $filter)
$w.IncludeSubdirectories = $false
$w.NotifyFilter = [System.IO.NotifyFilters]'FileName, LastWrite, Size'
$w.InternalBufferSize = 32768  # bytes; larger buffer reduces risk of overflow during bursts
$w.EnableRaisingEvents = $true

# Optional: a small in-memory debounce cache by path
$debounce = [System.Collections.Concurrent.ConcurrentDictionary[string, datetime]]::new()
$debounceWindow = [TimeSpan]::FromMilliseconds(300)

$action = {
  param($sender, $eventArgs)
  try {
    $path = $eventArgs.FullPath
    if (-not (Test-Path -LiteralPath $path)) { return }
    if ($path -notlike '*.json') { return } # extra safety if writers bypass the filter

    # Debounce: skip if we processed the same path very recently
    $now = Get-Date
    $last = $using:debounce.AddOrUpdate($path, $now, { param($k,$v) $now })
    if (($now - $last) -lt $using:debounceWindow) { return }

    # Give writers a moment to finish very short bursts
    Start-Sleep -Milliseconds 100
    if (& $function:Wait-FileReady -Path $path -TimeoutSec 30) {
      try {
        $json = Get-Content -LiteralPath $path -Raw -ErrorAction Stop | ConvertFrom-Json -ErrorAction Stop
        # TODO: call your processor here (e.g., Invoke-MyIngest -InputObject $json)
        Write-Host ("Processed {0}" -f $eventArgs.Name)
      } catch {
        Write-Warning ("Failed processing {0}: {1}" -f $path, $_.Exception.Message)
      }
    } else {
      Write-Warning ("Timed out waiting for file to be ready: {0}" -f $path)
    }
  } catch {
    Write-Warning ("Unhandled watcher error: {0}" -f $_.Exception.Message)
  }
}

# Register for multiple events to cover different writers
$subs = @()
$subs += Register-ObjectEvent -InputObject $w -EventName Created -Action $action
$subs += Register-ObjectEvent -InputObject $w -EventName Changed -Action $action
$subs += Register-ObjectEvent -InputObject $w -EventName Renamed -Action $action

try {
  Write-Host "Watching $path for $filter (Ctrl+C to stop)"
  while ($true) {
    Wait-Event -Timeout 1 | Out-Null
  }
} finally {
  foreach ($s in $subs) { Unregister-Event -SubscriptionId $s.Id }
  $w.Dispose()
  Get-Event | Remove-Event -ErrorAction SilentlyContinue
  Write-Host 'Watcher disposed. Goodbye.'
}

3) Test it quickly

Use an atomic move to simulate an external writer: write to a temp file, then move into the drop folder to avoid partial writes.

$drop = 'C:\drop'
$tmp = Join-Path $env:TEMP ([System.IO.Path]::GetRandomFileName())
'{"id":1,"msg":"hello"}' | Set-Content -LiteralPath $tmp -Encoding UTF8
Move-Item -LiteralPath $tmp -Destination (Join-Path $drop 'test.json')

Practical tips and gotchas

Filtering and scope

Extensions: Use Filter = '*.json' and a second guard in your action for defense-in-depth.
Subdirectories: Set IncludeSubdirectories as needed; avoid watching huge trees unless necessary.
Notify filters: Commonly FileName, LastWrite, and Size are sufficient.

Handling load spikes

InternalBufferSize: Increase to 32 B or 64 B to reduce dropped events, especially when thousands of files arrive at once.
Back-pressure: If processing is heavy, queue work items. For example, add file paths to a BlockingCollection and have worker runspaces drain the queue.
Duplicates: Expect multiple Changed events; debounce as shown or keep a short-lived set of processed paths and timestamps.

Safer processing

Avoid partial reads: Always verify readiness (stable size + exclusive open) before reading.
Validate input: For JSON, catch ConvertFrom-Json errors and validate schema before acting.
Least privilege: Run the watcher account with minimal access to only the required folders.
Quarantine on error: Move invalid or unreadable files to a _quarantine directory for later inspection.

Lifecycle and cleanup

Unregister and dispose: Always Unregister-Event, Dispose() the watcher, and clear pending events to prevent memory leaks.
Graceful shutdown: The try/finally block guarantees cleanup even on Ctrl+C.
Logging: Prefer Write-Information or a structured logger; consider Start-Transcript or writing to a rolling log file.

Cross-platform notes

PowerShell 7+ with .NET Core supports FileSystemWatcher on Windows, Linux, and macOS. Behavior can vary slightly across filesystems (e.g., network shares, case sensitivity).
Network shares: Watchers over UNC paths may miss events if the server throttles notifications. Consider local staging or periodic reconciliation as a fallback.

Security and reliability

Path traversal: Do not execute file content. Treat data as untrusted; validate and sanitize thoroughly.
Long paths: On Windows, enable long path support or normalize paths to avoid surprises.
Retry strategies: If Wait-FileReady times out, retry later or alert. Avoid infinite loops on corrupt files.

Putting it all together

With an event-driven pattern, you get immediate reactions to new files, fewer resource spikes, and simpler control flow compared to polling. The essential ingredients are:

FileSystemWatcher with tight filters and tuned buffer sizes.
Register-ObjectEvent handlers for the events you care about (Created, Changed, Renamed).
Safe read guard to avoid partial/locked files.
Debounce and queueing for resilience under bursty workloads.
Clean shutdown with unregistration and disposal to prevent resource leaks.

Adopt this pattern to build fast, reliable, and maintainable file-driven automations in PowerShellfrom ingestion pipelines and ETL drop-folders to build artifact processors and compliance collectors.

Quick checklist

Use Filter and NotifyFilter to limit noise
Bump InternalBufferSize for bursts
Verify file readiness before reading
Handle Created/Changed/Renamed as needed
Debounce duplicate events
Unregister + dispose on exit

What you get: instant feedback, less polling, cleaner exits, safer processingand fewer 3 a.m. on-call surprises.