Tmp File Upload Plugin

Stream large file uploads into temporary files instead of memory, so requests far larger than available memory are parsed safely.

Installation

npm install @orpc/node@beta

pnpm add @orpc/node@beta

yarn add @orpc/node@beta

bun add @orpc/node@beta

Setup

Use TmpFileUploadHandlerPlugin to parse file uploads into temporary files. Bodies the standard parser would buffer into an in-memory File, and multipart/form-data file parts, are streamed to disk instead. Every other body is left to the standard parser.

import { TmpFileUploadHandlerPlugin } from '@orpc/node'
import { RPCHandler } from '@orpc/server/node'

const handler = new RPCHandler(router, {
  plugins: [
    new TmpFileUploadHandlerPlugin({
      /**
       * The directory temporary files are created under. Each request that
       * spools an upload gets its own subdirectory inside it, removed when
       * the request finishes.
       *
       * @default os.tmpdir()
       */
      tmpDir: './uploads',

      /**
       * The size limit for each kind of request body. Every kind is required
       * when the option is given, so none is left unbounded by accident; set
       * a kind to `Number.POSITIVE_INFINITY` to deliberately leave it
       * unlimited.
       *
       * @default unlimited for every kind
       */
      maxBodySize: {
        /**
         * Content parsed into memory: JSON, URL-encoded forms, and the plain
         * fields of a multipart body. Usually the lowest of the three limits,
         * because this content cannot stream anywhere.
         */
        memory: 1024 * 1024,

        /**
         * Upload content streamed into temporary files: file bodies and the
         * file parts of a multipart body combined.
         */
        file: 10 * 1024 * 1024 * 1024,

        /**
         * Content consumed as a stream: event streams and raw binary streams.
         * Enforced while the stream is consumed, so an oversized stream fails
         * at the reader. Usually the highest of the three limits, because this
         * content is consumed on the fly.
         */
        stream: Number.POSITIVE_INFINITY,
      },
    }),
  ],
})

Working with Uploaded Files

Procedures receive ordinary File instances and can read them as usual, in constant memory because the content is read lazily from disk. Each one is a TmpFile exposing the path of its backing file, so an upload can be kept with a cheap rename instead of a copy:

import { TmpFile } from '@orpc/node'
import { rename } from 'node:fs/promises'

const uploadVideo = os
  .input(z.object({ video: z.file() }))
  .handler(async ({ input }) => {
    if (input.video instanceof TmpFile) {
      await rename(input.video.path, `./videos/${crypto.randomUUID()}`)
    }
  })

Limiting Body Sizes

Each request body counts against the limit matching what it actually costs: memory bounds content parsed into memory, file bounds content streamed into temporary files, and stream bounds content consumed as a stream. A multipart body splits across the first two, its fields against the memory limit and its file parts against the file limit, and as a whole, framing included, it is bounded by the sum of both.

The plugin subsumes the Request Limit Plugin while allowing large uploads and streams without raising the ceiling for everything else. A declared Content-Length over the applicable limit rejects immediately, and the limits stay enforced while the body streams in, so a lying length cannot bypass them.

When the Request Compression Plugin is present, the plugin orders itself to receive the decompressed bytes, so every limit applies to the real payload rather than the compressed wire size.

Learn More

For implementation details, see the source code.