Netty

Title: Netty

Preface

References:

Java NIO

Three Main Components

Channel & Buffer

A Channel is somewhat similar to a stream; it is a bidirectional conduit for reading and writing data. Data can be read from a Channel into a Buffer, and data can be written from a Buffer into a Channel. Traditional streams are either input or output, whereas Channels operate at a lower level.

Common Channels include:

• FileChannel
• DatagramChannel
• SocketChannel
• ServerSocketChannel

A Buffer is used to hold data being read or written. Common buffers are:

• ByteBuffer
• MappedByteBuffer
• DirectByteBuffer
• HeapByteBuffer
• ShortBuffer
• IntBuffer
• LongBuffer
• FloatBuffer
• DoubleBuffer
• CharBuffer

Selector

A Selector works with a single thread to manage multiple channels and retrieve the events that occur on those channels. The channels operate in non‑blocking mode, so the thread is not stuck on any single channel. This is ideal for scenarios with a very large number of connections but low traffic.

Calling selector.select() blocks until one or more channels become ready for read/write. When such events occur, the select method returns, and the thread can handle the events.

ByteBuffer

Basic Overview

ByteBuffer is a class in Java NIO (non‑blocking I/O) that provides the ability to work with raw byte data. It is mainly used for:

• Reading from and writing to files, networks, and other I/O sources
• Supplying a buffer that supports asynchronous or non‑blocking I/O operations

Two Types

ByteBuffer comes in two varieties:

• Regular**ByteBuffer**: created via ByteBuffer.allocate(int capacity); the buffer’s memory resides in the JVM heap.
• Direct**ByteBuffer**: created via ByteBuffer.allocateDirect(int capacity); the buffer’s memory is allocated in the operating system’s direct memory, typically used for high‑performance I/O.

Feature	Regular ByteBuffer	Direct ByteBuffer
Memory location	JVM heap	OS direct memory
Access efficiency	Lower (requires copies)	Higher (avoids copies)
Garbage collection	Managed by JVM	Managed by OS, not affected by JVM GC
Suitable for	Small data and infrequent I/O	Large data and high‑frequency I/O

Use cases

• Regular**ByteBuffer** – good for small files or occasional I/O.
• Direct**ByteBuffer** – good for large files, high‑frequency I/O, or performance‑critical scenarios.

Key Properties

Important ByteBuffer properties:

• capacity: total size of the buffer
• position: current read/write pointer
• limit: the limit up to which data may be read (i.e., the size of valid data)

When a buffer is created, position = 0 and limit defaults to the capacity.

In write mode, position indicates where the next byte will be written, and limit equals the capacity. The diagram below shows the state after writing 4 bytes:

(illustration omitted)

After calling flip(), position is set to 0 (the start of the readable region) and limit is set to the previous position (the amount of data written).

When 4 bytes have been read, the state looks like this:

(illustration omitted)

Calling clear() resets the buffer to its initial write state:

(illustration omitted)

Switching from read mode back to write mode mid‑stream can be done with compact(). This compresses the unread portion to the front and then switches to write mode, as shown:

(illustration omitted)

Common API

Debug tool – a utility class that can display the internal state of a buffer after each API call.

Creating Buffers

• **static ByteBuffer allocate(int capacity)**: allocate a new byte buffer with the specified capacity (in bytes)
• **static ByteBuffer wrap(byte[] array**: wrap an existing byte array in a ByteBuffer. No copy is made; the buffer uses the supplied array directly

Property Operations

• int capacity(): returns the buffer’s total capacity (maximum number of bytes it can hold)
• int limit(): returns the current limit (the highest index that may be read or written)
• int position(): returns the current position
• ByteBuffer position(int newPosition): sets the position to a new value
• ByteBuffer limit(int newLimit): sets a new limit
• ByteBuffer rewind(): resets the position to 0
• **void mark()**: marks the current position; later you can return to it with reset()

Note: both rewind and flip clear any previously set mark.

• **void reset()**: resets the position to the value that was set by the most recent call to mark(); together these APIs implement rewind()

Reading/Writing Data

• ByteBuffer put(byte b): writes a single byte at the current position and increments the position by 1
• byte get(): reads a single byte from the current position and increments the position by 1
• ByteBuffer put(byte[] src): writes multiple bytes from a byte array into the buffer
• ByteBuffer get(byte[] dst): reads multiple bytes from the buffer into a byte array

State Management

• ByteBuffer flip(): switches to read mode, setting the limit to the current position and resetting the position to 0
• ByteBuffer clear(): clears the buffer, preparing it for writing; position becomes 0 and limit is set to capacity
• boolean hasRemaining(): checks whether there are remaining bytes to be read
• int remaining(): returns the number of bytes between the current position and the limit

Slicing and Copying

• ByteBuffer slice(): creates a new ByteBuffer that is a slice of the current buffer (shares the underlying data)
• ByteBuffer duplicate(): creates a new ByteBuffer with the same content but independent position and limit

File Programming

FileChannel

FileChannel is a class in the Java NIO (New I/O) library for efficient file read/write. Unlike traditional stream I/O, FileChannel supports buffer‑based operations (Buffer) and offers extra features such as memory‑mapped files and file locking.

Features

• FileChannel is bidirectional—it can read and write data.
• FileChannel can only operate in blocking mode, unlike SocketChannel (see below) which supports non‑blocking.

Acquisition

You cannot open a FileChannel directly; you must obtain it through one of three stream types:

• FileInputStream: the resulting FileChannel is read‑only
• FileOutputStream: the resulting FileChannel is write‑only
• RandomAccessFile: can read or write depending on the construction mode of RandomAccessFile

Common API

• Reading data – use read(ByteBuffer dst) to read from FileChannel into ByteBuffer. The return value is the number of bytes read; -1 indicates end‑of‑file.

• Writing data – typically involves:

  1. Calling put() to put data into ByteBuffer
  2. Calling flip() to flip ByteBuffer into read mode
  3. Looping with while and repeatedly invoking channel.write(buffer) until all buffered data are written to the channel.

  Note: write() may not write the entire contents of buffer to FileChannel in one go, so you must keep calling it inside a while loop.

• Getting file size – size() returns the size in bytes.

• Closing the channel – after file operations, close FileChannel to free resources. Invoking the close() method on FileInputStream, FileOutputStream, or RandomAccessFile will indirectly close FileChannel.

• Forcing data to disk – the OS usually caches writes for performance. Call force(true) to force both file contents and metadata (e.g., permissions) to be flushed to disk, ensuring durability.

• Truncating a file – FileChannel provides truncate(long size) to truncate the file to a specified length; excess bytes are discarded.

• File position control – position() and position(long newPosition) get and set the current position of a FileChannel, useful for random access.

• Memory‑mapped files – map() can map a portion or the whole file into memory, which is especially handy for large files because it allows direct memory access. map() returns a MappedByteBuffer object.

Channel‑to‑Channel Data Transfer – code example showing data transfer between two channels.

Path & Paths

Path is a class introduced in Java NIO to represent a file‑system path. It lives in the java.nio.file package, and the Path interface replaces the old java.io.File, offering a more flexible and efficient way to work with files.

Path represents a path, which can be absolute or relative. Path objects are created via Paths.get() or FileSystems.getDefault().getPath(), providing a more intuitive, chainable API.

Common API

• Retrieve basic path information:

  • Path getFileName(): returns the file name
  • Path getParent(): returns the parent path, or null if there is none
  • Path getRoot(): returns the root component (e.g., C:/ or /); returns null if there is no root
  • int getNameCount(): returns the number of name elements (excluding the root)
  • Path getName(int index): returns the n‑th name element (index).

• Path operations:

  • **Path resolve(Path other)**: resolves the current path against another, producing a new path. If the argument is absolute, the result is that absolute path (other → other).
  • **Path relativize(Path other)**: computes the relative path from the current path to a target path; both must share the same root.
  • **Path normalize()**: normalizes the path by removing redundant . and .. components.
  • **Path toAbsolutePath()**: converts a relative path to an absolute one.

• Path comparison:

  • boolean startsWith(Path other): checks whether the current path starts with a given path.
  • boolean endsWith(Path other): checks whether the current path ends with a given path.

• Converting to a File:

  • File toFile(): converts a Path to a File so that legacy java.io APIs can be used.

Files

The Files class offers many static methods for file and directory operations, typically requiring a Path argument.

Common tasks include:

• Checking file status:

  • boolean exists(Path path): does the file or directory exist?
  • boolean isDirectory(Path path): is the given path a directory?

• Creating and deleting:

  • Path createFile(Path path): create a file (throws if it already exists)
  • Path createDirectory(Path path): create a directory
  • void delete(Path path): delete a file or directory; throws DirectoryNotEmptyException if the directory is non‑empty

• Reading and writing:

  • **List readAllLines(Path path)**: read all lines of a file, returning List
  • **void write(Path path, byte[] bytes)**: write a byte array to a file, overwriting existing content

• Copying and moving:

  • **void copy(Path source, Path target, CopyOption... options)**: copy a file or directory; throws FileAlreadyExistsException if the target already exists
  • **void move(Path source, Path target, CopyOption... options)**: move a file or directory

• Traversing directories: before JDK 1.7 you had to recurse manually; since 1.7 you can use Files.walkFileTree for directory traversal.

Note: Deletion is dangerous—ensure that recursively deleted directories contain no important data.

Network Programming

Blocking

In blocking mode, the relevant methods cause the thread to pause:

• ServerSocketChannel.accept blocks the thread until a connection is established.
• SocketChannel.read blocks the thread until data is available to read.

A blocked thread does not consume CPU while paused, but it is essentially idle.

With a single thread, blocking calls interfere with each other and the program can hardly work; multithreading is required.

However, multithreading introduces new problems:

• On a 32‑bit JVM each thread consumes ~320 KB; on a 64‑bit JVM each thread consumes ~1 MB. With many connections you can quickly run out of memory (OOM). Moreover, too many threads cause frequent context switches, degrading performance.
• Thread pools can reduce the number of threads and context switches, but they do not solve the fundamental issue: if many connections stay idle for a long time, all pool threads become blocked. Therefore blocking I/O is unsuitable for long‑lived connections; it works only for short‑lived ones.

Code example:

(omitted)

Non‑Blocking

In non‑blocking mode, the related methods never pause the thread:

• When ServerSocketChannel.accept finds no pending connection, it returns null and the thread continues.
• SocketChannel.read returns 0 when no data is available, allowing the thread to perform other work (e.g., read from another SocketChannel or call ServerSocketChannel.accept()).
• When writing, the thread only waits for the data to be placed into the channel; it does not wait for the OS to actually transmit the bytes.

Even so, the thread continuously polls for readiness, which can waste CPU cycles when there is no activity.

During data copying, the thread is still effectively blocked (this is what Asynchronous I/O—AIO—aims to improve).

Code example: server side; client code unchanged.

Multiplexing

Overview – a single thread can use a Selector to monitor read/write events on multiple channels; this is called multiplexing.

• Multiplexing applies only to network I/O; ordinary file I/O cannot benefit from it.
• Without a selector, a non‑blocking thread would spend most of its time doing nothing. A selector ensures that:
  • It attempts to connect only when a connection‑ready event occurs.
  • It reads only when a read‑ready event occurs.
  • It writes only when a write‑ready event occurs (a channel may not always be writable; when it becomes writable, the selector fires a write event).

Advantages

• One thread plus a selector can monitor many channels; the thread works only when an event occurs, avoiding wasted effort.
• The thread is fully utilized.
• Fewer threads are needed.
• Context‑switch overhead is reduced.

Basic Operations

• Creation

• Registering (binding) channel events – the channel must be in non‑blocking mode; FileChannel lacks a non‑blocking mode and therefore cannot be used with a selector.

• Event types that can be registered:

  • connect – triggered when a client successfully connects
  • accept – triggered when the server accepts a connection
  • read – triggered when data is available to read (may fire later if the receiver is slow)
  • write – triggered when the channel is ready for writing (may fire later if the sender is slow)

• Listening for channel events – three methods can be used to detect whether events have occurred.

(content truncated)

---

Originally written by Li Wei (李唯_) and published in Chinese on 后端技术栈全书 (Full-Stack Backend Engineering). Translated and adapted for DriftSeas with permission.