2023.08.07 - Patterns for building LLM-based systems and products

2023.08.07 – Patterns for Building LLM‑Based Systems and Products

Type vs Interface & Tribute to the Father of Vim & Introducing Dust & Future School

This Week’s Highlights

Patterns for building LLM‑based systems and products https://eugeneyan.com/writing/llm-patterns/
This post is about practical patterns for integrating large language models (LLMs) into systems and products. We draw from academic research, industry resources, and practitioner know‑how, and try to distill them into key ideas and practices. There are seven core patterns. I’ve also organized them along the spectrum of improving performance vs. reducing cost/risk, and closer to the data vs. closer to the user. Related:

Type vs Interface: Which Should You Use in 2023? https://www.totaltypescript.com/type-vs-interface-which-should-you-use
The difference between type and interface is small enough that you can use either without many problems. The TypeScript team recommends defaulting to interface and only using type when needed. I’d like to recommend the opposite. The features of declaration merging and implicit index signatures are surprising enough that they should scare you off using interfaces by default. Interfaces are still my recommendation for object inheritance, but I’d suggest using type by default. It’s a bit more flexible and a bit less surprising.

How we build containerized services at GitHub using GitHub https://github.blog/2023-08-02-how-we-build-containerized-services-at-github-using-github/
Learn how we build containerized services that power micro‑services on the GitHub.com platform and many internal tools.

The value of a prototype is in the insight it imparts, not the code https://addyosmani.com/blog/prototypes/
Prototyping lets us fail fast and cheap, or get the data to make a concrete decision on direction. It helps answer the question, “What happens if we try this?”. Most significantly, prototyping provides us with guardrails to fail safely and productively. Failure is an integral part of the learning process.

Things we always remind ourselves while coding https://changelog.com/posts/things-we-always-remind-ourselves-while-coding
Note to self: slow down and think… but don’t overthink it.

In‑Depth Reading

Meituan Front‑End Development Framework Rome: Practice and Evolution Trends https://tech.meituan.com/2023/08/03/meituan-rome-practice.html
When the Rome project was first approved, we already had the underlying infrastructure ready: a converged publishing system and a base architecture built on S3 (Meituan’s internal storage service) with static‑dynamic separation. However, the upper‑level development framework and component libraries were numerous and inconsistent. Problems included a large team size leading to high learning and hand‑off costs, low overall development efficiency, and difficulty reusing engineering capabilities across teams.

At the start, we built the front‑end framework on a pure static S3 architecture. This stemmed from early issues with our Node.js‑based full‑stack architecture:

Business units initially focused on internal admin scenarios, so page‑load speed and SEO were low priorities.
The Node.js ecosystem was still immature; front‑end engineers had to handle dynamic scaling, peak‑traffic handling, etc., raising business risk.
Additional pain points were high machine costs, demanding developer skill requirements, and hiring challenges.

A Tale of Evading JavaScript Anti‑Debugging Techniques https://www.nullpt.rs/evading-anti-debugging-techniques
Debuggers are invaluable tools that let developers pause code execution and analyze its behavior at any moment. By using a debugger, developers can efficiently identify and fix issues, making it an indispensable part of the toolkit.

Understanding React Server Components https://vercel.com/blog/understanding-react-server-components
Learn the fundamentals of React Server Components to better understand why (and when) to adopt them.

I’m Betting on HTML https://catskull.net/html.html
With the rise of large‑language‑model‑based AI, semantic HTML is more important than ever. At its core, the internet transmits data that helps humans interact with the world as they perceive it. The freedom that HTML/CSS/JS provide is a double‑edged sword because access to data has become limited. Instead of open, accessible data formats and APIs, we’re confined within the walled gardens of major tech companies that run massive social‑media platforms.

Rebuilding the Replit Filetree for Superior Performance and Accessibility https://blog.replit.com/filetree-updates
The original filetree worked for small projects but showed performance problems as projects grew. This became especially noticeable when we used Replit to build Replit—our web repo had around 8,000 files. We decided to revamp the filetree, and today we’re excited to share the improvements, focusing on performance and accessibility.

A Foundations‑First Approach to Product Design https://www.viget.com/articles/a-foundations-first-approach-to-product-design/
Today’s consumers expect greatness from the products they use. To get there, start with the basics.

End‑to‑End Testing for Confluence Forge Apps https://blog.developer.atlassian.com/end-to-end-testing-for-confluence-forge-apps/
When building a Forge app for an Atlassian product, a robust testing strategy is essential. End‑to‑end testing lets you continuously monitor app quality and reliability. In this post we explore how to use Mocha and Webdriver.io to test two Confluence apps, providing code samples and a video of the tests running to help you get started quickly and confidently.

Fresh Finds

Products & Miscellaneous

Splitting the Web https://ploum.net/2023-08-01-splitting-the-web.html
There’s an increasing chasm dividing the modern web. On one side, the commercial, monopoly‑riddled, media‑adored web—a web with only one objective: making us cl

(content truncated)

Originally written by Ping Xia (平侠) and published in Chinese on Web技术周刊 (Web Tech Weekly). Translated and adapted for DriftSeas with permission.

2023.08.07 - Patterns for building LLM-based systems and products