Both DuckDB and esProc SPL can be embedded in applications as computing engines. This article will compare which is more lightweight. The term “lightweight” refers not only to size, but also to simplicity in development and maintenance.

DuckDB is indeed convenient to use; you can directly import it in Python and get started, and it integrates smoothly with Java ecosystem via JDBC. esProc primarily targets the Java ecosystem – its 15MB jar can be easily deployed within a project, enabling seamless execution. For non-Java programs, invocation is achieved through an HTTP interface. Both installation packages are quite small, exhibiting lightweight characteristics.

esProc scripts are interpreted and support hot deployment, enabling computational logic modifications without service restart. In this aspect, esProc is on par with DuckDB.

Differences in their lightweight nature are particularly evident in cross-data source mixed computation scenarios. Although DuckDB supports common file formats such as CSV and Parquet, as well as some databases like MySQL, it requires developing deeply customized connectors for each data source separately. Consequently, mainstream relational databases like Oracle and SQL Server remain unsupported, and it is even more challenging to support NoSQL databases like MongoDB. When users need to perform cross-source computations between MySQL and Oracle, the lack of official connectors typically necessitates resorting to Python for importing. The inclusion of such “glue code” not only complicates the technology stack but, more critically, burdens the system architecture, violating the lightweight principle.

In contrast, esProc SPL employs a “native interface + light encapsulation” approach, achieving natural compatibility with all relational databases through JDBC and enabling access to unstructured data sources, such as MongoDB and Kafka, with only a shallow encapsulation based on native interfaces. This standardized extension mechanism enables support for dozens of data source types, covering all scenarios like files, databases, API interfaces, and message queues. Moreover, users can rapidly extend through reserved extension interfaces, truly realizing a lightweight experience of “connect and compute immediately”.

In addition, DuckDB has a critical flaw when handling complex computations: SQL inherently lacks flow control capabilities. Basic functionalities like for/if are unavoidable for even moderately complex business logic. However, since SQL cannot handle these operations, and DuckDB provides no supplemental mechanisms like stored procedures, users have to resort to external languages with flow control capabilities, such as Python, to brute-force solutions when encountering such requirements. This not only results in fragmented and verbose code but also necessitates maintaining two distinct technology stacks. It’s akin to building a crane just to move bricks—far from being lightweight and agile.

esProc’s SPL directly integrates flow control into data processing language, encompassing features such as loops, conditionals, and exception handling. It can handle familiar SQL queries while also replacing Python for flow control, making it a comprehensive language solution. Programmers no longer need to juggle between SQL and Python, the technology stack is simplified, and the overall performance is more lightweight.

Being lightweight isn’t just about having the smallest installation package; it’s like when moving—you can’t just look at the suitcase size, you need to see if one suitcase can hold all your belongings. DuckDB may appear small and nimble, but when it comes to scenarios requiring cross-data source association computations or writing business logic with loops and conditionals, it still needs external assistance. It’s like a rice cooker that promises one-touch cooking, but if you want to steam buns, you still need to connect an external steamer.

The cleverness of esProc lies in its ability to handle complex tasks on its own. Whether it’s a database or an API, as long as it can connect, it can compute—even perform mixed computations. Whether it’s simple statistics or complex rules, a single set of script syntax handles it all. It’s like a transforming toolbox—it looks the size of a screwdriver, but when opened, it reveals wrenches, pliers, and drills. Most importantly, there’s no need to search everywhere for accessories, which is what truly makes it lightweight.