Nim Redis

Motivation

I've been using Nim as my preferred development language since about 2020, even using it to complete some coursework (with approval from the professors). For one side-project, I wanted to utilize Redis Graph (now forked as an entirely separate product, Falkor DB) due to the nature of the data I was working with. Although Nim had several Redis client implementations, none of them support Redis Graph, and each had a few limitations I didn't love. So I wrote my own!

Why Build It?

There were several shortcomings, in my opinion, of existing Nim implementations of Redis clients:

• Either sync only or async only, no multi-sync support
• Used exceptions
• Socket type limitations (mostly only relevant for testing)
• Edge cases around handling some of the newer Redis protocols (ex. missing support for streams)

To exand a little further on all of these:

MultiSync

Nim has fantastic compile-time macro support, to the extent that async in Nim is implemented at the library level. Consequently, you can do really cool things like create multisync functions, which the compiler will automatically generate sync and async versions of, if some conditions are met.

This means that library users have the flexibility of choosing what makes more sense for them, and is a very nice way of resolving the function colouring problem for many simple cases.

Exceptions vs Results

While the "Nim way" of doing things generally prefers exceptions, they unfortunately are rather unpleasant to work with in async contexts. Not only are your stack traces much more complex and difficult to read (this is partially innate complexity as a consequence of async in general, but also particular complexity due to the macro transforms that Nim's async is built off of). Additionally, Nim's fantastic effect system isn't able to help check for lack of exceptions in async macros because the result of the macros produces code that can raise, even if your user code is exception-safe. Using results for the sync version allows us to prove that all of the library code is exception-safe.

Sockets

This feature came about mostly accidentally, as I started by writing a serialization/deserialization library for Redis, and was working with string streams. Once that was done, I realized that this API was incompatible with the existing Nim Redis clients, and with the socket API itself. So I decided to make a wrapper to expose Nim's sockets as streams, with the added benefit that this was entirely pluggable, so users could create other implementations that conformed to the interface (for example, a file stream).

Redis Streams + Pub/Sub

This was a later addition to the library, as I started working on a new real-time project, for which I wanted to use Redis as the message broker. However, at the time I was looking, no Nim libraries supported both pub/sub and streams. This ended up being a fairly simple addition, as the existing codebase was pretty extensible.

Current Thoughts & Learnings

Redis is a very cool platform, and I've ended up using my Nim client library in several other projects, so it was absolutely a worthwhile time investment. Not only did I learn much more about Redis (internals and protocol), but I also saved myself time downstream (ease of debugging, less error-prone code, etc).

One avenue I didn't adequately explore is if it would've been better to have contributed to an existing project instead of building my own, but Redis clients are pretty small libraries, and I felt there were substantial enough differences between what I was looking for and what existed that it would've basically been a complete transformation of an existing library. But still, I should've looked into it a bit more, and perhaps reached out to other authors.