Posts

A cruicual aspect of distributed systems is to have a clear consensus on who is the current leader. Consensus algorithms help us in determining a leader and provide us with the constitution for what happens when the leader inevitably goes down. Enter Raft, the algorithm that promises to make consensus more understandable. While its popular peer Paxos has an air of mystique, Raft is the algorithm that rolls up its sleeves and says, “Let’s make this easy to follow.” ...

fmt.Printf is like the Swiss Army knife of Go when it comes to string formatting. Whether you’re debugging, logging, or trying to impress your peers with some beautifully formatted output, fmt.Printf has your back. So, let’s dive into some of the most commonly used formatting verbs and flags in Go’s fmt package. The Basic Placeholders Here’s the bread and butter of fmt.Printf—the simple formatting verbs: %v: Prints the value in a default format. It’s like the wild card of formatting. ...

Go is a wonderfully simple language—until it isn’t. Sure, you’ve got your basic if, for, switch, and even goto (although, let’s be honest, no one likes to admit using it). But when you dig deeper, Go offers some extra gems that make it stand out: defer, panic, and recover. These three can seem like the “mystery meat” of Go, but once you understand how they work, they’re not only powerful but also fun to use (with a few “gotchas” along the way). Let’s dive in. ...

Vectors are fundamental entities used to represent quantities that have both direction and magnitude. Whether you’re working in machine learning, physics, or computer graphics, vectors have a crucial role. However, the raw magnitude of a vector is not always useful in certain computations. This is where vector normalization comes into play. Vector normalization is a process that scales a vector so that it has a unit length of 1 but retains its direction. This operation is particularly useful in various applications, including machine learning, physics simulations, and computer graphics. ...

When data is stored across multiple servers in a distributed system, it is crucial to determine the order in which operations occurred to maintain consistency and ensure the system behaves correctly. But why can’t we rely on the system timestamps? Non-monotonicity of System Clocks: System clocks are not guaranteed to be strictly increasing over time (monotonic). For instance, when servers synchronize their time using the Network Time Protocol (NTP), the clock may be adjusted backward if it was ahead of the actual time. This backward adjustment can create confusion in determining the true order of events, as a later operation might appear to have occurred before an earlier one. Crystal Oscillator Drift: System timestamps are generated based on the server’s internal clock, which relies on a crystal oscillator. Over time, this oscillator can drift, causing the server’s clock to become slightly inaccurate. To correct this drift, NTP is used, but this correction process can cause time to “jump” backward or forward, further complicating event ordering. Incomparable Clocks Across Servers: Even if each server had a perfectly accurate clock, the timestamps from different servers cannot be directly compared. Each server’s clock might be slightly ahead or behind others, leading to inconsistent time comparisons across the system. Lamport Clocks to the Rescue To address these challenges, Lamport Clocks are used. Lamport Clocks provide a way to assign a logical timestamp to events in a distributed system, ensuring a consistent order of events. ...

Qdrant vector DB supports updating multiple points using batch_update_points(). Batch update points can take in multiple different operations as parameters and run them as a single API operation. batch_update_points() supports 4 kinds of update operations. UpsertOperation DeleteOperation UpdateVectorsOperation DeleteVectorsOperation SetPayloadOperation OverwritePayload DeletePayloadOperation ClearPayloadOperation batch_update_points() supports mixing up different kinds of operations in one request. For example you could perform an UpsertOperation for point ids [1, 2, 3] along with a delete operation on point id [4, 5, 6]. ...

As applications demand more from databases and disk performance struggles to keep up, the way we handle I/O operations becomes crucial. Buffered I/O is like the memory foam of database operations. Instead of writing directly to the disk, data is first written to a buffer in memory. The operating system then decides when to flush this data to the physical disk. This approach makes the disk appear faster because the OS can batch operations, reducing the frequency of disk writes. It’s like a smooth operator, managing spikes in I/O demand and keeping things running consistently. ...

Managing multiple versions of Python can be tricky, especially when juggling different projects requiring their own specific environments. That’s where Pyenv comes in. It’s a handy tool that makes switching between Python versions effortless. In this article, we’ll explore how Pyenv works, the concept of shims, and why these features make Pyenv so useful. What Pyenv Offers Pyenv is a versatile tool that provides several key features to help manage your Python environment: ...

As content becomes more and more freely available, it has become kind of a full time job to identify what to watch next. The streaming services have their own recommendation algorithms but they work in silos. In this article we are going to build a bare bones recommendation system using Qdrant vector search. We will be using semantic similarity to find movie and tv show recommendations. Disclaimer: I am no expert in AI, recommendation systems or vector databases. This post is an experiment to move towards that point in this high dimensional vector space. ...

A mixin is a class that provides some functionality that can be easily incorporated in other classes. Mixins usually provide some standalone functionality that can be reused in many different classes. Python supports Multiple inheritance. Multiple inheritance means that a class can inherit from multiple parents. Multiple inheritance also implies that the order of parent classes becomes important. Python has the concept of MRO. MRO or Method Resolution Order is the order in which Python looks for a method in a hierarchy of classes. The order goes from left to right, which means the right most can be considered as the base(-est) class. The methods in the classes on the right will get over written by the methods in the classes on the left. ...