In the last two posts we understood the basics of fourier transform, how to speed up the DFT calculations with FFT, then looked at how we can use the FFT algorithm to solve different problems such as time series analysis, pattern matching in strings, convolution etc. Part 1 An excursion…

In this post I am going to solve a bunch of algorithmic problems to demonstrate the power of FFT algorithm and how it extends beyond time series analysis. For understanding the basics of FFT go through my previous post An excursion into Fast Fourier Transform — The Basics and Time…

Let’s begin by understanding how Fourier Transform (or more specifically Discrete Fourier Transform) works and how it is useful in software engineering. Here we will look at a common problem of times series analysis. Discrete Fourier Transform Given any function f(x), one can represent it as a sum of sine and cosine functions…

Ever since I learnt about data structures and algorithms in college, hash tables has been the most famous and most commonly used data structure in real life software engineering. …

In this post, I am going to build a simple distributed key-value store using JAVA and sockets for networking. I am going to show how to use Zookeeper as a coordination service in a cluster with multiple partitions and replication. Following are the functions of Zookeeper service in this system: …

“If the only tool you have is a hammer, it is tempting to treat everything as if it were a nail.” — Abraham Maslow In the field of software engineering, every choice you make requires making one or more tradeoffs. …

This post is a gist of our efforts in migrating from managed Spark in Synapse and Databricks to Kubernetes for our production workloads. Without the nitty gritty of migration efforts, lets get into the details of how did we setup our Spark cluster and how our Spark application runs on…

In a distributed hash table, a hash table with N keys is partitioned into M partitions or nodes. This is done in order to handle large number of keys and queries that exceeds the limitations of a single machine CPU and RAM. For e.g. if the size of a hash…

Having worked on variety of problems over the past 10 years or so, one of the most challenging aspect that still remains for me is how to tackle inter-dependency problems in OOPs efficiently? The confusions that arises when dealing with problems that require dealing with inter-dependency in an OOPs are…