Building a Cloud Storage Kubernetes Operator with Go and Operator SDK

In the last article, we looked at Mutating Admission Webhooks as a way to extend Kubernetes. In this article we’ll explore another concept: Kubernetes Operators.

Kubernetes Operators

The Kubernetes docs defines operators as:

Operators are software extensions to Kubernetes that make use of custom resources to manage applications and their components

This might seem a little abstract right now, but we’ll try to explore this concept further by actually implementing an Operator.

Autobucket Operator

In this article we’ll build “Autobucket Operator”, a Kubernetes operator that automatically manages Cloud Object Storage (like GCP Cloud Storage Buckets or S3 Buckets) for a Kubernetes Deployment. …

A “magic” way to inject a file into Pod Containers

Have you ever noticed that when you create Pods in Kubernetes, the containers (usually) contain an authentication token file located at /var/run/secrets/ ? You can try it out by running the following command in your cluster:

$ kubectl run busybox --image=busybox --restart=Never -it --rm -- ls -l /var/run/secrets/
# output

Side note: You can actually opt out of this behaviour in Kubernetes versions 1.6+

Let’s now imagine that we want to automatically add a magic “hello.txt” file into all (or a group) of pod container filesystems, without explicitly attaching a volumeMount to each pod spec. …

If you’ve been building web services on GCP Cloud Functions or Cloud Run, then you might have noticed that Google Cloud has been missing an important piece of infrastructure: an API Gateway.

While AWS has had its Amazon API Gateway for a few years now, GCP had no equivalent service (there was Cloud Endpoints but I think the scope is a bit different), leaving developers to build custom solutions … But not anymore ! GCP has recently announced the brand new Google Cloud API Gateway, so I thought I’ll give it a try. …

Having worked with a few Distributed Systems over the years such as Apache Kafka and Ethereum nodes I was wondering what it would take to build a minimal one. In this article we’ll be building a micro “kafka-like” distributed logging app with super simple features, using Go and Hashicorp Consul. You can follow along using this github repository.

Go + Consul: A solid combination for building Distributed Systems

What is Consul ?

Looking at the definition in the intro section of the official website:

Consul is a service mesh solution providing a full featured control plane with service discovery, configuration, and segmentation functionality

That might sound a bit scary, but don’t leave just yet…

Do you deploy your web apps to cloud virtual machines such as Digital Ocean Droplets or Scaleway Cloud Instances ?
If your app saves any kind of data to disk and you’re worried about data loss due to software bugs, human errors or malicious actors (amongst other risks) then you need a backup strategy.

Retro Data Backups :)

Cloud Volume Backups

Most cloud providers provide a Hot Backup feature which takes a snapshot of your volume data without the need to stop your instances during the backup process (but there is a data loss risk involved, please read up on this topic and make an informed decision).

Elevator Saga ported to Go WASM input

A few days ago, I wanted to take Go WebAssembly out for a spin, but I wasn’t sure what to build. Maybe a game ? but I didn’t really want to build a game from scratch. So I remembered the Elevator Saga, a programming game by Magnus Wolffelt I stumbled upon a few years ago: The goal is to transport people up and down building floors by controlling elevators, writing the rules in Javascript. What if I modified it to accept WebAssembly flavoured Go instead of JS !

Meet the Go Wasm Elevator Saga. The github repo is available here.

Go Wasm Elevator Saga

How it works

Gophers Meet Badgers

Over the years I’ve used a number of Job Queues such as Sidekiq, Kue, or Faktory and they are often a central piece of many apps I build. As embedded databases are so popular in the Go world, I thought I’d try to create an open source embedded job queue for Go apps: Goblero

What is a Job Queue ?

Job Queues are also known as Background Job Queues, Task Queues or Work Servers. They can be very handy to handle asynchronous tasks or long running tasks that don’t belong in a web request handler for example. …

Go + Serverless = ❤️

Google Cloud Platform now supports Go 1.11 for Cloud functions. Go and Serverless are 2 things I’m very excited about these days so I decided to give this new GCP feature a try and build a basic Image Resizer in pure Go using the disintegration/imaging package.

Resizing gophers

Sometimes you just have gopher images that are too large to display on your website. They would eat up the precious bandwidth of your users ! Like this massive one:

Big gopher generated using

So we’ll be transforming it into this smaller, more reasonable gopher:

Testing a Node.js + AWS Lambda + API Gateway app

Serverless computing and FaaS (Function as a Service) are planned to grow massively over the next few years. And each major cloud provider already has an offering: AWS Lambda, Google Cloud Functions, Azure Functions… But what does it mean for us web developers ? How can we adapt our development workflow when moving from traditional server based applications to “serverless” ? Let’s explore the testing side of the story !

AWS Lambda was first introduced in November 2014

Whenever I’m experimenting with a new technology, one of the first questions that comes up is: how do I write automated tests ? I think testing is a very important…

A hands-on use case

The transparency issue

Public blockchains like the Ethereum mainnet keep an immutable record of transactions, but the issue is that the ledger is 100% transparent and anybody with access to an Ethereum node has access to all of the data on the blockchain.

When discussing potential projects with clients recently, whenever I explained to them this aspect of public blockchains, most of the time it is not exactly what they had in mind. Few businesses actually want to make their data public and there are many reasons for this: Competition, intellectual property, consumer privacy …

This is when permissioned blockchains come into play:

Permissioned Blockchains & Quorum

Adil H

Backend & Cloud Engineer (Golang)

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store