NOTE: The themes are varied, and some links below are affiliate links. Software engineering, Spark, history, python. Expect a similar wide range in the future as well. You can check all my weekly readings by checking the tag here . You can also get these as a weekly newsletter by subscribing here.

Testing and debugging Apache Airflow

We've been using Airflow for almost a year (on my suggestion). I'm not super-happy with it, and testing it is one of the pain points. Knowing how to do it is half the battle.

Mind control, levitation and no pain: the race to find a superman in sports

You must lose

I can't Do Anything for Fun Anymore; Every Hobby Is an Attempt to Make Money

You've probably felt the same

Databricks open-sources Delta Lake to make data lakes more reliable

I'm looking forward checking the code for this (should be equivalent to Databricks' Delta product)

Tolkien was right: Scholars conclude Beowulf likely the work of single author

In off the moors, down through the mist beams, god-cursed Grendel came greedily loping.

Don’t inherit Python built-in dict type

Remember kids, what you want to do is either Mapping (read-only) or MutableMapping abstract base classes from collections.abc

How to Integrate Your Stuff into Spark SQL

A pity you need a custom build of Spark in this case (and fighting with EMR to install it)

Using Direct Writes to Significantly Increase the Performance of Spark Workloads

If you happen to use Qubole and Hive, this can be a winner.

Low-memory symbol indexing with bloom filters

I love bloom filters, and have been using the metals Scala language server for a while now in emacs (spacemacs).

CONSTANT Anti-Patterns : tlaplus

That's no constant

Why software projects take longer than you think -a statistical model

We all know we are bad at estimates as humans, mathematical models just can confirm this

How to knit a sports car with carbon fibre - The new black arts of manufacturing

Rapid prototyping and construction with carbon fibre can change a lot of things in manufacturing

🎥 Build a roguelike in Rust

Using libtcod. Also a blogpost series available here.

Newsletter?

These weekly posts are also available as a newsletter. These days (since RSS went into limbo) most of my regular information comes from several newsletters I’m subscribed to, instead of me going directly to a blog. If this is also your case, subscribe by clicking here.