Note some links below are Amazon affiliate links. Buy at your own risk 🤣

Work

During the first half of this year I have been leading a migration to Spark of several systems, one relatively straightforward and the other based on GraphFrames and a graph with ~1 billion nodes. The results so far seem excellent. The second half of the year has been mostly GDPR compliance improvements (I've read the whole law... twice!), looking for additional large-scale data providers and preparing our systems for global scale (around 6x event data capabilities). Has been fun and challenging.

Life

• In February I moved from East Finchley to Chiswick. This meant I became a Friend of Kew Gardens, and have been there almost every weekend I've been in London. And you could catch me a bit earlier at The Coffee Traveller, enjoying a latte with a carrot cake muffin.

• In February, we spent a few days in Puigcerdà for my birthday. Not as nice as we expected. The food was good, though (as expected).
• In March we visited Edimburg again, Laia was attending EdiYarn Festival. I spent the days at Waterstones, eating cupcakes and getting my reading list under control.
• In May I attended AWS Summit in London. Was ok-ish. Lack of coffee made us grumpy.
• In June I attended SoCraTesUK 2018 in Dorking, together with Carlos. I had an excellent time there, and will repeat next year if possible. We presented together Welcome to Apache Spark. I also learnt some Cobol.

 

• In June as well, I presented How does that PySpark thing work? And why Arrow makes it faster? at the Python Barcelona meetup. I think people enjoyed it.

• At the end of June we visited Bath for Laia's birthday. Nice town, but a bit packed with tourists. We were lucky England was playing some soccer match and could visit the Roman baths without people.
• In July we published the first podcast of PyBCN (in Spanish). It's not monthly, next one was in... December 😆
• In August we roasted a bit under the heat wave hitting the UK by visiting Mayfield Lavender field.
• In September I attended Scala Italy in Florence, where I got the opportunity to visit Osteria del Cinghiale Bianco again. Excellent.

• In September I presented How does that PySpark thing work? And why Arrow makes it faster? at the Spark London meetup, feedback was positive as well.
• In October, I attended Spark Summit Europe in London. Met a lot of people and had a terrific time.

• In October we took a few days off so we could disconnect, and spent them on Hay-on-Wye, as some tradition we follow says. We bought a bunch of books (I think there are no books by Roger Zelazny left there) and I did nothing. Bliss. Also, best scones ever.
• In November I helped a bit from afar with the organisation of PyDay 2018. I couldn't attend, but looked excellent. We also told the world PyBCN is now an organisation you can become a member of.
• In December, I attended Scala Exchange for the third time, and gave a lightning talk about TLA+ (Formally Verifying Complex Systems Using TLA+). Looks like people enjoyed my talk, and I had an excellent time as usual.

I made minor contributions (typos, small fixes) to several open source projects (Spark, typeshed, Monix, bloop,...), and also created a couple for my enjoyment: experimenting with Processing in Scala, sbt-ifttt and rb-zsh-theme.

Learning

This year I got a bit deeper into functional Scala, using GraphFrames and (in general) graph operations on big data. Got acquainted and got to hate Apache Airflow. I improved my knowledge of Akka (adding clustering and sharding to the "could do"). I also learned TLA+ and I plan to dive deeper into this rabbit hole the coming year. Started learning Rust. Made some minor stuff using HyperApp, and have been considering learning PureScript so I can write something close to "enjoyable" Javascript. I have also started playing the ukulele.

Reading

My top pick for the year is without any doubt Thinking in Systems: A Primer among books I read this year (19 books). Didn't read much non-fiction, and can't recommend any this year. According to Pocket (which I started using around June, after the GDPR fiasco of Instapaper) I read the equivalent to 18 books in blog posts since then.

Music/events

This year I attended a couple concerts, both in London: Hayseed Dixie and Steve'n'Seagulls. I ended the year listening to a lot of covers by Leo Moracchioli and progressive metal/instrumental metal like Polyphia (which eventually become boring) and Intervals (which are better). All the links in this section are good renditions by each on Youtube

Gadgets

Main one is that Apple finally upgraded the MacBook Air so I can have one with 16GB of RAM, so I got that one as soon as I could. It also implied getting a USB C hub and I also got a power brick, both from Anker, so I could simplify my desktop cable tangles (also several USB A to C and USB C-C cables). In no particular order, I also got

• An acoustic ukulele (soprano) and soon after, I got this electric travel ukulele (concert). To go with it I got an AmPlug (clean)

• A mountie very early in the year. Sadly it doesn't work with my new Air, but with the old one it was a blast.
• A Brydge keyboard for my iPad (heavy, but excellent)
• A SoundCore Motion Q speaker from Anker. We're getting another one soon to get stereo. Excellent.
• A Microsoft Wedge Mouse. I saw a picture and when I saw the price (just 17€) I got one. It works very well, and is very small. And works with a rechargeable AA battery, which I like as form factor.
• A eInk display for my Raspberry Pi Zero W.
• An hourglass. I love hourglasses. You can hear the sand from this one.
• A cardholder. I already had a wallet by  Bellrow, and they deliver quality.
• A Luna Display (so far a bit disappointed with the inability to map correctly my keyboard layout between Mac and iPad, so can't type at all, but the stream quality is very good)
• A bunch of twisty puzzles. I didn't have any 2x2 and I really like it now. Also, magnetic 3x3 are excellent.

• The Little Lisper for my birthday

• An excellent compact fountain pen from Kaweco (the liliput)
• To go with it, I got a 0.7mm pencil from them as well (the classic brass)
• I got also a 2mm titanium pencil from Kickstarter
• A knucklebone from AroundSquare. Excellent for manipulating while thinking
• A new spinning top, the Rain Drop from NWTops (to join last years' Yakima)
• Several (1, 2, 3, 4) cardistry decks. The Mandala is incredibly gorgeous, the Edge is a design I love (I have the deck as well) and when I saw the trainers I knew I wanted a couple, just because.

Down into some net rabbit hole, I stumbled upon a review of Work Clean. I chuckled: a productivity book, philosophizing about how cook’s approach to preparation (mise en place) would fix all our problems? Bring it on, I have a long commute.

Actually, I found the book surprisingly good if you stick to the practical side and discard the hyperbole. As an occasional home cook with a big dose of planning and OCD built into me, I got the point.

Here I’ll share some of the key takeaways I got. I recommend you get the book, by the way.

Commit to being honest with time. Plan daily.

If you are anything into GTD (David Allen’s Getting Things Done), daily planning, weekly reviews and in general a big amount of overhead in your system is probably already a given. But this is probably #1 in my book. If you plan your day (preferably the evening before or at least, before really starting it) you’ll have a better sense of where you stand and how you are doing with your day. Proactive, not reactive.

Immersive focus vs process time

Remember, anything that needs your input or feedback on a project to keep it moving forward is process work.

This is a distinction I had never paid attention to. Process time is work you can “pay in advance�. In a kitchen, this would be starting the oven so it’s hot when you need it, while you chop veggies. Roasting bell peppers: they’ll roast while you prepare something else. Process work was just scrubbing and placing into the oven.

In more mundane, office style work, this would be part of delegating, communicating and unblocking your peers. Start the day unlocking and unblocking the work of others. Communicating, requesting what you need lets you keep the day going.

Always be unblocking

Try to unlock all your projects, goals, missions. The question to ask is what is keeping me from pushing forward? What is the show stopper?

Likewise, try to unblock (specially if you are the blocker) your colleagues. If you work in an Agile environment you’ll be familiar with the standup process, where a daily status meeting is quickly held by members of the team, and is when any blockers (operational, logistic, personal) are raised.


(or you can also take the Always be knolling approach)

Counting mistakes: keeping a mistake log

Kaizen is a fabled Japanese term in the manufacturing/productivity world. It is translated usually as constant improvement. But there is nothing to improve if you don’t know where you are failing.

Keeping a mistake log is the way to continually improve. Find where you failed (no need to be anything life threatening: if you misplace your keys, it’s a failure), log it and strive for improvement. We do this constantly at work: any failure generates a postmortem, and from it we plan to prevent the same mistake from happening again.

Distraction routines

Another interesting approach. What to do when you can’t concentrate or need a break from your current project? In other words, have tasks, projects or goals for when you don’t feel like working more on what you are doing now. Who has not sorted their closet to avoid other work? This is the same, but without avoiding the important work. It’s only a break.

Finisher

Try to finish everything you start. The more projects you have lying around unfinished, the larger burden you’ll feel. If you can’t finish something, prepare the task for picking it up. Write down the next steps, prepare any tools needed. Schedule a follow up session.

The approach to routines

Routines, for the author are not actions (nor habits) but placeholders for actions. You can have a commute routine: you are sitting, bring so and so devices, can do so and so. You can fill this routine with several different actions.

Having a clear idea of which kind of actions can be fillers for each of your routines is key to optimizing your time (if you want to).

Conclusion

It may not have been a groundbreaking read, but I was surprised by the interesting ideas and the exposition. Focusing on cooks was kind of a fun take for a productivity book, and what made me pick what I would have otherwise skipped. Pick it up, you may get some interesting idea.

I’m trying to make these posts a tradition (even if a few days late). I thought 2016 had been a really weird and fun year, but 2017 has beaten it easily. And I only hope 2018 will be even better in every way. For the record, when I say we, it means Laia and me unless explicitly changed.

Beware, some of the links are affiliate links. I only recommend what I have and like though, get at your own risk :)

Work

Everything work related has gone up. More work, better work, more interesting work. Good, isn’t it?

As far as my consulting job in London, the most relevant parts would be:

• Lead a rewrite and refactor of the adserver (Golang) to improve speed and reliability.
• Migrated a batch job from Apache Pig to Apache Spark to be able to cope with larger amounts of data from third parties (now we process 2x the data with 1/10th of the cost).
• Planned an upgrade of our Kafka cluster from Kafka 0.8.2 to Kafka 0.10.1, which we could not execute as well as planned because the cluster went down. Helped save that day together with the director of engineering when that happened.
• Was part of the hiring team, we’ve had one successful hire this year (passed probation, is an excellent team member and loves weird tech). Hopefully we enlarge our team much more in the coming year.
• Put a real time service in Akka in production, serving and evaluating models generated by a Spark batch job.

We also moved offices, now we have a free barista “on premises�. Free, good quality coffee is the best that can be done to improve my productivity.

In April I got new business cards (designed by Laia, you can get your own design if you want, contact her):


I kept on helping a company with its SEO efforts, and as usual patience works. Search traffic has improved 30% year-to-year, so I’m pretty happy with it. Let’s see what the new year brings.

I became technical advisor of a local startup (an old friend, PhD in maths is a founder and works there as data scientist/engineer/whatever), trying to bring data insights to small and medium retailers. I help them with technology decisions where I have more hand-to-hand experience, or know where the industry is moving.

Life

Traveling up and down as usual (2-3 weeks in London, then l’Arboç, then maybe somewhere else…) sprinkled with some conferences and holidays.

Regarding life, the universe and everything, what I’ve done and where I’ve been

• In February we visited Hay-on-Wye again, for my birthday
• In March I convinced Holden Karau (was easy: she loves talking about Spark :D) to be one of our great keynote speakers at PyData Barcelona 2017
• In late March we visited Edinburgh and Glasgow
• In early May I attended PyData London to be able to prepare better for ours. Met some great people there.
• A bit later in May I visited Lisbon for LX Scala, thanks Jorge and the rest for the great work
• And at the end of May, we held PyData Barcelona 2017, where I was one of the organisers. We had more than 300 attendees, enjoying a lot of interesting talks. Thanks to all attendees and the rest of the organising committee... We made a hell of a great conference

• Mid-June, I gave my first meetup presentation, Snakes and Ladders (about typing in Python as compared with Scala) in the PyBCN meetup
• In late June, we visited Cheddar and Wells

• In September I visited Penrith for the awesome (thanks Jon) Scala World 2017. Looking forward to the 2019 edition.
• In early October we visited San Sebastian for the Python San Sebastian 2017 conference. We ate terribly well there (we can recommend Bodegón Alejandro as one of the best places to eat anywhere in the world now)
• Mid-October we visited Bletchley Park. Nice.
• In late October we (Ernest Fontich and myself) submitted our paper Normal forms and Sternberg conjugation theorems for infinite dimensional coupled map lattice. Now we need to wait.
• In November we visited Brussels (Ghent and Brugge too), and took an unofficial tour of the European Council with a friend who works there.
• In December I attended for the second time Scala Exchange, and the extra community day (excellent tutorials by Heiko Seeberger and Travis Brown). Was even better than last year (maybe because I knew more people?) and I already got my tickets for next year.

• In December we attended a wine and cheese pairing (with Francesc, our man in Brussels, and Laia) at Parés Baltà. They follow biodynamic principles (no herbicides, as natural as they can get, etc) and offer added sulfite free wines, too. They are excellent: neither Laia nor I drink, and we bought 4 bottles of their wines and cavas.

Last year I decided to start contributing to open source software this year, and I managed to become a contributor to the following projects:

• Apache Spark
• PureConfig
• ScalaFix
• ScalaFmt (well is still pending review and I need to work a bit more on it... had forgotten!)
• Impromptu (a very small thing in a very interesting library, PR merged as I write these lines, technically it was my first PR in 2018)
• Scala Native (the merit goes to Guillaume Massé, I just wrote a POC)
• Spark-Testing-Base

I wanted to contribute to the Go compiler code base, but didn’t find an interesting issue. Maybe this year.

Learning

This year I didn’t push courses/learning as strongly as last year... Or at least this is what I thought before writing this post.

• In January I enrolled in the Functional Programming in Scala specialisation in Coursera. Only course left is the Capstone project, I will be finishing this January if nothing goes wrong
• During Spring I took Udacity’s Deep Learning Nanodegree Foundation
• In May I took a spoon carving course with Forest+Found (they are the authors of The Urban Woodsman and The Wild Dyer)

• In August I took Apache Kafka Series - Learn Apache Kafka for Beginners, with the rest of the courses in the series waiting for me having more time available.
• In September I tried to learn knitting and lace, but it does not seem to suit me.
• In September I enrolled in a weekly Taichi and Qi Gong course by Mei Quan Tai Chi. Will repeat for the next term
• In December I started learning about Cardistry

Reading

I have read slightly less than last year (36 books vs 44 last year), and the main victim has been fiction. Haven’t read much, and the best... has been the re-read of Zelazny’s Chronicles of Amber. Still excellent. I have enlarged my collection of Zelazny books, now I have more than 30.

As far as non-fiction goes, I have specially enjoyed:

• Essentialism: given how many things I do at once, this book felt quite refreshing
• Rich dad, poor dad: Nothing too fancy, just common sense. Invest on having assets (money-generating items) instead of liabilities (money-sucking items, like the house you live in)
• 10% Entrepreneur: Links very well with the above. Being a 10% entrepreneur is a natural way to invest in your assets.
• The Checklist Manifesto: Checklists are a way to automate your life. I have read several books around this concept (“creating and tweaking systemsâ€�, as a concept) and it resonates with me. If I can automate (even if I’m the machine), it’s a neat win.
• The Subtle Art of Not Giving a F*ck: Recommended read. For no particular reason. I’ve heard that the audiobook version is great, too.

Music/events

This year I have listened mostly to Sonata Arctica. We attended their concert in Glasgow (March) and it was awesome, they are really good live. This was a build up for KISS at the O2 in London (May) which was totally terrific. And followed by Bat Out of Hell (opening day!) in London. It was great, and probably the closest I’ll ever be to listening Meat Loaf live. Lately I’ve been listening to a very short playlist I have by Loquillo, and also Anachronist.

We have also attended a performance by Penn and Teller (excellent), and IIRC we have also watched just one screening: The Last Jedi (meh, but Laia liked it).

Gadgets

This year I have gotten hold of a lot of gadgets. I mention only the terribly useful or interesting

• From last year, iPhone 7 “smallâ€�. Not happy with it. Battery life sucks big time, I got a external Mophie battery for it.
• Mid-year: Apple Watch Series 2. Pretty cool, and more useful than I expected.
• Late this year: AirPods. THEY ARE AWESOME
• Laptop foldable cooling support. While taking the deep learning course my Air got very hot, and I needed some way to get it as cool as possible.
• Nutribullet. My morning driver is banana, Kit Kat chunky, milk, golden flax seed, guarana.
• Icebreaker merino underwear. I sweat a bit, and get easily chaffed on the side of my legs (where it contacts my underwear). Not any more: not only is wool better at sweat-handling, but the fabric also feels better on the skin. And not, does not feel hot in the summer.
• Double Edge Shaving. I hated shaving (and actually just kept my beard trimmed so it was never a real beard or a clean shave...) and this razor (not this one specifically, safety razors are pretty much all the same) has changed that. Now I shave regularly and enjoy it a lot (together with this soap and this after shave balm)
• Chilly bottles. They work really well to keep drinks cold or hot. I’ll be getting their food container soon.
• Plenty of lightning cables. You can never have enough of these. I also got this great multi-device charger, ideal for traveling.
• Compact wallet. I’ve been shown the ads so many times I finally moved from my Tyvek wallets to one from Bellroy. It is very good.
• Book darts. Small bookmarks that don’t get lost, look great and can double as line markers. Also, they don’t add bulk to a book, so you can have many in the same book without damaging it at all. They are great, I’m getting a second tin in my next Amazon order of stuff.
• Two frames from an artist I saw showcased in our previous office (they had exhibits downstairs). Blue Plaque Doors and Hatchard’s, by Luke Adam Hawker.

On the fun side, I also have a spiral didgeridoo, a proper Scottish bagpipes, a Lego Mindstorms I have not played with yet :( and an Arduboy. Oh, and a Raspberry Pi Zero Wireless.