In this post I'd like to reflect over some things I've learned in my 10+ years working in Information Technology, and the things that helped me becoming the (DevOps) engineer I am today.

Read

You're reading this sentence, so you're already on the right way. 😁

In IT, most of our knowledge has been written down and is easily accessible for everyone with access to the internet: Wikipedia, personal blogs, university websites and many other pages especially from the pre-social media time.

I love to read blogs. To keep up with them, I use Newsblur, a hosted RSS reader that has been an reliable tool for me for nearly 10 years now (since Google Reader died).

RSS has many advantages over social media-based aggregators:

• You can carefully select your sources and have full control
• You'll receive everything from them in a chronological order
• No FOMO: If you're busy for a few days and can't keep up, your feed reader will keep everything and provide it when you're ready.

Apart from that, reading documentations from start to end can be quite an interesting experience. For example, the Python documentation has a HOWTO section with chapters that explain basic and advanced feature more into detail, like how to implement descriptors (the "magic" behind most Python ORMs).

Write a lot of code

In German we have a saying: "Es ist noch kein Meister vom Himmel gefallen" (no master has ever just fallen out of the sky).

To become good in a craft, and building software is also a craft, you need to practice it. You can read about it all day in theory, but you'll only find true understanding and an actual feeling about it when you consistently do it.

These things don't necesarily need to be useful, the only thing that matters is that it's something that motivates you: Build a small game, automate something you else do manually or a tiny website just for yourself.

I started teaching myself PHP when I was 15. Back then, I had an old computer in my grandparent's attic without direct access to internet - anything I needed from the web had to be transferred from a different place using burned CDs. So I got things I could use offline: XAMPP and the PHP manual compiled as Windows Help File.

Over many weekends, I read through the entire PHP manual, learning about functions and many obscure features of the language even seasoned developers rarely ever heard of (not a bad thing though). I also downloaded many open source applications, my favorite being phpBB, and studied their code and tried to change it. Later I started my own projects and at some point were even able to publish them - unfortunately not much from that time has survived moving around multiple times.

Setup a homelab

If you cannot get the necessary practice at work, or you have spare time and some space at home, having a homelab is a great way to learn about and discover many aspects in IT operations.

This includes setting up operating systems and networks, and allows you to play in a safe environment without having to fear breaking business-critical production systems.

This doesn't need to be on a level to rival even medium sized company networks like many setups on /r/homelab, in my case I acquired a refurbished HP EliteDesk 800 and installed Proxmox on it, an easy to use Open Source virtualization solution.

Using that, I've been able to play with a lot of different tools, like:

• OPNSense, a BSD-based firewall distribution
• Talos, a Linux+Kubernetes distribution
• Rundeck, an automation tool we also use at work

Learn to write documentation

The software development world is going more remote and async, thanks to both the pandemic and more diverse teams that are distributed over the globe. This aspect requires you to be an effective communicator using text.

My favourite motto in that regard has been "Explicit is better than implicit" from the Zen of Python.

If you wanna practice, how about:

• Write down how you set up your own infrastructure, let it be your computer, Raspberry Pis or personally used servers. You can do that using Word, Google Docs or a tool like mdbook.

• Starting writing a blog. Even if it's just for yourself.

Another important aspect is to visualize things. Take your time to learn a tool like diagrams.net, and not just to dump information, but to also make it visually appealing. I've used PlantUML before, but stopped using it as results are often ugly and confusing, and it's a lot of work to make them look somewhat decent.

Here's a random example illustration from my Blog. Images are often easier to read than pure text.

Keep a work diary

Something I've done most of my entire career: Writing a diary.

This includes:

• What I did on a certain day
• If something special happened or I did something important
• For meetings, what was discussed (not for standups or other "Walk the board"-likes)

This doesn't have to be fancy prose, my notes are just a long collection of (nested) bullet points. In the end, they're just for yourself.

I general just use a set of folders managed using VS Code, but you can also use tools like Obsidian if you want more comfort.

While this sounds like a lot of work, you'll quickly get used to it, and especially later in your career, when your time share shifts from code to meetings, it's great to have a place to dump the information without losing or forgetting it.

On some days I spend 5+ hours in videofalls. Without my diary, I'd totally lose track which meetings I joined even the day before. Or maybe I'm getting old 😅

How about starting right now? You're just a folder and a Markdown file away from your first entry!

Don't stay at the same place for 5+ years

Especially at the start of your career, I'd recommend to switch department or even companies at least every 3-5 years: This way, you can see completely different environments and connect with new people that will challenge your previous assumptions about the world.

Here's a very unscientific graph about my learning curve at a job:

When starting a new position, you'll generally have a ramp up phase where you have to learn a lot about the new environment and the people working there. After a while, this slows down as the rate of new things decreases. After some years, you've created a cozy comfort zone and might start to resist leaving it.

Also in IT, it's the best way to level up your salary and other benefits.

I've seen people coming from university and staying at the same place for 10+ years. They never had to leave their comfort zone, which lead them to never question or update their skillset. Their mindset wasn't open or flexible either. 😔

Be curious about people and their work

Especially when joining a new company, it's essential to quickly understand it's culture and how to navigate it internally. Each company over time develops an internal culture, and the key to success is to understand it, as well as the reasons why things are the way they are.

Every rule has a history. Red tape doesn't magically appear, it's often the result of things that went wrong. The entire regulation in aviation and financial is based on the failures of the past.

To achieve this, schedule virtual or real coffee breaks with people from different departments and ask them about what they do. Most people love talking about themselves :)

Some companies also have set up tools like "Icebreaker" bots to randomly match people from different places together. Embrace it if available.

Be open about different backgrounds and mindsets

In my time working for a Cologne-based company, there was a classic Enterprise IT, as well as multiple software development teams.

And they didn't really like each other.

A major friction between software development and operations is their different mindsets and business requirements:

• Development often goes by "Move fast and break things":
  The business needs new features, better today than tomorrow.

• Operations likes to say "Never touch a running system":
  Their major metric is uptime, and any change to a system risks that.

In a more concrete example: Development wanted an SFTP server. So they set one up in the cloud. Then operations came along: "Hey, what about security? And updates? And documentation?"

All valid questions, but development was upset that operations was slowing down things again.

Because I started my career in a similar place than the Enterprise IT, I understood their mindset and could understand their objections. But as I also worked for a long time in software development, so I could also speak their language as well.

In the end, I somewhat became an ambassador for both sides that helped ease misunderstandings and align both sides to deliver results.

Moral of the story: Instead of seeing others as evil blockers, try to understand where they come from and what their goals are, this can help you achieve your own.

Think ahead

When doing large or complex tasks, it can always help to imagine every step on the way, writing them down and then acting on them - with course corrections if necessary.

Concrete examples: A few years ago, I had to convert a MySQL database from utf8 to utf8mb4.

At this time, I was working on a large platform that required a constant uptime, as it was hosting hundreds of websites and many business-critical application for customers. Because of this, the migration that to work as smooth as possible.

To achieve this, I did two things:

1. I tested the process in a separate environment over and over again.
2. I made a plan for the migration day.

The latter took a bit of time, as for every step I needed to do, I asked myself:

• What are the dependencies? If I do X, what else will happen?
• Who will be affected by this, who do I need to talk to?

For example, shutting down the infrastructure at the start:

• Migration the database breaks every service using it, so they need to be stopped first. For some services, this can take some time, e.g. batch processing systems.
• These powered off services affect our customers, so we need to chose a time where it's less critical (Friday late evening) and we need to tell all customers 2 weeks before.
• I also had to ensure that our datacenter's support was available at that time, in case something went wrong.

For the migration itself, it had to plan ahead: What if something goes wrong? Can I continue, do we need to abort? If we abort, how can I recover the database to the previous that? Who does that, and how long does it take?

At the end, I had a clear plan what needs to be done in what order, and the entire migration went through smooth - and no customer complained.

Milk the mentor

A suggestion by a friend of mine.

Try to find mentors in your company or outside, they can boost your career in incredible and unexpected ways:

• Experts in their domain can you help better understand topics.
• Pack leaders ^[1] can help you better understand people and how to work with them.
• Strategists ^[2] can help you achieving your goals effectively.

Ask them questions. Let them review your ideas and your code. Pair program with them. Ask them even more questions.

And let them ask you questions to force yourself to fully understand and reflect your own knowledge.

Also, teaching things just by yourself can also lead to going into the wrong direction or picking up bad habits. A good mentor can detect and correct that.

Having Junior developers can also help more experienced devs to grow by learning how to teach: Communicating thoughts and sharing knowledge is essential, especially when going up the career ladder.