What’s the story Rory?
Is that the time of the year that you start to reflect about what you did in this year and what you want to do next year. I definitely manage to do a lot of nice things this year, but I want to highlight some ones;
I am really back to Inline Hockey; I managed to break my hand, at the end of April, inline skating around the city; I did some presentations in big conferences this year and I completed one year working with Kotlin. My current joke is that I work with the three K; Kotlin, Kafka and Kubernetes. I did a lot of Raspberry Pi projects and kind of explored Virtual Threads a lot.
Picture 01: Inline Hockey
But, anyway, the idea here is not a retrospective blog, where I’ll focus on what I did this year, but just a review of the things that I saw, learned or found as a trending topic, and if you already know me, you know that I basically write only about Java, Kafka and Raspberry Pi, and because of that I’ll divide this blog into those three sections.
Virtual Threads was the main trending topic of the year, even the people that aren’t developers are talking about it. It is still in early stages and could change a lot until Java 21, the next LTS (Long Term Support) version that will be released September next year. But I think this brings more visibility to Java.
Project Loom aims to drastically reduce the effort of writing, maintaining, and observing high-throughput concurrent applications that make the best use of available hardware.
I believe that more than understanding and knowing how to use Virtual Threads, the main benefit will come once the servers and the frameworks implement for you.
Another topic that got my attention was about Data oriented programs, despite this was not strong in the data community, in the Java community a lot of articles and blogs were published in this topic. Looks like a lot more will come, and I’m kind of curious to see this both in the java world and in the data world. Is definitely something to keep an eye out for in 2023.
The biggest surprise for me was the Graalvm company announcement that they will give OpenJDK the native compile. With that, the Graalvm will follow the Java cadence to release every six months a new release.
Adoption of Native Java will also continue to grow, and projects like Leyden and proposed Project Galahad will be trending next year.
Microservice of course is still a relevant topic and a lot of nice content was generated this year. I still recommend checking out the Microservices patterns book.
And talking about book, I saw this two books as the trending ones;
Modern Software Engineering – Doing What Works to Build Better Software Faster
I always joke that you should follow and get engaged with your local communities and I really recommend you to check out Foojay, Friends of OpenJDK and Soujava, the biggest Java community around the world.
It is not a surprise to say that Coroutines is one of the topics that interests me a lot.
I found this blog post with a really nice history of Coroutines and some comparison and analysis with Java, Project Loon and Virtual Threads.
I see Kotlin winning some traction, and it will be nice to see what will happen with Kotlin and Java in the next two to three years. Not only that, but I have the feeling that history will repeat itself, the same thing that happened with Scala less than ten years ago.
Kotlin vs Java will be a good topic to be aware of.
Scala is not trending anymore, even in the Java and Data community.
Scala 3.0 has good changes, but I still think that Scala created a lot of problems with the version migration and this kind of decreased the Scala traction.
I still believe that every Software Engineer should learn Scala as a Functional paradigm language. Once you learn Scala you never will program the same way.
In my twenty years of experience, I work close to the data world for the most part. I kind of joke a lot that I’m the Software Engineer in a Data Engineer team.
Apache Kafka Raft (KRaft) is the consensus protocol that was introduced to remove Apache Kafka’s dependency on ZooKeeper for metadata management.
The Apache Software Foundation has released Apache Kafka 3.3.1 with many new features and improvements. In particular, this is the first release that marks KRaft (Kafka Raft) consensus protocol as production ready. In development for several years, it was released in early access in Kafka 2.8, then in preview in Kafka 3.0.
KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum
Every new Kafka release a lot of new nice things come and definitively is one thing to be aware of.
Another thing that got my attention was the release of a Low code platform by confluent. This will be interesting to follow.
Data Mesh was the main trending topic of the year.
Data Lake House is another trending topic that grows a lot this year. Many data platform vendors pitch the lakehouse vision today. That’s the same story as the data lake in the Hadoop era, with few new nuances. Put all your data into a single data store to save the world and solve every problem and use case. In the last ten years, most enterprises realized this strategy did not work. The data lake is great for reporting and batch analytics, but not the right choice for every problem. Besides technical challenges, new challenges emerged: data governance, compliance issues, data privacy, and so on. Data streaming increasingly becomes the real-time data fabric between all the different data platforms and other business applications leveraging the real-time Kappa architecture instead of the much more batch-focused Lamba architecture.
Decentralized data mesh with valuable data products and this will be the next trending topic.
Apache Hudi, Apache Iceberg and Delta Lake have everything to grow more and more. Those are the products to pay attention to.
Of course, ETL vs ELT and other variations, were in the news and DBT falling in popularity.
DuckDB was a surprise that got a lot of traction.
Another trending topic was the ChatGPT & DALL-E2
Rust got a lot of traction, and it will be nice to check the growth like Kotlin.
Rust vs Python
And talking about Rust, a project that grew a lot in the last three years was Red Panda. Is deftly something that you need to check out.
A book to check out is the Fundamentals of Data Engineering and by coincidence Red Panda is giving the book for free.
Fundamentals of Data Engineering
Following my joke that you should follow and get engaged with your local communities, I recommend checking out Data Hackers and Confluent on Slack.
It is kind of cliche, but SQL was the top programming language of 2022.
For me, this year was about Raspberry PIco and Raspberry Pi Zero 2, loads of articles and blogs with nice content and some guide tutorials as well.
PI4J v2 was a nice surprise as well. It changes how it works and definitively is something to check it out. A bonus point here is that now there is a Kotlin version as well.
A book to check out is the Open circuit
This was the year of the Supply Chain problems, but in the end we have some hope.
Those were the topics that I found interesting this year, and of course I probably forgot some things.
I tried to cover my main three areas, and there are a lot of things to cover here. Don’t panic if you don’t know those things. Don’t try to understand everything, and just focus on your topics of interest.