Speed up microservices development

One service, one repository right ? I want to challenge this common belief with a concrete example. In my case, I was able without compromise, to deliver 10 times faster by breaking this preconceived idea.

Few words about me and my working environment first. I am a software engineer, graduated in 2018 from Télécom Physique Strasbourg and University of Strasbourg with major in networks and IoT. I am the co-funder of Botcrypto.io and I worked for two years at Orange as a lead developer for network services.

You build it, you run it !

This is the mantra I deal with since I began my professional career. My favorite tools for that so far are Gitlab, Python and Docker (Swarm 😅). So what I’m going to present is based on my experiences using these tools.

First I am going to expose you the “common way” I know so far to organize the development and the release of microservices stack(s). Then, in the next section, I will present you my suggestion and show you how it’s different.

The scenario

Let’s say we develop a new application and we will use microservices for that. We will need to develop two services, let say a front “A” and a backend “B” with both dedicated scope of features for our clients.
These services will expose HTTP APIs. The service “A” will need for some of its endpoints to request the service “B”.

What matters for the microservices development ? IMHO

0) The 12FA and good practices (docs + tests + automation) — even if it’s not microservices btw 😉
0-bis) Avoid as much as possible to develop microservices in different techno as mastering a single techno is already a huge effort.

1) Services “A” and “B” development can be split to dedicated teams.

2) Services “A” and “B” can be deployed individually whenever there is no breaking change in “B” service (so the communication “A” — “B” isn’t broken). For that we want to set the version for each service deployed.

3) DRY: Don’t Repeat Yourself. So the app will be delivered sooner and maintenance is simplified (cf. 0-bis).

One service, one repository (the common way)

We will have two repositories dedicated for services “A” and “B”, and eventually a third one for the deployment in different environments (staging / production).

Repositories for “A” and “B” both have these components:
- A pipeline for automation that generally ends with release(s) to a registry.
- As there is releases, there is packaging, so one package with the code base of the service.

What takes a lot of time with that organization ?
Mainly two things…

• “B” API is a dependency for the service “A”, any change in the “B” API imply a development and new release for the “A” service. If it is not the case in your architecture, please let me know 🙏
  In other words, any application’s feature in the scope of the service “B” will probably require two releases (“B” then “A”) before testing on the staging environment ! If you’re lucky and everything is fine on the first try (and we know that staging is here because it doesn’t happen), the feature can be deployed for our clients. If tests on staging fails, you’re good to probably make two new releases before the next try… ⏰
  Even with automation, making a release is time consuming and is a very rigorous exercise, especially if many services are implied.
  This issue is even more important when projects starts because there is generally many changes in the APIs !
• Repetitions between projects. Packaging, releases, pipelines, … If both “A” and “B” team follow common practice (that is a good idea too), a major change in common practice will impact all repositories. We can minimize the repetitions but in the next section I show you how it won’t be necessary.

When the number of services increase, the complexity and the time to release increase exponentially… 😕

Use a single repository !

I hear my old managers crying reading this title 😝
How can it be a good idea ?! It’s a monolithic application ! It’s a bloatware! Teams can not work with that ! lol…

I was developing an application with “only” 3 microservices. With the first approach, bringing a new feature to the app involving a release of all services required me at the very very least 2 hours. But It was 8 hours in average as a mistake in a service involved new releases…
After merging the repositories, a new feature was more 1 hour in average. And what did I loose by doing that ? Nothing… And even better, I get rid of many repetitions between projects and my tests were much more simple ! 😃

How did I merge these repositories ? This was possible because all services were developed using the same techno (python).

1. Each service is not anymore a package but a sub-package. The difference is almost just one new directory created. The team work is absolutely not impacted as long as it is clear that there is one sub-package per service so they MUST NOT be imported in other sub-package (not too difficult to understand I hope).
2. The pipeline produce only one package, one docker image versioned. What change between services is only the command used to start the service. The versions for each service to deploy are in variables, so we can deploy services individually (and rollback them individually) exactly as before.
3. For tests, we can use every sub-package ! That is a huge huge +… Because we didn’t need to wait for the release of dependencies to test the code, the dependencies are already right there !
   Let’s re-use our example with services “A” and “B”. Before, in a feature of “A” that use “B”, I needed to wait for “B” to be released so I can actually test the behavior of “A” feature. Any slight change in “B” API imply to make a new release of “B” before re-test the behavior of “A” feature…
   This is when most of the time was lost. Now, the team “A” can in one shot make a merge request with the new feature tested eventually including changes for the “B” team→ it’s far better than in the previous model for the team work IMO.

All the points that matters for the microservices development mentioned previously are verified. It’s faster, it’s better for team work, we get rid of useless releases, we get even the deployment at the same place. So worth it or not ?.. *La question elle est vite répondue* 😜

Conclusion

I have a very little experience building and deploying microservices stacks, so what I wrote may sounds like bullshit to you and you’re probably right 😅

So far I have tested my model with 5 services build and deployed in parallel and I’m very pleased with that. I saw new projects starting making what I think is a mistake, so this is why I wanted to share with you my point of view.

I will be very happy to read your comments !

#Dev #Ops #µServices