Sign in Subscribe
9 min read activiti7

Activiti Cloud: CI/CD approach for Java Libraries and Beyond

This blog post describes our approach to CI/CD for our core Java Libraries. This includes the Process Engine, new APIs, and new Spring Boot Starters for the Runtime and Modeling Services of our Cloud Native Building Blocks.

We strongly believe that these practices will enable us to scale our development process to build a stable set of libraries and services. Due the complex nature of the tools that we are building, having a stable base and a core set of practices to follow will enable us to sustain a constant pace of development and delivery.

We have separated Core Java Libraries from Spring Boot Starters due to the relationship with Spring Boot and Spring Cloud respectively. We have also separated the Runtime and Modeling Services due the nature of the changes in these projects and to make sure that they can be released independently.

The following sections describe the main conventions that we have adopted and how the flow of changes will be propagated during a normal development cycle.

Java Core Libraries

We have worked hard to make sure that we enforce consistency across repositories, but at the same time ensure that each repository can evolve independently from the others. We are working under the premise that a PR merged to the develop branch will end up with a new release for the artefacts contained in the repository.

activiti-core-repositories.png

For this reason we have switched to use fixed version releases instead of using SNAPSHOTS. Every PR merged into develop will cause the patch version (SemVersion: Major.Minor.Patch) to be incremented and released to our Alfresco Nexus repository.

We are also creating BoMs (Bill of Materials) in each repository and now each repository has the same relative parent. For Activiti Core related repositories that is Activiti Build.

In practice, this means that each repository will have a Parent Pom which is used parent for all the modules inside that repository and a BoM which is used to declare all the dependency management for consumers of the projects contained in the repository. Each Parent Pom is used to contain and propagate only maven plugins and to define the internal maven dependency management.

Even if the previous diagram looks simple, there are a lot of details that are important to achieve a truly CI/CD approach. Here are some of the highlights:

  • Activiti Build (org.activiti.build)
      • Parent Pom (pom.xml): this Parent Pom is used as Parent of all the Core Activiti Projects. For that reason, this Parent Pom includes all the shared Maven Plugin Configurations.
      • BoM (activiti-parent-dependencies/pom.xml): this parent BoM includes the dependency management of Spring Boot which is going to be shared by all the children.

Other Conventions

We use properties (maven properties) to make references to artifact versions.

Each repository will have a property associated to it and the name of this property will follow the following convention: <{repo-name}.version> </{repo-name}.version>

Examples of this properties tag: <activiti-build.version>, <activiti-api.version>,<activiti.version> and <activiti-dependencies.version>. Notice that the repository names match the property text.

In each repository you will find:

Changes propagation and Continuous Delivery approach

While working with these projects we want to ensure that we don’t block streams of work to produce milestone releases. In other words, we don’t want to freeze repositories to do releases. We want to allow the constant evolution of each repository independently of the release plans.

In order to achieve this dynamic, the following diagram describes the "updatebot" configurations to generate PRs for the changes in each repository.

updatebot-changes-activiti-core.png

This flow shows that for every change (that generates a release) in the Activiti Build repository, all the children will receive a PR with the updated version after the change is verified. Due the nature of Activiti Build (just a parent pom defining plugins and BoM definition) the rate of change at the top level will be quite low.

At this point, Activiti API will generate a new version of itself and will create a PR on Activiti Core Common, Activiti (Core) and Activiti Dependencies updating its own version and its own BoM.

For these repositories, Activiti Dependencies (which only contains a BoM) acts as a version aggregator for consumers to use. Activiti Dependencies serves as gate to control, test and decide when to release a new version set for all these projects. Based on recent findings, we can also use the Maven Enforcer plugin to enforce version convergence (thanks to @igdianov). This enforcement can be monitored by the pipeline to only merge the PRs when the version convergence is valid. Once we manage to release Activiti Dependencies, our consumers (in this case Activiti Examples) will be notified that a new version is available for consumption which can be automatically merged.

Activiti Cloud Runtime: CI/CD approach for Spring Boot Starters

For Activiti Cloud Runtime Spring Boot Starters we have the following dependency chart

activiti-cloud-repos

For Activiti Cloud each repository has a single parent Activiti Cloud Build (arrows were removed from the previous image to reduce the complexity of the diagram). Following the same approach as Activiti Build, the parent only includes the relevant plugins that apply to all of its children. Activiti Cloud Build also defines a parent/master BoM which in this case defines the Spring Cloud dependency (Finchley.SR1 at the time of writing this document).

An important detail to notice here, is that Activiti Cloud Build doesn’t depend on Activiti Dependencies. The main reason for this is that not all the Activiti Cloud building blocks depend on Activiti Core libraries, for that reason the Activiti Dependency is not introduced at this level.

For Activiti Cloud we also have an aggregator project called Activiti Cloud Dependencies which collects all the individual repositories BoMs and it is designed to be used by consumers of our services. Activiti Cloud Dependencies is the main link between the Spring Boot Starters world provided by Activiti Cloud and services that can be packaged as Containers (Docker images).

After Activiti Cloud Service Common repository all the children are independent so they can evolve without being blocked by their parents. For development purposes, developers changing one service can always reference the fixed version of the service to test the changes in live environments. In other words, every change in Activiti Cloud Service Common or above will be propagated to all the children, but changes in the child will only promote the change to the aggregator (Activiti Cloud Dependencies BoM). This enables us to do A/B testing of new features without fully releasing a new Labeled Activiti Cloud Dependencies for external consumption.

The repositories covered by Activiti Cloud Spring Boot Starter projects are as follows:

Starting with the parent (shared) projects:

Depending on these projects are the Activiti Cloud Spring Boot Starters:

The Spring Boot Starters generated by these repositories are:

  • Activiti Cloud Audit @EnableActivitiAudit
  • Activiti Cloud Query @EnableActivitiQuery
  • Activiti Cloud Runtime Bundle @EnableActivitiRuntimeBundle
  • Activiti Cloud Connector @EnableActivitiCloudConnector
  • Activiti Cloud App Service @EnableActivitiAppService

This means that if you add Activiti Cloud Dependencies (Master Cloud BoM) you will be able to include any of these starters in your Spring Boot Application.

Activiti Core and Activiti Cloud dependencies

It is important to understand how things relate dependency wise to understand where changes will impact projects down the chain. The following chart represents the dependency of modules between Activiti Core and Activiti Cloud:

activiti-core-and-activit-cloud-rel.png

It is important to notice that Activiti Cloud Runtime Bundle is the only one consuming Activiti Core Dependencies. This means that Activiti Cloud Query Service, Activiti Cloud Audit Service and Activiti Cloud Connectors will be only impacted by changes in Activiti Core Common and Activiti Cloud Service Common.

It is also important to understand that Core Common and Cloud Service Common are covering cross cutting concerns such as Security, Tracing, Identity, Alfresco DBP conformance, Rest Docs, Tests, Metadata, etc. and these cross cutting concerns modules shouldn’t change as often as the service themselves.

Activiti Cloud Modeling: CI/CD approach

In the same way as Activiti Cloud Runtime, the Activiti Cloud Modeling repositories produce Spring Boot Starters related to the modeling backend services.

activiti-cloud-modeling.png

Spring Starters

  • Activiti Cloud Process Model Service @EnableActivitiProcessModelService
  • Activiti Cloud Apps Service @EnableActivitiCloudAppsService

Release Process, Example Docker Images and Acceptance Tests

Our release process only covers releasing Maven artifacts to Maven Central (Activiti Core and Activiti Cloud), but we do have a set of example services (which generate Docker Images) that need to be tagged and tested against the released artifacts.

The following diagram describes the repositories involved:

activit-cloud-examples.png

As you can see, Activiti Cloud Application services, and Activiti Cloud Runtime Acceptance Tests are all consumers of Activiti Cloud Dependencies BoM. This makes them perfect examples of how you should build your applications, how to include the aggregator BoM and how to configure the Starters for your Spring Boot Applications. They also include Docker Files, Kubernetes Descriptor Files and HELM Charts definitions.

Activiti Cloud Infrastructure is just used for the examples and these services doesn’t depend on the Starters or any Activiti Cloud Dependencies. We are just bundling them together for our own consumption (Integration Testing and Example Deployments).

Repositories involved in runtime examples which are consuming Activiti Cloud Dependencies master BoM:

Activiti Cloud Modeling examples follow the same approach:

activiti-cloud-modeling-examples

Modeling Repositories which creates docker images are:

Summing up

We are planning to extend our release procedures to include all these example layer which includes creating and tagging docker images as well as publishing HELM charts. While we know that this is a considerable amount of work, we believe that these practices and convention based approach will permeate to teams building solutions with these building blocks, ultimately providing a SaaS approach to deliver solutions.

Activiti Cloud HELM charts are still the recommended way to deploy our components and the Activiti Cloud Full Example HELM chart deploys all the services as described in the previous sections.

If you have question, want to contribute to the project or want to make it better please get in touch via Gitter or Github Issues.

Published by:

Mauricio Salatino

You might also like...

Nov
18

🇺🇸 KubeCon NA 2024 recap

Ok, but what's all the fuss about it? For me, there were three topics that I was interested to see how they develop: - AppDev & Dapr Graduation - Platforms - Extending Kubernetes (Multi-cluster, multi-tenant, LLMs, Multi cluster Batch Jobs)
8 min read
Nov
08

🇺🇸 Next week: KubeCon NA 2024

I am extremely eager to meet all the amazing folks attending KubeCon NA 2024 in Salt Lake City USA. I am really looking forward to meeting as many practitioners as I possibly can to have amazing conversations.
5 min read
Nov
05

🇬🇧 KCD UK - London Oct 2024

I had the pleasure of presenting with Abby from Syntasso about the intersection between platform and development teams. We focused on three key challenges development teams face when working with complex infrastructure. https://www.diagrid.io/blog/when-platform-teams-meet-developers
2 min read
Jul
03

KCD Munich - Platforms, Cloud Native and Communities

I had an amazing time at KCD Munich this week. KCDs are always fun because you get to see where the communities are, which topics are hot and you meet a lot of project maintainers. Kudos to the organizers who did an amazing job at having an amazing third edition of the conference.
2 min read
Jun
17

🇪🇸 DevBCN / KCD Barcelona 2024

Another DevBCN conference done! This year was special because the first KCD Barcelona happened. I started the KCD Track on
3 min read