How to Use Event Hubs and Service Bus Testcontainers in Spring Boot

Introduction

Testcontainers has released dedicated modules for Azure Event Hubs and Azure Service Bus emulators.

These modules greatly simplify the use of these emulators compared to the custom solutions I previously implemented using generic containers and docker compose in:
1. Using Azure Event Hubs Emulator as a Test Container
2. Using Azure Service Bus Emulator as a Test Container

In this article, I’ll show how to use these new modules in Spring Boot by applying the singleton containers pattern.

Singleton Containers Pattern

With the singleton containers pattern, containers are started only once and reused across all test classes.

This approach is especially valuable for Event Hubs and Service Bus emulators, as each depends on additional containers. Starting and stopping these dependencies for every test class—or worse, every individual test—would be highly inefficient and slow down the entire test suite.

In our setup, all required containers are started in a base test class, and all other test classes extend it. This ensures containers are initialized only once, while their configuration remains available everywhere.

We then inject the container connection properties into the Spring Boot application context using @DynamicPropertySource, which lets us define Spring properties dynamically at runtime based on container output.

Using Event Hubs Module

Dependencies:

<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>azure</artifactId>
    <version>1.21.3</version>
    <scope>test</scope>
</dependency>

Base class (on GitHub):

abstract class AbstractIntegrationTest {
    public static final String AZURITE_IMAGE = "mcr.microsoft.com/azure-storage/azurite:3.33.0";
    public static final String EVENTHUBS_IMAGE = "mcr.microsoft.com/azure-messaging/eventhubs-emulator:2.0.1";

    private static final Network network = Network.newNetwork();

    private static final AzuriteContainer azurite = new AzuriteContainer(AZURITE_IMAGE)
            .withNetwork(network);

    private static final EventHubsEmulatorContainer emulator = new EventHubsEmulatorContainer(EVENTHUBS_IMAGE)
            .acceptLicense()
            .withNetwork(network)
            .withConfig(MountableFile.forClasspathResource("eventhub-emulator-config.json"))
            .withAzuriteContainer(azurite);

    static {
        azurite.start();
        emulator.start();
    }

    @DynamicPropertySource
    static void registerPgProperties(DynamicPropertyRegistry registry) {
        registry.add("spring.cloud.azure.eventhubs.connection-string=", emulator::getConnectionString);
        registry.add("spring.cloud.azure.eventhubs.processor.checkpoint-store.connection-string", azurite::getConnectionString);
        registry.add("spring.cloud.azure.eventhubs.processor.checkpoint-store.endpoint", () -> String.format("http://localhost:%d/devstoreaccount1/", azurite.getMappedPort(10000)));
    }
}

Using Service Bus Module

Dependencies:

<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>azure</artifactId>
    <version>1.21.3</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>com.microsoft.sqlserver</groupId>
    <artifactId>mssql-jdbc</artifactId>
    <version>12.8.1.jre11</version>
    <scope>test</scope>
</dependency>

Base class (on GitHub):

abstract class AbstractIntegrationTest {
    public static final String SQL_SERVER_IMAGE = "mcr.microsoft.com/mssql/server:2022-CU14-ubuntu-22.04";
    public static final String SERVICEBUS_IMAGE = "mcr.microsoft.com/azure-messaging/servicebus-emulator:1.1.2";

    private static final Network network = Network.newNetwork();

    public static MSSQLServerContainer<?> mssqlServerContainer = new MSSQLServerContainer<>(SQL_SERVER_IMAGE)
            .acceptLicense()
            .withPassword("yourStrong(!)Password")
            .withCreateContainerCmdModifier(cmd -> cmd.getHostConfig().withCapAdd(Capability.SYS_PTRACE))
            .withNetwork(network);

    public static ServiceBusEmulatorContainer emulator = new ServiceBusEmulatorContainer(SERVICEBUS_IMAGE)
            .acceptLicense()
            .withConfig(MountableFile.forClasspathResource("/servicebus-emulator-config.json"))
            .withNetwork(network)
            .withMsSqlServerContainer(mssqlServerContainer);

    static {
        mssqlServerContainer.start();
        emulator.start();
    }

    @DynamicPropertySource
    static void registerPgProperties(DynamicPropertyRegistry registry) {
        registry.add("spring.cloud.azure.servicebus.connection-string", emulator::getConnectionString);
        registry.add("emulator-ampq-port", emulator::getFirstMappedPort);
    }
}

Summary

Use Testcontainers’ Azure Event Hubs and Service Bus modules with the singleton containers pattern to speed up Spring Boot integration tests.

Start emulators once in a base test class, inject their connection strings with @DynamicPropertySource, and extend that class in your tests for fast, consistent, and fully local testing.

For examples of extending and using the base class visit the GitHub repository: azure-testcontainers-in-spring-boot

Cloud Foundry: How to Autoscale a Spring Boot Application Using Custom Metrics

I have created a demo Spring Boot project showing how you can scale a Spring Boot application on Cloud Foundry using custom metrics.

Custom metrics scaling allows you to scale applications based on your own defined metrics, such as job queues or pending tasks, aligning scaling with business needs and workload patterns.

This demo shows an application that is autoscaled based on the custom metrics it provides to the autoscaler.

cf custom metrics autoscaling


For code and more details visit the GitHub repository: https://github.com/mirkoiv/cf-custom-metrics-autoscaling-demo

Concept

An Autoscaler service exposes a custom metrics URL and an endpoint for sending metrics.

The URL is defined in VCAP_SERVICES:

{
  "autoscaler": [
    {
      "credentials": {
        "custom_metrics": {
          "mtls_url": "https://autoscaler-metrics-mtls.cf.example.org"
        }
      }
    }
  ]
}

and the endpoint is:

{{mtls_url}}/v1/apps/{{appGuid}}/metrics

where appGuid is you application id defined in VCAP_APPLICATION[‘application_id’]

To authenticate, the app uses the X.509 certificate and private key provided in the environment variables CF_INSTANCE_CERT and CF_INSTANCE_KEY.

The custom metrics payload is:

{
  "instance_index": "{{instance_index}}",
  "metrics": [
    {
      "name":"my_counter",
      "value": 7
    }
  ]
} 

where instance_index is available in the environment variable CF_INSTANCE_INDEX.

Using Azure Service Bus Emulator as a Test Container

This sample project shows how to use the Azure Service Bus emulator as a test container for local development and integration testing.

The Service Bus emulator, like the Event Hubs emulator, doesn’t have a dedicated Testcontainers module and relies on Azure SQL Edge.

You can initialize the Service Bus emulator as a test container in two ways:

1️⃣ Code Compose: Setting up containers programmatically with the GenericContainer class.
2️⃣ Docker Compose: Using a Docker Compose file for container orchestration.

For more details, visit the GitHub repository: https://github.com/mirkoiv/servicebus-emulator-as-testcontainer

Building Service Bus emulator as a Test Container

Service Bus emulator does not have a Testcontainers module (yet), and additionally the emulator requires Azure SQL Edge.

There are two ways to initialize the Service Bus Emulator as a test container:

1️⃣ code compose – Composing through code using the GenericContainer class – AbstractServiceBusContainerCodeCompose
2️⃣ docker compose – Using a Docker Compose file and ComposeContainer class – AbstractServiceBusContainerDockerCompose

Key Differences Between the Two Approaches

In the first case, tests or applications interact directly with the ports exposed by the generic emulator and storage test containers. In the second case, an additional ambassador container is created, serving as a proxy to the generic emulator and storage containers.

Code Compose:

Docker Compose:

For more details and run samples, visit the GitHub repository: https://github.com/mirkoiv/servicebus-emulator-as-testcontainer

Simplify Local Development with nip.io: A Wildcard DNS Service

Introduction

When developing applications locally or in a testing environment, setting up domain names for different services can be a hassle. Configuring a local DNS server or modifying
/etc/hosts repeatedly is cumbersome and inefficient.

This is where nip.io comes in—a free, wildcard DNS service that makes it easy to resolve domain names to IP addresses without manual configuration.

What is nip.io and how it works?

nip.io is a wildcard DNS service that automatically maps domain names to IP addresses. Any subdomain formatted as x.x.x.x.nip.io will resolve to the corresponding IP address x.x.x.x. This service is particularly useful for local development, testing, and staging environments.

The concept behind nip.io is straightforward:

  • Any hostname following the pattern x.x.x.x.nip.io will resolve to x.x.x.x.
  • There’s no need for additional configuration or DNS setup.

When a request is made to a nip.io domain, the DNS lookup process translates the subdomain (which contains the IP address) into the corresponding numerical address. This means that an active internet connection is required to perform the DNS resolution.

The resolution process works as follows:

  1. The browser or application queries the DNS for the given nip.io subdomain.
  2. The nip.io DNS server extracts the IP address embedded in the subdomain and returns it.
  3. The client then connects directly to the resolved IP address.

Since nip.io operates as a public DNS service, an active internet connection is necessary to resolve the domain. If a network has restricted or no internet access, the DNS lookup for nip.io domains will fail, making it unusable in completely offline environments.

In such cases, alternatives like modifying the /etc/hosts file or setting up a local DNS resolver would be required.

Examples:

Domain NameResolves To
192.168.1.100.nip.io192.168.1.100
test.10.0.0.1.nip.io10.0.0.1
app.127.0.0.1.nip.io127.0.0.1

When running a local web server, you can access using nip.io subdomain instead of localhost:

python -m http.server 8000

Now, you can access your server via app.127.0.0.1.nip.io:8000 instead of localhost:8000

Cons of Using nip.io

While nip.io is highly convenient, there are some drawbacks to consider:

  1. Limited Reliability – Since nip.io is a free service, it may experience downtime or service disruptions.
  2. Potential Security Risks – Because anyone can generate a nip.io domain, it could be misused for malicious activities.
  3. SSL Certificate Challenges – I don’t know if it’s possible to generate a valid SSL certificate if needed, but you can always use a self-signed certificate.
  4. Dependence on External Service

Using Azure Event Hubs Emulator as a Test Container

This sample project demonstrates how to utilize the Azure Event Hubs emulator as a test container for local development and integration testing.

The Event Hubs emulator does not have a Testcontainers module and depends on the Azure Storage emulator (Azurite).

There are two ways to initialize the Event Hub Emulator as a test container:

1️⃣ Code Compose: Setting up containers programmatically with the GenericContainer class.
2️⃣ Docker Compose: Using a Docker Compose file for container orchestration.

For more details, visit the GitHub repository: https://github.com/mirkoiv/event-hubs-emulator-as-testcontainer

Building Event Hubs emulator as Test Container

Event Hubs emulator does not have a Testcontainers module (yet), and additionally the emulator requires Azure Storage emulator (azurite).

There are two ways to initialize the Event Hub Emulator as a test container:

1️⃣ code compose – Composing through code using the GenericContainer class – AbstractEventHubContainerCodeCompose
2️⃣ docker compose – Using a Docker Compose file and ComposeContainer class – AbstractEventHubContainerDockerCompose

Key Differences Between the Two Approaches

In the first case, tests or applications interact directly with the ports exposed by the generic emulator and storage test containers. In the second case, an additional ambassador container is created, serving as a proxy to the generic emulator and storage containers.

Code Compose:

eventhubs emulator - code compose

Docker Compose:

eventhubs emulator - docker compose

Using Spring Cloud Azure with Event Hubs Emulator – Workaround

Microsoft recently released the Azure Event Hubs emulator, which could simplify local development and testing.

Unfortunately, at the moment, the Azure Event Hubs emulator cannot be directly used with Spring Azure Cloud 4.x/5.x. This is because the Azure SDK for Java does not support the new connection string parameter ‘UseDevelopmentEmulator’ and cannot connect to the unsecured AMQP port.

However, there is a workaround solution available.

Feel free to explore examples and the workaround solution at https://github.com/mirkoiv/azure-event-hubs-emulator-installer

Problem

The emulator runs on the unsecured AMQP 5672 port. To interact with the emulator, you need to use a predefined connection string that includes the new parameter ‘UseDevelopmentEmulator’.

This new parameter is utilized within the Azure SDK to establish a connection to the emulator over port 5672. Otherwise, the connection will default to the secured AMQP port 5671, requiring a certificate for authentication.

Spring Azure Cloud versions 4.x/5.x depend on an SDK that does not support the new connection string parameter. This results in an ‘Illegal connection string parameter name: UseDevelopmentEmulator’ exception.

You can attempt to use a newer version of the SDK dependency (azure-core-amqp 2.9.4), which does support the new parameter. However, be aware that it also contains a bug (Azure/azure-sdk-for-java#40938) preventing the setup of an insecure connection to the emulator.

Workaround

A workaround solution involves using a self-signed certificate and a reverse proxy with SSL termination in front of the emulator.

For details visit https://github.com/mirkoiv/azure-event-hubs-emulator-installer/blob/main/Sample-Code-Snippets/Java/spring-cloud-azure-samples/README.md