Paperphyte/Assignments/Embedded/Real-Time/DeLaval

Case studyEmbedded/Real-TimeOngoing

1,000 tests in parallel, answers in minutes.

DeLaval builds the future of milking technology. In the Satin team we own the build environment and the automated tests that safeguard quality. The assignment: move the entire build environment from on-prem to the cloud — and lift development teams used to local hardware, so they get what the cloud actually does better instead of lifting 1-to-1.

ClientDeLaval

PeriodOct 2021 – Present

RolesDevOps, Delivery Engineer, Cloud Ops Engineer

ServiceDelivery engineering

01 / Problem

Tests ran at hardware pace — not the developer's.

Software for milking robots is tested against a complete system environment, not individual functions. On-prem means the number of environments equals the number of machines in the room: tests queue, feedback takes hours, and a developer who waits switches tasks and loses context.

The easy path would have been lifting the same set of machines to EC2 and calling it cloud. The assignment was explicitly the opposite: teams should get what the cloud actually does better — elasticity and isolation per test case — which requires changing platform and working habits at the same time.

ConstraintOne test case requires a whole system environment — not a process.

ConstraintTeams were used to local hardware; tools without a mindset shift wouldn't have been enough.

ConstraintBuilds and artifacts lived on-prem in Jenkins and JFrog Artifactory.

ConstraintFlaky tests couldn't be told apart from real failures without history.

02 / Approach

One container per test case.

Instead of copying the machine park, ACME was built: a framework that raises a complete system environment in Docker within each test session and runs one container per test case in AWS Fargate and Batch. Elasticity becomes a property of the test framework, not something the developer has to book.

Oct 2021Into the Satin teamOwnership of the build environment and the automated tests safeguarding quality in DeLaval's milking technology — with on-prem as the starting point.

2021 – 2022ACME takes shapeA framework that sets up a complete system environment in Docker within each test session and orchestrates one container per test case in AWS Fargate and Batch.

2022Artifactory to AWSJFrog Artifactory was migrated from on-prem to AWS and operated daily — the artifacts must move before the build can.

2022 – 2023Data instead of gut feelingTest results are converted from pipelines to InfluxDB and shown in Grafana: trends, flaky tests and grounds to act.

2023 – nowJenkins out, GitLab inPipelines were migrated from on-prem Jenkins to GitLab CI/CD, with daily operation of GitLab Runners on the Docker Autoscaler executor.

2024Decommissioning without dramaAt the move to JFrog SaaS, the self-hosted Artifactory was decommissioned as planned — the whole lifecycle, not just the setup.

The principle

Don't lift the machine park to the cloud. Lift the way of working — and let the platform follow.

03 / ACME

Autotest Cloud Managed Environment.

ACME is the framework that turns a test session into infrastructure: it describes the environment, raises it, distributes the test cases and tears everything down again. Four steps, the same path every time.

Step 01The environment is describedA complete system environment is defined in Docker as part of the test session.

Step 02The session is raisedACME starts the environment and distributes the test cases — one container per test case.

Step 03Runs in Fargate/BatchUp to 1,000 isolated tests run in parallel instead of queueing.

Step 04Results and teardownResults go to InfluxDB/Grafana; the environment is torn down leaving nothing behind.

Decisions around the framework

DecisionWhyConsequence

One container per test caseIsolation is cheaper than debugging tests that interfere with each other.Parallelism becomes a resource question instead of a scheduling question.

Fargate and Batch, not an EC2 fleetTest load is bursty: hundreds of environments for minutes, then zero.Capacity is paid per run; no machine park to maintain between sessions.

Terraform for the platformEnvironments must be reproducible and reviewable, not manually configured.Changes go through code and review — even for the build environment.

InfluxDB + Grafana for test dataOne red test says nothing; a hundred runs over time say everything.Flaky tests are identified by trend instead of anecdote.

GitLab CI/CD as one pathTwo parallel pipeline worlds double the maintenance and split the teams.On-prem Jenkins could be decommissioned; one shared path from commit to test result.

04 / Outcome

Measurable, every month.

Isolated tests in parallel1,000One container per test case in Fargate and Batch instead of queueing for local machines.

Feedback to developersMinutesDown from hours — fast enough to stay in the same task.

vCPU hours per month30,000Nearly 3.5 years of real time, consumed in one month.

Builds per month80,000GitLab Runners with the Docker Autoscaler executor in daily operation.

Beyond the volume, how the team reasons changed: test results are converted from pipelines to InfluxDB and visualised in Grafana, so trends and flaky tests show up in data instead of being judged on gut feeling. JFrog Artifactory was migrated from on-prem to AWS, operated daily for three years, then smoothly decommissioned at the move to JFrog SaaS — a lifecycle that went all the way, not just a migration.

05 / Technology

Eleven technologies, grouped.

Cloud & runtimeAWS EC2AWS ECSAWS FargateAWS Batch

Infrastructure & codeTerraformPython

DeliveryGitLab CI/CDJenkinsJFrog Artifactory

ObservabilityInfluxDBGrafana

Similar assignments

← All assignments

Say hello

Got a problem worth solving?

Chat with us on WhatsApp. We reply within 48 hours.