`deferred-maintenance` CLI tool

December 14, 2020 | IN Blog, DevOps, Security Engineering, Security Operations and Security Engineering, Tool | BY Erich Smith
circle
circle

As we go through our days as developers, there is a tendency for certain types of maintenance activity to be deprioritized or forgotten about. It might be because it’s too granular, or it’s not important enough to reach out and capture that as a project management issue or a ticket to be prioritized later.

We have adopted a microservices architecture so our developers are constantly creating git repositories, hundreds of them. Fairly often there are cases where a developer realizes there’s an ongoing change that needs to happen to the system. It could even be a cross-cutting change, perhaps a new coding style or paradigm as a package update for specific tooling. For example, we use Terraform for our infrastructure. Occasionally when Terraform updates to new major versions, those updates cause breaking changes to our infrastructure code.

Why a `deferred-maintenance` CLI tool

Deferred-maintenance is a tool I wrote because I’d been feeling some pain when it came to coordinating these small system maintenance chores. Maintaining a record of small, recurring tasks that are buried in Jira, Bitbucket or even in slack is time consuming and error prone. Deferred-maintenance will help track down and manage those tasks. 

Individually these maintenance tasks might feel minor, but at scale it’s beyond reasonable expectations for any one developer to see these cross-cutting changes through to completion for all of the cyber assets that we own and operate. 

Deferred-maintenance is a tool that allows a developer to capture the notion that, “There’s this ‘task’ that’s supposed to happen, and it needs to happen to a subset of our assets.” Patching multiple hosts would be another example of how deferred-maintenance could be used. Determine where you are responsible for small, recurring maintenance tasks, and there’s a likelihood deferred-maintenance will come in handy. 

We need a way of marking these systems as needing a certain kind of maintenance, and binding that with an appropriate length of time to accomplish the task, as well as link to more details. “Where can I go for the definitive instructions on how to do this thing?”

The JupiterOne Graph

The deferred-maintenance tool is a simple way to annotate this in a JupiterOne graph. It provides a mechanism where we can provide alerts and dashboards where we can gather ongoing metrics about the tasks. 

From a security operations perspective, a more important function of the tool is to allow us to reason transparently about the risk to our systems. It might be the case where only 85% to 95% of our repositories might actually receive the needed maintenance. Real world schedule demands creep in, and the task itself is too small on an individual basis, giving it a low priority. What this creates over time is that code repositories might be in any number of states of back-level disrepair. They have fallen behind and are not included in any maintenance schedule. As new developers are brought onto the team, they don’t have any visibility into what it takes to surface the existing shortcomings. This can easily become viewed as technical debt, especially as the system starts to age.

2020-12-14 deferred-maintenance 01

2020-12-14 deferred-maintenance 02

Retaining and distributing corporate memory

deferred-maintenance allows you to issue CLI commands to a large selection of items. A command might be to update all repositories to the latest npm packages. You can relate back to a slack comment or an existing issue in a Jira ticket. You can then set a time frame on a “reasonable time to fix”. In this example, the tool would create a link in the JupiterOne graph from a bitbucket repository to a deferred-maintenance Finding entity.

The intent of the tool is to be able to query JupiterOne with statements such as, “Find all deferred-maintenance findings whose due date is in the past, where we’ve gone seven days past the expected deferment for this maintenance.” This allows us to see the potential risk to the system. If this were typically stored as a Jira ticket, or as a comment in slack, it would be harder to find the location and determine context and reason.

Download the deferred-maintenance CLI tool

Maintaining a record of small, recurring tasks that are buried in Jira, Bitbucket of even in slack is time consuming and error prone. deferred-maintenance will help track down and manage those tasks. 

You can download the deferred-maintenance tool through our github project. It  can be used by anyone using JupiterOne. If you don’t have JupiterOne, the free version can be found here. We would appreciate your feedback if you use the tool. 

For updates on demo videos and release of new, free tools, fill in the form below. No spam. No sales pitches. Just the good stuff coming your way about once a week.

Additional resources