Merlin’s Evolution: Multi-Operator CLI and Peer-to-Peer Magic

Over the past year, I’ve been working on making significant updates to Merlin in my free time. Today, I’m ready to release version 2 of Merlin. Some of the more significant updates include:

• A new command-line interface (CLI) application which allows multiple operators to use Merlin at the same time
• Peer-to-peer (p2p) agent communications over SMB, TCP, UDP, and both bind and reverse connections
• Passive P2P agent communications
• Configurable agent authentication methods
• Configurable agent communication data encoding and encryption
• Structured logging

For more in-depth information, review the release notes at the following repositories:

• https://github.com/Ne0nd0g/merlin
• https://github.com/Ne0nd0g/merlin-cli
• https://github.com/Ne0nd0g/merlin-agent

Command-Line Interface

in the past, one of the major drawbacks to using was that only a single operator could issue commands at a time; all control had to be done from the single place where the server was running. Hearing this feedback, I created a new CLI program that allows multiple operators to connect to the Merlin server from any workstation as opposed to having to authenticate to the server and connect to a screen session. The server will continue to run after a CLI client has disconnected.

The CLI uses gRPC to communicate with the server over a TLS connection and requires password authentication. There are several command-line arguments that can be used to configure the gRPC connection. Refer to the server documentation and cli documentation for a full list of options. Most notably, the server can be configured to require mutual TLS authentication with the CLI. This allows both the client and server to authenticate to each other to push back on having the traffic intercepted.

The system for CLI commands has been completely overhauled. Each command implements the Command interface, making it easier for people to add their own commands. In the previous version, all commands were essentially a giant switch block in the same file without a defined structure.

The “help” information for a command was historically very limited to just a couple of words. Now the CLI contains the same information found on the Wiki. Issue help <command> or place one of the following arguments after any commands to see the in-depth help information: “help”, “-h”, “ — help”, “?”, “/?” (Figure 1).

I wanted to quickly highlight that there is a reconnect command that allows an operator to re-establish a connection with the server if it is lost without having to restart the entire CLI application (Figure 2).

One downside at this moment in time is that the CLI does not show which operator executed a specific command and all output is sent to all connected CLI clients. I do plan to improve this in a future update.

Peer-to-Peer Agent Communications

The biggest feature update for version 2 of the agent is peer-to-peer agent communications. Merlin agents can now communicate over the smb-bind, smb-reverse, tcp-bind, tcp-reverse, udp-bind, and udp-reverse connections.

The most important thing to remember is that the agent must be configured with its associated listener universally unique identifier (UUID) either at compile time or runtime. This can be done with the -listener command line argument or the LISTENER argument for the Makefile. Without this, the server will not know how to decrypt and deconstruct the agent traffic. If the server receives traffic that it doesn’t understand, it will try to use all instantiated listeners to brute-force decompose the agent traffic.

Additionally, the peer-to-peer agents use the -addr command-line argument to specify the address they either bind or connect and NOT the -url argument used for the HTTP-based protocols.

For bind agents, use the link command from the parent Agent’s menu to connect to the child agent. For reverse agents, use the listener command from the parent Agent’s menu to setup the listener BEFORE executing the peer-to-peer child agent.

A listener must be configured on the Merlin server so that it can accept and process peer-to-peer agent traffic. It is important to note the peer-to-peer listeners do not actually bind to any network interfaces on the server because all traffic comes in as an embedded message from an egress HTTP agent and listener. Because of that, it really doesn’t matter what peer-to-peer protocol listener type is created; it only matters that the following configurable options are the same on the listener and the agent: Authenticator, PSK, Transforms and ID (Figure 3). The other options such as the protocol, interface, and port are for future functionality to generate agents from the server.

The ID for the listener will automatically be generated if left blank. Alternatively, this Listener’s ID can be manually set to a UUID previously used with existing agents.

Passive P2P Agent Communications

Merlin Agents are configured with a sleep value that dictates how frequently they should check in with the server to see if a job exists. This value is set through the agent’s -sleep command-line argument, the SLEEP variable in the Makefile, or from the CLI using the sleep command. The value provided to the argument must include a duration. For example, 10s is for ten seconds while 10m is for ten minutes.

The peer-to-peer agents can be configured with a negative sleep value like -10s. The actual value does not matter, as long as the value is negative. This tells the child agent to not send any traffic to the parent agent to check for jobs. Instead, if the server has a job that an operator issued, the parent agent will send it down to the peer-to-peer child agent. This is useful to reduce the amount of network traffic on the internal network if there is no need for communications. Because there is no check in messages from the peer-to-peer child agent, the agent will show as dead in the CLI. The checkin command can be issued to force the child agent to check in with the server.

Configurable Authentication

Both the Merlin agents and their associated listener can be configured to use different authenticators. Currently, only the OPAQUE and NONE options are available. Merlin has always used OPAQUE registration and authentication, so that is not new. If you would like to remove agent authentication altogether, use the NONE value. The Agent’s pre-shared key (PSK) will still be used to encrypt traffic but will not be used to authenticate to the server.

The HTTP based listeners have always utilized a JWT to authenticate agent traffic to the HTTP server. This is different than the PSK that is issued to authenticate the agent to the server. However, this has caused some users problems when they are testing on a virtual machine that does not have its time synced to the same time server. This causes the JWT to be created in the future or to show as “expired” in the server. To help facilitate this, the HTTP listener has a JWTLeeway configurable options that specifies how much time difference there can be between the server and the agent (Figure 4).

Because Merlin does not have a database, a server restart will require all agents to re-authenticate to get a valid JWT from the server; this is because the server’s JWT key changed. The HTTP listener now has a JWTKey configurable option to provide a base64-encoded JWT key so that the same key can be used between server restarts, thuspreventing the need for agents to re-authenticate.

Configurable Data Encoding and Encryption

This version of Merlin allows the communication data for the agent to be encoded and encrypted (collectively called transforms) in a configurable way. The transforms can be applied in any order and as few or as many times as desired. The currently available transforms are: aes, base64-byte, base64-string, hex-byte, hex-string, gob-base, gob-string, jwe, rc4, and xor.

THE LAST TRANSFORM IN THE LIST MUST BE gob-base

Prior to this release data was transformed into a JSON Web Encryption (JWE) and then gob-string encoded (Figure 5).

Figure 5 — Agent Data in JWE Format

You can see that the decoded JWE uses thePBES2-HS512+A256KW algorithm with 3,000 rounds of AES256 to encrypt the payload data inside the JWE (Figure 6).

Figure 7 contains an example of hex-string,aes,xor,gob-base data.

Structured Logging

Merlin now uses the Go log/slog package for both the Merlin Server and CLI. Log data is written to a file for both and is also written to standard out for the Merlin server (Figure 8). This allows the data to be consumed in a structured manner.

This release also includes additional Merlin Server command-line flags to control the amount of log output. The -trace flag enables the most amount of output and walks through entering every function along the way. The -extra flag is used to see the HTTP request/response data. The -debug flag enables the lowest amount of debug information. They are cumulative so trace includes the data from the extra and debug flags.

Miscellaneous

Aside from the list at the start of the post, I made some other miscellaneous changes made that aren’t quite as eye catching. For example, there was a significant overhaul in code base that worked toward implementing Domain-Driven Design (DDD) and SOLID design principles to significantly enhance the ability to extend the project. I was forced to do this when I began implementing peer-to-peer agent communications and due to my poor design choices, circular package references plagued me. That’s OK given that Merlin is how I learned to program in Go and as the years have passed, I’ve gotten better at programming. It’s natural to create better and better code year after year. Some of the most notable areas I incorporated these design principles was in creating entity packages to separate out the data structures, repositories to store data structures, and services to operate on those structures.

To continue the conversation, stop by the #merlin channel in the BloodHound Slack.

---

Merlin’s Evolution: Multi-Operator CLI and Peer-to-Peer Magic was originally published in Posts By SpecterOps Team Members on Medium, where people are continuing the conversation by highlighting and responding to this story.