Riding the InfoRail to Exploit Ivanti Avalanche

We find we are dealing with the Tomcat Web Application plus other services. “Wavelink Information Router” seems to be the most interesting of them, due to the following description: “Coordinates communication between Wavelink processes…”.

At this stage, it all seemed complicated. Because of this, I used the following methods to gain some more insight:
• Network traffic analysis
• Static analysis
• Log file analysis

After a while, I was able to see a bigger picture. It seemed that the Avalanche Web Application is not able to perform tasks by itself, not even a login operation. However, it can freely use different services to perform tasks. The Information Router, also known as the InfoRail service, stands in between, and is responsible for the distribution of messages between the services. The InfoRail service listens on TCP port 0.0.0.0:7225 by default.

As can be seen in the above diagram, the communication flow is straightforward. The Web Backend creates a message and sends it to the InfoRail service, which forwards it to the appropriate destination service. The destination service handles the message and returns the response to the Web Backend, once more via the InfoRail service. In the default installation, all these services are running on the same host. However, it is possible to place some of these services on remote machines.

On the other hand, it is not easy to get more detailed information about the messages. Network traffic analysis was not of much use, as the messages seem to be obfuscated or encrypted. In order to learn something more about them, I had to conduct a detailed code analysis.

The following section presents an overview of the InfoRail protocol and all the basics needed to:
• Recreate the protocol and messages
• Understand its basics
• Understand the payload delivery mechanism

InfoRail Protocol Basics

A typical InfoRail message consists of three main parts:
• Preamble
• Header
• Optional XML payload

The following picture presents a message structure:

The InfoRail Preamble

The preamble always has a length of 24 bytes. It consists of the following parts:
• Bytes 1-4: Length of the whole message
• Bytes 5-8: Length of the header
• Bytes 9-12: Length of the payload
• Bytes 13-16: Length of the uncompressed payload (payload can be optionally compressed)
• Bytes 17-20: Message ID (typically an incremented number, starting from 1)
• Byte 21: Protocol version (typically 0x10)
• Bytes 22-23: Reserved • Byte 24: Encryption flag (payload and header can be optionally encrypted) – 0 or 1

The InfoRail Header

The header of the typical message consists of multiple keys together with their corresponding values. Those keys and values are included in the header in the following way:
• 3 null bytes
• 0x02 byte
• 3 null bytes
• 1 byte which provides the length of a key (e.g. 0x08)
• 3 null bytes
• 1 byte which provides the length of a value (e.g. 0x06)
• Key + value

Here’s an example key and value:

         \x00\x00\x00\x02\x00\x00\x00\x08\x00\x00\x00\x06h.msgcat999999

As was mentioned before, a header can contain multiple keys. The most important ones that appear in most or all messages are:
• h.msgcat - in typical requests, it is equal to 10 (request)
• h.msgsubcat - information about the request type, thus it is crucial for the payload processing
• h.msgtag - usually stores the JSESSIONID cookie, although no method that verifies this cookie was spotted, thus this value can be random.
• h.distlist - specifies one or more services to which the message will be forwarded by InfoRail

The header must be padded with null bytes to a multiple of 16 bytes, due to the optional encryption that can be applied.

The Payload

As was mentioned before, the majority of messages contain an XML payload. An XStream serializer is used for the payload serialization and deserialization operations.

Different serialized objects can be transferred depending on the target service and the message subcategory. However, in most cases we are dealing with the RequestPayload object . The following snippet presents an example of the XML that is sent during an authentication operation. In this case, Avalanche Web Backend sends this XML to the Enterprise Server service, which then verifies user credentials.

RequestPayload.xml

As you can see, this message contains a serialized UserCredentials object, which consists of the loginName, encrypted password, domain, and clientIpAddress. You can also see that the RequestPayload contains the web cookie (sessionId). However, this cookie is never verified, so it can be set to any MD5 hash. As the payload can be optionally encrypted (and compressed), it must be padded to a multiple of 16 bytes.

Typically, every service implements its own ObjectGraph class, which defines the instance of the XStream serializer and configures it. Here’s a fragment of an example implementation:

ObjectGraphPart.java

When the service receives the XML payload, it deserializes it with the ObjectGraph.fromXML method.

Encryption

As was mentioned before, the header and the payload of the message can be encrypted. Furthermore, the payload can be compressed. InfoRail does not use the SSL/TLS protocol for encryption. Instead, it manually applies an encryption algorithm using a hardcoded encryption key.

As both the encryption algorithm and the encryption key can be extracted from the source code, the potential attacker can perform both the decryption and encryption operations without any obstacles.

Distribution to Services

The InfoRail service needs to know where to forward the received message. This information is stored in the h.distlist key of the message header. Values that can be used are stored in the IrConstants class. Variables that start with IR_TOPIC define the services where the message can be forwarded. The following code snippet presents several hardcoded variables:

DistributionVariables.java

If the sender wants the message to be forwarded to the license server, the h.distlist value must be set to 255.3.2.8.

Message Processing and Subcategories

This is probably the most important part of the protocol description. As was mentioned before, the message category is included in the message header under the h.msgsubcat key. As seen in the last line of the code snippet below, services protect themselves against the handling of an unknown message and, if the provided message subcategory is not implemented, the message is dropped.

Message processing may be implemented in slightly different ways depending on the service. Here, we are going to take a brief look at the Avalanche Notification Server. The following snippet represents the processMessage method found in the MessageDispatcher class:

processMessage.java

At [1], the message subcategory is retrieved from the header. Then, it is passed to the MessageProcessorVector.getProcessor method to obtain a reference to the appropriate message handler

At [2], the switch-case statement begins.

If the category is equal to 10 (request), the processInfoRailRequest method is invoked at [3]. Its arguments contain the handler that was retrieved at step [1]. We can have a quick look at this method here:

processInfoRailRequest.java

Basically, if the handler is not null, the code will invoke the handler.ProcessMessage method.

Finally, we must investigate how handlers are retrieved. The most important part is as follows:

MessageProcessorVector.java

At [1], a HashMap<Integer, IMessageProcessor> is defined.

At [2] and subsequent lines, the code invokes the setProcessor method. It accepts the message subcategory integer and the corresponding object that implements IMessageProcessor, such as AnsCredentialsHandler.

At [3], the setProcessor method is defined. It inserts the subcategory and appropriate object into the HashMap defined at [1].

At [4], getProcessor retrieves the handler based on the provided subcategory.

To summarize, there is a HashMap that stores the message subcategories and their corresponding handler objects. If we send a message to the Avalanche Notification Server with the subtype equal to 3706, the AnsTestHandler.processMessage method will be invoked.

Let’s have a look at one example of a message handler, AnsTestHandler:.

AnsTestHandler.java

At [1], the SUBCATEGORY variable is defined. It is common for the message processors to define such a variable.

At [2], the processMessage method is defined.

At [3], it retrieves the XML payload from the message.

At [4], it deserializes the payload with the ObjectGraph.fromXML method. It then casts it to AnsTestPayload, although the casting occurs after the deserialization. According to that, if the ObjectGraph does not implement any additional protections (such as an allowlist), we should be able to do harm here.

Success! We have identified the message processing routine. Moreover, we were able to quickly map the subcategory to the corresponding fragment of code that will handle our message. It seems that right now, we should be able to craft our own message.

Is There Any Authentication?

At this point, it seemed that I had everything necessary to send my own message. However, when I sent one, I saw no reaction from the targeted service. I looked at the InfoRail service log file and spotted these interesting lines:

It seems that I had missed an important part: authentication. Having all the details regarding the protocol and encryption, I was able to quickly identify a registration message in the network traffic. It is one of the rare examples where the payload is not stored in the XML form. The following screenshot presents a fragment of a registration message:

Several important things can be spotted here:
         -- reg.appident – specifies the name of the service that tries to register.
         -- reg.uname / reg.puname – specifies something that looks like a username.
         -- reg.cred / reg.pcred – specifies something that looks like a hashed password.

After a good deal of code analysis, I determined the following:
         -- Uname and puname are partially random.
         -- Cred and pcred values are MD5 hashes, based on the following values:
                  -- Username (.anonymous.0.digits).
                  -- Appropriate fragment of the key, which is hardcoded in the source code.

Once more, the only secret that is needed is visible in the source code. The attacker can retrieve the key and construct his own, valid registration message.

Finally, we can properly register with the InfoRail service and send our own messages to any of the Avalanche services.

Just Give Me Those Pwns Please – Code Execution on Four Services

At this stage, one can verify the services that do not implement a allowlist in the ObjectGraph class. I identified 5 of them:

For no particular reason, let’s focus on the StatServer. To begin, we must find the topic and subcategory of some message that will deserialize the included XML payload. According to that, the InfoRail protocol message headers should contain the following keys and values:
• h.distlist = 255.3.2.12
• h.msgsubcat = 3502 (GetMuCellTowerData message)

In this example, we will make use of the AspectJWeaver gadget, which allows us to upload files. Below is the AspectJWeaver gadget for the XStream:

AspectJWeaver.xml

Several things can be highlighted in this gadget:
• The iConstant tag contains the Base64 file content.
• The folder tag contains a path to the upload directory. I am targeting the Avalanche Web Application root directory here.
• The key tag specifies the name of the file.

Having all the data needed, we can start the exploitation process:

The following screenshot presents the uploaded web shell and the execution of the whoami command.

As you can see, the Web File Server loads several Tomcat JAR packages. You might perhaps also be familiar with the great technique discovered by Michael Stepankin, which abuses unsafe reflection in Tomcat BeanFactory to execute arbitrary commands through JNDI Lookup. The original description can be found here.

In summary, we can execute the following attack:

1. Set up malicious LDAP server, which will serve the malicious BeanFactory. We will use the Rogue JNDI tool.
2. Register with the InfoRail service.
3. Send the message containing the CVE-2021-39146 gadget, targeting the Web File Server pointing to the server defined in step 1.
4. The Web File Server makes an LDAP lookup and retrieves data from the malicious server.
5. Remote code execution.

The following screenshot presents the setup of the LDAP server:

The next snippet presents the CVE-2021-39146 gadget, which was used for this Proof of Concept:

jndiGadget.xml

If everything goes well and the lookup was performed successfully, Rogue JNDI should show the following message and the code should be executed on the targeted system.

When a user authenticates through the Avalanche Web Application login panel, the backend transfers the authentication message to the Enterprise Server. This service verifies credentials and sends back an appropriate response. If the provided credentials were correct, the user gets authenticated.

During this research, I learned two important things:
• The attacker can register as any service.
• The authentication message is distributed amongst every instance of the registered Enterprise Server.

According to that, the attacker can register himself as the Enterprise Server and intercept the incoming authentication messages. However, this behavior does not have much of an immediate consequence, as the transferred password is hashed and encrypted.

The next question was whether it is possible to deliver the attacker’s own response to the Avalanche Web and whether it will be accepted or not. It turns out that yes, it is possible! If you would like to deliver your own response, you must properly set two values in the message header:
• Origin – The topic (ID) of the Avalanche Web backend that sent the message.
• MsgId – The message ID of the original authentication message.

Both values are relatively easy to get, and thus the attacker can provide his own response to the message. It will be accepted by the service, which is waiting for the response. An example attack scenario is presented in the following figure (read from top to bottom ).

The attack scenario works as follows:
         -- The attacker tries to log into the Web Application with the wrong credentials.
         -- Web Application sends the authentication message.
         --The InfoRail service sends the message to both Enterprise Servers: the legitimate one and the malicious one.
         --Race time:
                  --The legitimate server responds with the “wrong credentials” message.
                  --The malicious server responds with the “credentials OK” message.          --If the attacker’s server was first to deliver the message, it is forwarded to the Avalanche Web Application.
         --The attacker gets authenticated.

Please note that the “Login message” targeting the attacker’s server is not present here on purpose (although it is in reality transmitted to the attacker). I wanted to highlight the fact, that this issue can be exploited without a possibility to read messages. In such a case, the attacker has to brute-force the already mentioned message ID value. It complicates the whole attack, but the exploitation is still possible.

To summarize this part, the attacker can set up his own malicious Enterprise Server and abuse a race condition to deliver his own authentication response to the Web Application. There are two more things to investigate: what does the response message look like and are we able to win the race?

Authentication Handling

The following snippet presents an example response to a login message. Please note that those messages are way bigger during legitimate usage. However, for the Proof of Concept, I have minimized them and stored only those parts that are needed for exploitation.

The response message consists of several important parts:
• It contains a responseObject tag, which is a serialized User object.
• It contains a responseCode tag, which will be important.

At some point during authentication, the Web Backend invokes the UserService.doLogin method:

doLogin.java

At [1], the UserCredentials object is instantiated. Then, its members are set.

At [2], the authenticate method is called, and the object initialized at [1] is passed as an argument:

authenticate.java

At [1], the UserLogin object is initialized.

At [2], the UserCredentials object is serialized.

At [3], the message is being sent to the Enterprise Server, and the Web Backend waits for the response.

At [4], the responseCode included in the response is verified. We want it to be equal to 0.

At [5], the userLogin.authenticated is set to True.

At [6], the userLogin.currentUser is set to the object that was included in the responseObject.

At [7], the method returns the userLogin object.

Basically, the response should have a responseCode equal to 0. It should also include a properly serialized User object in the responseObject tag.

Finally, we analyze the fragment of UserBean.loginInner method that was responsible for the invocation of the doLogin function.

loginInner.java

At [1], the doLogin method is invoked. It retrieves the object of UserLogin type (see the returns of previous code snippets).

At [2], it sets this.currentUser to the userLogin.currentUser (the one obtained from the response).

At [3], it sets various other settings. These will not be relevant for us.

One very important thing to note: the Web Backend does not compare the username provided in the login panel and the username retrieved by the Enterprise Server. Accordingly, the attacker can:
• Trigger the authentication with the username “poc”.
• Win the race and provide the user “amcadmin” in the response.
• The attacker will then be authenticated as the “amcadmin”.

To summarize, it seems that there are no obstacles for the attacker and his response should be handled by the Web Backend without any problems. Next, we turn our focus to winning the race.

Winning the Race

In a default installation, the Enterprise Server and InfoRail services are located on the same host as the Web Backend. This makes the exploitation of the race condition harder, as the legitimate communication is handled by the local interface, which is much faster than communication through an external network interface.

Still, the attacker has some advantages. For example, he does not have to generate dynamic responses, as the response payload can be hardcoded in the exploit. The following table presents a general overview of actions that must be performed by both the attacker and the legitimate Enterprise Server.