Finding the Evil in TLS 1.2 Traffic – Detecting Malware on Encrypted Traffic


TLS (Transport Layer Security) is the well-known protocol to securely provide privacy and data integrity between two communicating applications. Since TLS 1.1 has insecure MD5 and SHA1 algorithms, Internet has migrated to the TLS 1.2 version. Now Internet is communicating with TLS 1.2 with secure cipher suites to encrypt and establish sessions with users and applications. Malware authors use this protocol for malicious activity which includes (Data exfiltration, Malware beaconing, Phishing, Command and Control, drive-by download), Threat report 2020 from top Antivirus Industries states “quarter of malware used TLS to succeed and evade detection”

Before Moving into the analysis of traffic and key takeaway, I would like to speak about TLS 1.2 handshake to understand the concept clear and there are no barriers in your future analysis with TLS traffic.

TLS 1.2 Handshake Process:

  • ClientHello – Your Client will send a Hello message with the list of cipher suites which is supported by the browser.
  • Server – Server responds and picks a random cipher suite and contains the public key of the domain in which the originating user trying to connect, then ServerHelloDone.
  • Client key exchange – Client session key encrypted with the server’s public key
  • Finished – Handshake is complete and informs the client that all subsequent messages will be encrypted with the session key.

Go with the Flow:

Its highly recommended to understand the handshake process well to drill down and find the malicious activity.

  1. ClientHello – Internal client machine tries to make a handshake with External IP which is highly suspicious and Client Hello Packet is sent with a list of supported cipher suites from clients.
Internal Client shows the list of supporting cipher suits to the external IP.

2. ServerHelloDone – Server responds and picks a random cipher suite and contains the public key of the domain in which the originating user trying to connect, then ServerHelloDone. Below external IP picks the Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030).

TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030) Cipher Suite selected by External IP

The above Image shows, external IP shares the Certificate details to internal IP which contains the possible ( CN Name, Organization Name, and Country Name ). In this scenario, This certificate is signed by “Lets Encrypt” and CommonName which is hostname is akamacloud[.]tech.

Malware author has signed this domain with publicly available CA , I.E , Lets encrypt

Above Image shows , Certificate provide by lets encrypt is flagged as malicious by virustotal , as it has domain is associated with malware activity.

key Exchange completion

Above image shows , Server hello done is completed once the key exchange is completed.

Client key exchange – The change cipher spec protocol is used to change the encryption being used by the client and server. It is normally used as part of the handshake process to switch to symmetric key encryption. The CCS protocol is a single message that tells the peer that the sender wants to change to a new set of keys, which are then created from information exchanged by the handshake protocol. Check the below Image.

Client key exchange Completion

Now the Client session key will be encrypted with the server’s public key.Check the below Image.

TLS Session Ticket Created

Above Image refers to the change cipher spec protocol implemented from server side.

The above Images show, a handshake is completed and Now all messages are getting encrypted, Now your clients are directly making connections with Malware signed domains which are highly dangerous. Now Data exfiltration, Malware beaconing, Phishing, Command and Control, drive-by download can be using this encryption to hide malware and evade security controls.

Other Use cases :

Cobalt Strike Beacon Detection :

Becon is the process where the malware communicates with a C2 server asking for instructions or to exfiltrate collected data on some predetermined asynchronous interval. The C2 server hosts instructions for the malware, which are then executed on the infected machine after the malware checks in.

Cobalt Strike gives you a post-exploitation agent and covert channels to emulate a quiet long-term embedded actor in your customer’s network. Malleable C2 lets you change your network indicators to look like different malware each time.

The above Image shows, Infected host tries to get further instructions by using the fake amazon site, but it looks legitimate. While checking the IP address 138[.]201[.]149.51 is not at all belongs to, it just mimics to evade detections.

  • Later all connections are encrypted with client hello packets and the handshake process established, Now the malware can send instructions to a client in a fully encrypted manner which is highly dangerous.
  • Communication with the external host or Ip is made asynchronous interval to evade security controls. But still, this can be detected with good analytics approaches.

How to Detect this in SIEM?

  • Implement TLS profiling to known the clients who are using week cipher suites, which Hostnames it connects, and command Name ( CN ) domain which it connects.
  • Implement Zeek open-source software network analysis framework logs to your SIEM, which is great in profiling the TLS, HTTP, SMTP, and more protocols to find the anomaly.
  • Use TLS Fingerprinting with JA3 and JA3S Hashs, validate the hash. JA3 is a much more effective way to detect malicious activity over SSL than IP or domain-based IOCs.
  • JA3S uses the TLSVersion, Cipher, Extensions to make a hash, using this algorithm can detect any kind of malware profiled with SSL/TLS.
  • There are lots of firewalls or devices are available to inspect the SSL to find the malicious behavior. But Most experts say that inspecting the SSL/TLS is decrypting the communication and breaking someone’s privacy.
  • Malware authors always leverage the best encryption to deliver malware. Therefore use the AIML to detect the malware without decrypting it.
  • AIML plays an important role in detecting and developing advanced use cases, Getting attributes from packet captures or JA3S hashes can be used to apply the AIML algorithms such as logistic regression or any suitable algorithms that can be used to detect malware.
  • Self-signed certificates will have less validation, this can be detected with TLS certificate common Name ( SS ) – Which is the indicator of a self-signed certificate. But Most malware doesn’t use Self-signed certificates, Instead, they go with free SSL providers to get the certificate.


TLSv1.3 is the new version of TLS released in August 2018 destined to replace TLSv1.2.It has some additional It has been designed improved security and speed. The main changes include removing the specific ciphers and hashing algorithms and handshake roundtrips are reduced, Now ServerHello messages will be encrypted. looks like as a defender we will see some difficulty in handling such blind spot areas to analyze. Machine learning is the only option to solve the issues and find the malware !!!

Happy Encrypting !!!

Malware Samples Downloaded From Malware_traffic_analysis

agent tesla malware analysis agent tesla malware defence mechanism apt Bazarcall bazarcall loader Bazarcall malware Bazar Call Malware Bazarcall malware ioc cobalt strike iocs cobalt strike ips cyber threat intelligence dridex malware analysis dridex malware ioc Hancitor hancitor 2021 hancitor malware hancitor malware analysis hancitor malware ioc hancitor ransomware hancitor threat actor incident response tools kerberoasting attack latest threat intel malware malware analysis malware analysis sandbox malware analysis tool malware analyst MITRE network attack Network Traffic Analysis phishing detection techniques siem soc soc interview questions Threat analysis Threat Hunting threat hunting dns threat hunting tools threat intel feeds threat intelligence Trickbot trickbot ioc trickbot ioc 2021 windows event logs

Previous articleThreat Intelligence – HANCITOR Malware Latest IOCs
Next articleThreat Intelligence – Trickbot Malware Latest IOCs
Balaganesh is a Incident Responder. Certified Ethical Hacker, Penetration Tester, Security blogger, Founder & Author of Soc Investigation.


Please enter your comment!
Please enter your name here