Linux Audit Logs cheatsheet – Detect & Respond Faster

0

Why Linux is better than windows?


From an Operating system research point of view, Windows is not certainly old stuff, and it’s clear that people should be looking beyond it. After the launch of Linux, which is expected to govern the world in a few years, a sudden shift occurred. Because Linux OS is free, it is used by the majority of businesses. Even small businesses are adapting the Linux operating system.

The Linux Audit system is a useful feature for tracking security-related information. All the behavior of the machines/servers can be monitored by implementing Linux Auditing. It will be suitable for the organization in terms of security because the attackers are as cunning as a fox.

Use Cases of Linux Audit system:

  • Watching file access
  • Monitoring system calls
  • Recording commands run by a user
  • Recording security events
  • Searching for events
  • Running summary reports
  • Monitoring network access

Analysts should be aware of the audit logs while implementing the Linux auditing service. Organizations will be defining more custom rules to track the activities and audit services, which depends on the organization. In SIEM, there are 10 pre-defined audit rules. So, simply implementing audit services will not suffice. We must understand how to analyze the alerts. To analyze the alerts, we must be familiar with the log fields and directories.

List of Linux directories:

DirectoriesDetails inside the directory
/var/log/syslog or /var/log/messagesDisplays system-wide messages and information. Essentially, it’s a data log of every activities across the entire global system. Everything occurs on Redhat-based systems, such as CentOS or Rhel, will be recorded in messages. They go into Syslog on Ubuntu and other Debian systems.
/var/log/auth.log or /var/log/secureKeep track of successful and unsuccessful logins, as well as authentication processes. The amount of storage available is determined by the system type. Look at /var/log/auth.log for Debian/Ubuntu. Go to /var/log/secure on Redhat/CentrOS.
/var/log/boot.logKeep track of start-up messages and boot information.
/var/log/maillog or var/log/mail.logIt’s for mail server logs, which can be useful for postfix, smtpd, or other email-related services on your server.
/var/log/kernKernel logs and warning information are saved. Also important for resolving issues with custom kernels.
/var/log/dmesgIt serves as a central location for device driver communications. To view the messages in this file, use dmesg.
/var/log/faillogIt will keep track of failed login attempts. As a result, it’s useful for investigating suspected security breaches like credential hacks and brute-force attacks.
/var/log/cronIt will maintain a log of Crond-related messages (cron jobs). For example, it will keep the record of like When the cron daemon started a job.
/var/log/daemon.logKeeps track of background services that are operating but does not display them graphically.
/var/log/btmpkeeps a note of all failed login attempts.
/var/log/utmpKeeps track of current login state by the user
/var/log/wtmpKeeps track of every login and logout.
/var/log/lastlogEach login/logout is tracked in this log. 
/var/log/yum.logIt contains information about any yum command-based package installations. So you can see if everything went smoothly.
/var/log/httpd/It is a directory holding the Apache httpd daemon’s error log and access log files. The error log file records every error that httpd encounters. Consider memory issues and other system-related issues. All requests that come in via HTTP are logged by access_log.
/var/log/mysqld.log or /var/log/mysql.logEvery debug, failure, and success message, as well as the starting, stopping, and restarting of the MySQL daemon mysqld, is recorded in the MySQL log file. The directory is chosen by the system. /var/log/mariadb/mariadb.log is used by RedHat, CentOS, Fedora, and other RedHat-based systems. Debian/Ubuntu, on the other hand, uses the /var/log/mysql/error.log directory.
/var/log/pureftp.logUsing the pureftp process, it keeps track of FTP connections. Here you may find information about every connection, FTP login, and authentication failure.
/var/log/spoolerExcept for a few USENET mails, it usually contains nothing.
/var/log/xferlogKeeps track of FTP file transfers. File names and user-initiated FTP transfer information are included.

Before implementing the audit rules, familiarize yourself with the following fields.

Cheatsheet for analyzing Linux audit logs:

Field Name DescriptionExampleExample Description
typeThe type field contains the type of the record.type=Syscall
type=Path
type=user_auth
type=service_stop
The type: syscall conveys a message, that there was a system call to the kernel.
The type: the path will be logged to display file name path information.
The type: user_auth will be logged if there is an authentication attempt is detected.
The type: service_stop will be logged if any services are stopped.
msgThe msg field will provide a timestamp and a unique ID of the record in the form “audit(time_stamp: ID)”. Multiple records can share the same timestamp and ID if they were generated as part of the same Audit event. Various event-specific name=value pairs, promsg=audit(1623564119.769:179143)In the log, the time is captured in Unix epoch time. The format is 13 digits with a period before the last three (milliseconds) digits: 1623564119.769. Here, 1623564119.769=Time of the activity. We can convert the time using epochconverter.com. And 179143=ID of the event.
archRecords information about the CPU architecture of the system encoded in hexadecimal notation.arch=c000003eThe hexadecimal notation c000003e is converted as human-readable format as x86_64, which is an amd64, also known as Intel 64, i.e. 64-bit capable
syscallRecords the type of system call that was sent to the kernel.Syscall=59
Syscall=2
59 means “execve” i.e, system call executes the program referred to by pathname.
2 means “Open” i.e, system call opens the file specified by pathname.
There are more website in open source to find the relevant syscall name/description for each number.
successRecords whether a system call was successful or failed.success=yes
success=no
The success field records whether the system call recorded in that particular event succeeded or failed. No means the call did not succeed. Yes means the call got to succeed.
exitRecords the exit code returned by a system call. This value varies by a system call.exit=0Every Linux command executed by the shell script or user, has an exit status. 0 exit status means the command was successful without any errors. A non-zero (1-255 values) exit status means the command was a failure.
There are few websites available to find the Linux exit codes.
a0, a1, a2 a3The a0 to a3 fields record the first four arguments, encoded in hexadecimal notation, of the system call in this event. These arguments depend on the system call that is used and in the event logs, hexadecimal will be converted as text and shown along with it.Hexadecimal form: a0=a4567 a1=aa9680 a2=b858f0 a4=6cc
Text Form: a0=curl a2=-L a3=-s a4=http://ipecho.net/plain
Here the command is executed to get the public IP of the URL.
itemRecords the number of path records that are attached to this record.item=22 describes, there are 2 paths in the event.
ppidRecords the Parent Process ID (PPID).ppid=18996The parent process id refers to an owner who gave a command for initiating the activity.
pidThe pid field records the Process ID (PID).pid=765Here the above-mentioned ppid orders some child to do the activity. The activity done by the child process is mentioned in the pid. PID is the same as threat id.
auidThe auid field records the Audit user ID, which is the login uid. This ID is assigned to a user upon login and is inherited by every process even when the user’s identity changes (for example, by switching user accounts with the su – perk command).auid=5645678934The auid number represents the unique id of the user who has done the activity. Search with auid=*, the exact user will be displayed in the user field.
uid
gid
euid
suid
fsuid
egid
sgid
fsgid
1) The uid records the real user ID of the user who started the analyzed process.
2) The gid field records the group ID of the user who started the analyzed process.
3) The euid field records the effective user ID of the user who started the analyzed process.
4) The suid field records the set user ID of the user who started the analyzed process.
5) The fsgid field records the file system group ID of the user who started the analyzed process.
6) The egid field records the effective group ID of the user who started the analyzed process.
7) The sgid field records the set group ID of the user who started the analyzed process.
8) The fsgid field records the file system group ID of the user who started the analyzed process.
type=PATH msg=audit(12/22/2010 11:36:37.357:23) : item=3 name=/etc/group~
1)
type=SYSCALL msg=audit(12/22/2010 11:36:37.357:23) : arch=i386 syscall=rename
success=yes exit=0 a0=9164890 a1=91cf6f8 a2=9164890 a3=91cf6f8 items=4 ppid=2026
pid=2168 auid=rainer uid=root gid=root euid=root suid=root fsuid=root egid=root
sgid=root fsgid=root tty=pts0 ses=1 comm=vi exe=/usr/bin/vim.basic key=mykey
2)
type=SYSCALL msg=audit(12/22/2010 11:36:37.357:25) : arch=i386 syscall=open
success=yes exit=3 a0=9164890 a1=241 a2=1a4 a3=0 items=2 ppid=2026 pid=2168
auid=rainer uid=root gid=root euid=root suid=root fsuid=root egid=root
sgid=root fsgid=root tty=pts0 ses=1 comm=vi exe=/usr/bin/vim.basic key=mykey
3)
type=SYSCALL msg=audit(12/22/2010 11:36:37.393:26) : arch=i386 syscall=chmod
success=yes exit=0 a0=9164890 a1=81a4 a2=1 a3=0 items=1 ppid=2026 pid=2168
auid=rainer uid=root gid=root euid=root suid=root fsuid=root egid=root
sgid=root fsgid=root tty=pts0 ses=1 comm=vi exe=/usr/bin/vim.basic key=mykey
We need to track a user, whether it’s a real person or a system the user, so that we don’t lose track. For example, by using “sudo” or “su” the user will change their previlege. In this situation, we need one more sub-uid to track the activity. This is accomplished by assigning an extra UID. The AUID, which is different from the “normal” UID of the user, and is supposed to remain unchanged, whatever the user does.
Here we have 3 groups of events. Each group contains a single type=SYSCALL item, which describes what basic system call has been executed, and several other items describing pertinent details.
1) Action done on the path name=/etc/group,
2) they all refer to the executable exe=/usr/bin/vim.basic, which is the program used by the person editing that file.
3) they all point out that the user editing the file was uid=root, and had logged in as auid=rainer.
4) they describe a sequence of three events happening to the edited path name=/etc/group, ordered by time:
—– at 11:36:37.357:23, the path was renamed (syscall=rename) to name=/etc/group~
—– at 11:36:37.357:25, it was opened (syscall=open), presumably to write the new version, and finally
—– at 11:36:37.393:26, the file permissions were changed (syscall=chmod), most likely to reset them to the previous value.
ttyThe tty field records the terminal from which the analyzed process was invoked. Records the name of the controlling terminal. The value (none) is used if the process has no controlling terminal.tty=pts0The pts/0 is telling you which “pseudo terminal” the user is logged in on. In this case, it’s terminal #0. The “(:0.0)” tells you which hostname and displays what we are using.
A tty is a native terminal device, the backend is either hardware or kernel emulated.
sesThe “ses” field records the session ID of the session from which the analyzed process was started.ses=14567The number represents the ID of the current session.
commThis field records the command that is executed.comm=curl
comm=chmod
1) curl is used to transfer data from or to a server via HTTP, HTTPS, SCP , SFTP , and FTP
2) chmod is used to change the access mode of the file.
These are few examples. The command can be anything, which describes the exact activity executed on the machine.
exeThis field will display the path of the execution from where the activity was done.exe=”usr/bin/curl”Here the curl command was executed in the path “usr/bin”.
This is the same as windows logs, but here the directory names are different.
KeyThis field will display keyword associated with the rule which was the cause of the alert/event.Key=”recon”The keyword depends upon the rule. For each rule, there will be different keywords. Here the keyword recon means reconnaissance activity.
cwdThe cwd contains the path to the directory in which the system call was invokedcwd=”/home/perk”Here, the syscall was requested from the path “home/perk”
nameThe name field the path of the file or directory that was passed to the system call as an argument.name=”bin/chmod”Here, the syscall was towards the path “bin/chmod”
inodeFiles written to Linux filesystems are assigned an inode. The inode field contains the inode number associated with the file or directory recorded in this event. These unique IDs are used by the filesystem’s database to keep track of files.inode=657456The following command displays the file or directory that is associated with the 657456 inode number
modeThe mode field records the file or directory permissions, encoded in numerical notation.mode=0100755Here, the first 4 octets represent the fie type & the last 3 octets represent the permission of the file set by the user.
Permission numbers are:
0 = —
1 = –x
2 = -w-
3 = -wx
4 = r-
5 = r-x
6 = rw-
7 = rwx
So, here 755 means -rwxr-xr-x
PROCTITLEThe proctitle field records the full command-line of the command that was used to invoke the analyzed process. The field is encoded in hexadecimal notation to not allow the user to influence the Audit log parser. The text decodes to the command that triggered this Audit event.proctitle= 636174002F6574632F7373682
F737368645F636F6E666967
We can convert the proctitle to human readable text to know the executed command. Here the converted text is cat/etc/ssh/sshd_config

Conclusion:

Linux audit logs are not as difficult as we believe. It will be even more simple when we go through the log fields before progressing with the analysis.


Previous articleRed canary AtomicTest Harnesses – Tool for Mitre attack Execution
Next articleInsecure Direct Object Reference – Prevention and Detection of IDOR
Ambitious Blue Teamer; Enthused Security Analyst

LEAVE A REPLY

Please enter your comment!
Please enter your name here