*************
Writing Rules
*************
Gene rules have been designed to be straightforward to understand and to parse.
In order not to have to develop a custom rule parser, we have decided to use JSON
document as container. Then every rule, to work properly, has to follow a specific
format which is going to be described in this page.
Getting the format supported by the engine
==========================================
A starting point to understand the format of a rule is to use the ``gene`` command
line utility to get the format supported by your engine::
gene -template
This command line should return a JSONĀ document containing all the fields used
by your current version of ``gene``.
Rule Structure
==============
.. code-block:: JSON
{
"Name": "",
"Tags": [],
"Meta": {
"EventIDs": [],
"Channels": [],
"Computers": [],
"Traces": [],
"Criticality": 0,
"Disable": false
},
"Matches": [],
"Condition": ""
}
.. important::
The fields present in the template shown above are the ones used by the engine.
It means that **any** additional field will not impact the engine. This
trick can be used to document the rule. It is a good practice to add information
such as **Author**, **Comments** and eventual **Links** in the **Meta** section.
.. table:: Field Definition
+------------+------------+----------------------------------------------------+
| Field | Type | Description |
+============+============+====================================================+
| Name | string | Name of the rule |
| | | |
+------------+------------+----------------------------------------------------+
| Tags | []string | Contains a list of tags related to the rule. It |
| | | can be used to group rules |
| | | according to their tag(s). |
+------------+------------+----------------------------------------------------+
| Meta | dict | Contains a bunch of information related the trigger|
| | | of the rule. The information in there is used to |
| | | match against the "System" section of the Windows |
| | | events to speed up the match. |
+------------+------------+----------------------------------------------------+
| EventIDs | []int | List of Windows Event IDs the rule should match |
| | | against. If empty the rule will apply against any |
| | | Event IDĀ of the ``Channels`` (c.f. see next) |
+------------+------------+----------------------------------------------------+
| Channels | []string | List of channels the rule should apply on. If |
| | | empty, the rule will apply against any event of any|
| | | channel. |
+------------+------------+----------------------------------------------------+
| Computers | []string | List of computer names the rule should apply on. |
| | | If empty, the rule applies on all the computers. |
+------------+------------+----------------------------------------------------+
| Traces | []string | List of traces used to trace other events related |
| | | to the rule. A rule can be used to generate |
| | | dynamic rules with information from the event which|
| | | matched the rule. The syntax of each trace must |
| | | follow `Traces Format`_. |
+------------+------------+----------------------------------------------------+
|Criticality |0 < int < 10| The criticality level attributed to the events |
| | | matching the rule. If an event matches several |
| | | rules the criticality levels are added between them|
| | | and will never go above 10. |
+------------+------------+----------------------------------------------------+
| Disable | bool | Boolean value used to disable the rule. |
+------------+------------+----------------------------------------------------+
| Matches | []string | List of **Matches**, should follow the syntax of |
| | | `Matches Format`_ |
+------------+------------+----------------------------------------------------+
| Condition | string | String implementing the logic on the **Matches** to|
| | | trigger the rule. The syntax should be compliant |
| | | with `Condition Format`_ |
+------------+------------+----------------------------------------------------+
.. important::
The more precise **EventIDs** and **Channels** fields, the faster the rule is.
Those information are mainly used to filter out irrelevant events.
Matches Format
--------------
A **Match** can be seen as an atomic check which is done on every Windows Event
(pre-filtered using **Meta** section of the rule) going through the engine. Every
match can be referenced once or more in the **Condition** to create complex
matching rule. Currently, the latest version of the engine supports two kinds of
**Matches**.
.. important::
It is very important to remember that **Matches** only apply on the fields
located under the ``EventData`` section of Windows Events.
Field Matches
^^^^^^^^^^^^^
A **Field Match** is basically an **equality** or a **regex** check done on a
given **field value**. This kind of **Match** brings flexibility to the engine since
anything can be matched through regular expression.
**Syntax:** ``$VAR_NAME: FIELD OPERATOR 'VALUE'``
.. table:: Field Match Symbols Definition
+------------+----------------------------------------------------------------+
| Symbols | Description |
+============+================================================================+
| VAR_NAME | Name of the variable use to access the result of the **Match** |
| | in the **Condition**, it must be preceded by a ``$`` |
+------------+----------------------------------------------------------------+
| FIELD | Field to match with in ``EventData`` section of Windows Events |
+------------+----------------------------------------------------------------+
| OPERATOR | Operator to use for the match: |
| | * ``=`` : equal operator |
| | * ``~=`` : regexp operator (tells to compile VALUE as a regex)|
+------------+----------------------------------------------------------------+
| VALUE | Must be surrounded by **simple quotes** ``'``. This is the |
| | **value/regex** to match against to make **$VAR_NAME = true** |
+------------+----------------------------------------------------------------+
Match Workflow::
+-------+ +---------+
| Event | | Match |
+-------+ +---------+
| +----------+ |
+----> | Engine | <----+
+----------+
|
+---------------------------+
| Extracts value from FIELD |
+---------------------------+
|
+---------------------------+
| Does value match VALUE |
| according to OPERATOR ? |
+---------------------------+
|
^
YES / \ NO
/ \
/ \
/ \
+------------------+ +-------------------+
| $VAR_NAME = true | | $VAR_NAME = false |
+------------------+ +-------------------+
\ /
\ /
+--------------------+
| $VAR_NAME value is |
| used in condition |
+--------------------+
.. important::
Any regular expression must follow `Go regexp syntax `_.
Example:
""""""""
The following snippet shows a rule used to catch Windows Event log clearing attempts
using ``wevutil.exe``.
.. code-block:: JSON
{
"Name": "EventClearing",
"Tags": ["PostExploit"],
"Meta": {
"EventIDs": [1],
"Channels": ["Microsoft-Windows-Sysmon/Operational"],
"Computers": [],
"Criticality": 8,
"Author": "@0xrawsec"
},
"Matches": [
"$im: Image ~= '(?i:\\\\wevtutil\\.exe$)'",
"$cmd: CommandLine ~= '(?i: cl | clear-log )'"
],
"Condition": "$im and $cmd"
}
.. warning::
In order to match a single ``\`` Windows path separator, we need to use ``\\\\``
when using ``=~`` and ``\\`` when using ``=`` operator
Container Matches
^^^^^^^^^^^^^^^^^
An **Container Match** is a little bit more advanced since it can be used to extract
a part of a **field value** and check it against a container. For
instance, with this kind of **Match**, we are able to extract a **domain** information
contained in Windows DNS-Client logs and check it against a blacklist. Although,
implementing this use case would be possible with **Field Matches**, it
would be much slower due to regex engine. In addition the rule would need to be updated
at every new entry to check, however with **Container Match** only the container
(a simple separate file) needs to be updated. The speed is provided by the
container which is implemented in a form of a set data structure.
**Syntax:** ``$VAR_NAME: extract('REGEXP', FIELD) in CONTAINER``
.. table:: Container Match Symbols Definition
+------------+----------------------------------------------------------------+
| Symbols | Description |
+============+================================================================+
| VAR_NAME | Name of the variable used to access the result of the **Match**|
| | in the **Condition**, it must be preceded by a ``$`` |
+------------+----------------------------------------------------------------+
| FIELD | Field to extract from |
+------------+----------------------------------------------------------------+
| REGEXP | Regular expression used to extract a value from FIELD and check|
| | it against a **CONTAINER**. **REGEXP** must follow **named** |
| | regexp syntax ``(?Pre)`` |
+------------+----------------------------------------------------------------+
| CONTAINER | Container to use to check the extracted value |
+------------+----------------------------------------------------------------+
.. important::
* If a rule makes use of an **undefined container**, the rule will be disabled
at runtime and a warning message will be printed.
* A given container is shared across all the rules loaded into the engine
* Any regular expression must follow `Go regexp syntax `_.
Example:
""""""""
This rule shows an example of how to extract domains and sub-domains from Windows
DNS-Client logs and check it against a blacklist.
.. code-block:: JSON
{
"Name": "BlacklistedDomain",
"Tags": ["DNS"],
"Meta": {
"EventIDs": [],
"Channels": ["Microsoft-Windows-DNS-Client/Operational"],
"Computers": [],
"Criticality": 10,
"Author": "@0xrawsec",
"Comment": ""
},
"Matches": [
"$domainBL: extract('(?P\\w+\\.\\w+$)',QueryName) in blacklist'",
"$subdomainBL: extract('(?P\\w+\\.\\w+\\.\\w+$)',QueryName) in blacklist'",
"$subsubdomainBL: extract('(?P\\w+\\.\\w+\\.\\w+\\.\\w+$)',QueryName) in blacklist'"
],
"Condition": "$domainBL or $subdomainBL or $subsubdomainBL"
}
Traces Format
-------------
A trace is used to generate a new rule **on the fly** derived from both the rule
which triggered and the **Windows Event** which matched. This feature allows
the engine to do some **basic** correlation. The rule generated is very basic
and has a single match.
**Syntax:** ``EVENT_IDS:CHANNELS: NEW_FIELD OPERATOR EVT_VAL_FIELD``
.. table:: Trace Symbols Definition
+---------------+----------------------------------------------------------------+
| Symbols | Description |
+===============+================================================================+
| EVENT_IDS | Comma separated list of **Windows Event IDs** used to set |
| | EventIDs field of the new rule. If empty, default is to |
| | inherit from **the rule defining the trace**. |
+---------------+----------------------------------------------------------------+
| CHANNELS | Comma separated list of **Windows Event Log Channels** used to |
| | set **Channels** field of the generated rule. If empty, default|
| | is to inherit from **the rule defining the trace**. |
+---------------+----------------------------------------------------------------+
| NEW_FIELD | **Field name** to use for the **single Match** of the generated|
| | rule. |
+---------------+----------------------------------------------------------------+
| OPERATOR | Operator to use for the match: |
| | * ``=`` : equal operator |
| | * ``~=`` : regexp operator (tells to compile VALUE as a regex)|
+---------------+----------------------------------------------------------------+
| EVT_VAL_FIELD | Name of the field in the matching **Windows Event** to extract |
| | the value from and used as **VALUE** in the generated rule |
+---------------+----------------------------------------------------------------+
.. important ::
Keywords ``any``, ``ANY`` or ``*`` can be used instead of comma separated list
in **EVENT_IDS** and **CHANNELS** to respectively apply trace on any Event ID
and any Channel.
The concept behind the traces is maybe a little bit hard to get (and also to explain).
That is why, in the following snippet, I have tried to show what a generated rule
from a trace would look like.
.. code-block:: JSON
{
"Name": "GENERATED_NAME",
"Tags": ["inherited from triggering rule"],
"Meta": {
"EventIDs": ["inherited from triggering rule OR set from trace"],
"Channels": ["inherited from triggering rule OR set from trace"],
"Computers": ["inherited from triggering rule"],
"Traces": [
"inherited from triggering rule"
],
"Criticality": "inherited from triggering rule",
},
"Matches": [
"$m: NEW_FIELD OPERATOR 'ValueOf(CUR_FIELD) extracted from Matching Event'",
],
"Condition": "$m"
}
.. warning::
* Traces generation is not enabled by default by the engine, in order to enable
it, use the ``-trace`` command line switch
* When trace mode is enabled, many rules can be generated at runtime and the
engine will by design become slower since **any Windows Event** matches
**any** rule loaded.
* If **X** number of traces is defined, **X** rules will be generated at runtime when
**trace mode** is enabled and the rule matches a **Windows Event**
Example:
^^^^^^^^
The following rule will generate rules to trace **any Event ID** from channel
**Microsoft-Windows-Sysmon/Operational** where either the **ProcessGuid** or
**ParentProcessGuid** is equal to the **ProcessGuid** of the event which triggered the
rule.
.. code-block:: JSON
{
"Name": "MaliciousLsassAccess",
"Tags": ["Mimikatz", "Credentials", "Lsass"],
"Meta": {
"EventIDs": [10],
"Channels": ["Microsoft-Windows-Sysmon/Operational"],
"Computers": [],
"Traces": [
"*::ProcessGuid = ProcessGuid",
"*::ParentProcessGuid = ProcessGuid"
],
"Criticality": 10,
"Author": "0xrawsec"
},
"Matches": [
"$ct: CallTrace ~= 'UNKNOWN'",
"$lsass: TargetImage ~= '(?i:\\\\lsass\\.exe$)'"
],
"Condition": "$lsass and $ct"
}
Condition Format
----------------
A condition applies a logic to the different **Matches** defined in the rule.
If the result of the computation of the **Condition** is **true** the event is
considered as matching the rule.
.. table:: Allowed Symbols in Condition
+---------+----------------------------------------------------------------+
| Symbols | Description |
+=========+================================================================+
| ``$var``| Variable referencing a **Match** |
+---------+----------------------------------------------------------------+
| ``()`` | Used to group / prioritize some logical expressions |
+---------+----------------------------------------------------------------+
| ``!`` | Negates a **Match** or a grouped expression |
+---------+----------------------------------------------------------------+
| ``AND`` | AND logical operator |
+---------+ |
| ``and`` | |
+---------+ |
| ``&&`` | |
+---------+----------------------------------------------------------------+
| ``OR`` | OR logical operator |
+---------+ |
| ``or`` | |
+---------+ |
| ``||`` | |
+---------+----------------------------------------------------------------+
.. important::
For every **Windows Event** tested against a rule the **Condition** is evaluated in real
time **from left to right**. As a consequence, the order of the variables to
check might have a small impact on the rule performances. For more efficiency
always try to put the more restrictive ones first.
Example:
^^^^^^^^
The following rule is used to match suspicious explicit network logons, we can
see an example of an advanced condition.
.. code-block:: JSON
{
"Name": "ExplicitNetworkLogon",
"Tags": ["Lateral", "Security"],
"Meta": {
"EventIDs": [4624],
"Channels": ["Security"],
"Computers": [],
"Criticality": 5,
"Author": "@0xrawsec"
},
"Matches": [
"$logt: LogonType = '3'",
"$user: TargetUserName = 'ANONYMOUS LOGON'",
"$iplh1: IpAddress = '-'",
"$iplh2: IpAddress = '127.0.0.1'",
"$enddol: TargetUserName ~= '\\$$'"
],
"Condition": "$logt and !($user or $iplh1 or $iplh2 or $enddol)"
}