Graylog extractors for Sophos UTM logs
I have created nine extractors to get all the Sophos variables I needed to use.
Extractor 1 (scrip)
Regular expression: (?i)srcip="([\d\.]+)"
Condition: Only attempt extraction if field contains string
Field contains string: srcip="
Store as field: srcip
Extraction strategy: Copy
Extractor title: srcip
Extractor 2 (id)
Regular expression: (?i)id="(\d+)"
Condition: Only attempt extraction if field matches regular expression
Field contains string: (?i)id="(\d+)"
Store as field: id
Extraction strategy: Copy
Extractor title: id
Extractor 3 (name)
Regular expression: (?i)name="([\s\w,]+)"
Condition: Only attempt extraction if field matches regular expression
Field contains string: (?i)name="([\s\w,]+)"
Store as field: name
Extraction strategy: Copy
Extractor title: name
Extractor 4 (user)
Regular expression: (?i)user="([\w\d\s]+)"
Condition: Only attempt extraction if field matches regular expression
Field contains string: (?i)user="([\w\d\s]+)"
Store as field: user
Extraction strategy: Copy
Extractor title: user
Extractor 5 (action)
Regular expression: (?i)action="([\w]+)"
Condition: Only attempt extraction if field matches regular expression
Field contains string: (?i)action="([\w]+)"
Store as field: action
Extraction strategy: Copy
Extractor title: action
Extractor 6 (dstip)
Regular expression: (?i)dstip="([\d\.]+)"
Condition: Only attempt extraction if field matches regular expression
Field contains string: (?i)dstip="([\d\.]+)"
Store as field: dstip
Extraction strategy: Copy
Extractor title: dstip
Extractor 7 (group)
Regular expression: (?i)group="([\w\s\d]+)"
Condition: Only attempt extraction if field matches regular expression
Field contains string: (?i)group="([\w\s\d]+)"
Store as field: group
Extraction strategy: Copy
Extractor title: group
Extractor 8 (categoryname)
Regular expression: (?i)categoryname="([\w\d\s]+)"
Condition: Only attempt extraction if field matches regular expression
Field contains string: (?i)categoryname="([\w\d\s]+)"
Store as field: categoryname
Extraction strategy: Copy
Extractor title: categoryname
Extractor 9 (uri)
Grok pattern: %{URI}
Condition: Only attempt extraction if field contains string
Field contains string: url="
Extraction strategy: Copy
Extractor title: uri
The Grok URI pattern is as follows on my system:
%{URIPROTO}://(?:%{USER}(?::[^@]*)?@)?(?:%{URIHOST})?(?:%{URIPATHPARAM})?
I then created a Decorator under search for the Sophos stream I created:
Format String: ${action} - ${srcip}(user: ${user}, group: ${group}) - ${URIHOST}
Target Field: webfilter
Require all fields: unchecked
For the Sophos stream I created I setup IP based rules to have the Sophos logs enter the stream.