Lucene Query Syntax

Searching your Data

Filters are very easy to create or modify without making mistakes, but sometimes a filter won't be sufficient. You will run into limitations that only the Lucene Query Syntax can resolve. For example:

  • When two or more Filters are added, they will always work as a boolean AND statement
  • You can't use Wildcards or Fuzziness
  • Regular Expressions

Let's review basic Lucene Query Syntax:

Free Text Search: enter a text value and it will be searched within all Analyzed and not Analyzed fields

  • Example: entering the word exception will find all records containing the word exception.

Value in a Specific Field: search for a value in a field you specify

  • Example: verb:GET will find all of the records that contain the value GET in the field verb.

Range Value: search a field with a numeric data type for values within a specified range[starting-value TO ending-value]

  • Example: entering the following search response:[400 TO 599] will return all records where the response field has values between 400 to 599.

Boolean Operators: Use AND, OR, and NOT (always capitals letters) to combine multiple searches with Boolean operators

  • Example: NOT response:[200 TO 399] AND (verb:GET OR exception) to find all records that do not have the response field with values between 200 to 399 and have the verb field with the value GET or the word exception.

Exists: search for logs that contain a specific field

  • Example: _exists_:response will only show records that contain the field response regardless of the value they have.

Reserved Characters: should be escaped with a leading backslash: + - = && || > < ! ( ) { } [ ] ^ " ~ * ? : \ / 

  • Example: if you wanted to search for the following value request:/item/games/2697 you would need to write request:\/item\/games\/2697

Wildcards, Regexes and Fuzzy Searching:

Wildcards - you can use the ? character to match a single character or the * character for multiple characters:

  • Ma?s — Will match Mars, Mass, and Maps
  • Ma*s — Will match Mars, Matches, and Massachusetts

Regex - searching with regexes give you even more flexibility. Place your regex between forward-slashes (/), for example:

  • /m[ea]n/ — Will match both men and man
  • /[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}/ — Will match an IP address ranging from 0.0.0.0 to 255.255.255.255

Fuzzy - searching with fuzziness (~)will help matching terms that are similar in spelling using the Damerau-Levenshtein Distance to find terms with a maximum of two changes, and this can be useful when your data may have misspelled words, for example:

  • john~ — Will match, amongst others, jean, johns, jhon, and horn
  • jack~ — Will match, amongst others, mack, neck, jock, and mock