Here are a few things about lispy expressions to keep in mind when crafting a search. In prefix notation, the operators like AND, OR, and NOT appear before the operands, or search terms. Lispy expressions use a variant of prefix notation, a mathematical notation used to write equations. The way we craft our search affects how the lispy expression is written and how Splunk finds and returns events. Splunk uses the lispy expression to build bloom filters and locate terms in TSIDX files. Using major and minor breakers, Splunk has tokenized, separated, and reordered our search string as a lispy expression. We can see the parts of our original search. This takes us to the actual lispy expression Splunk has created from our search query. We look through the search log to find the phrase "base lispy". If we know what to look for, we can find the expressions that will be used to build a bloom filter and check against TSIDX files. There's a lot of information in the search log.
Then we open up the search job inspector and click on the link to the search log. We'll start with a search for an IP address. Let's look at how a search is broken up into tokens. Knowing how segmentation affects search performance can help us understand what makes some searches faster than others. For more information on segmentation, we have included a link in the notes. At search time, Splunk uses the tokens to build a search time bloom filter and match these terms in TSIDX files. At index time, these tokens become part of TSIDX files' lexicons, and are used to build bloom filters when buckets roll from hot to warm. The point of segmentation is to quickly return a set of tokens. Some minor breakers include forward slashes, colons, periods, hyphens, and dollar signs. The second pass of segmentation then finds minor breakers within the results of the first pass to break them up further. Some major breakers include spaces, new lines, carriage returns, tabs, brackets, exclamation points, and commas. Major breakers are used to isolate words, phrases, terms, and numerical data during the first part of segmentation. Segmentation happens in two stages, splitting events up by finding characters that we call major breakers and minor breakers. When it finds one, it reads the matching events from disk. During search, it compares the search tokens to event tokens, looking for a match between the two. It also tokenizes search terms at search time.
Earlier, we mentioned that Splunk tokenizes terms and events and stores them in TSIDX files at index time.
0 kommentar(er)
