Utils

GetLines

The GetLines function takes a string input and splits it into a slice of lines.
It reads the input line by line using a buffered reader and preserves the line endings.
Even the last line is included correctly, whether or not it ends with a newline character.
This function is useful for processing multi-line text, such as source code or configuration files, line by line.

lines := cc_lib.GetLines(string(input))

Parameters

Name	Type	Description
input	string	The multi-line string to split into lines.

Returns

Type	Description
[]string	Slice of strings, each representing a line from the input.

Tokenize

The Tokenize function splits a string into a slice of tokens using a regular expression.
It separates sequences of word characters (\w+) and non-word characters (\W+) as individual tokens.
This is useful for lexical analysis, parsing, or any scenario where you need to break text into meaningful units, including punctuation and symbols.

for _, line := range lines {
	tokens := cc_lib.Tokenize(line)
        for _, tok := range tokens {
            fmt.println("%s", tok)
        }
    }

Parameters

Name	Type	Description
tokens	string	The string to be split into tokens.

Returns

Type	Description
[]string	Slice of strings representing individual tokens (words and symbols).