stringsparser

go-strings-parser

strings.Split on steroids

A flexible Go library for parsing strings with support for separators, quoting, escaping, and character set validation.

Features

• Configurable separators - Support for multiple separator runes (spaces, tabs, commas, or custom)
• Quote handling - Single and double quote support for preserving spaces and special characters
• Escape sequences - Configurable escape processing (\n, \t, \\, \", \', etc.)
• Empty element control - Choose whether to preserve or skip empty elements
• Character set validation - Predefined charsets for POSIX paths, Windows paths, and alphanumeric text
• Custom processing - Apply custom transformation functions to parsed elements
• Detailed errors - Error reporting with character positions for debugging

Installation

go get github.com/4nd3r5on/go-strings-parser

Quick Start

import sp "github.com/4nd3r5on/go-strings-parser"

// Basic parsing with default options
result, err := sp.Parse("foo bar 'baz qux'")
// Returns: ["foo", "bar", "baz qux"]

// Parse with quotes
result, err := sp.Parse("hi 'hi hi' hello")
// Returns: ["hi", "hi hi", "hello"]

// Parse with escaped quotes
result, err := sp.Parse("hi \"hi \\\" hi\" hello")
// Returns: ["hi", "hi \" hi", "hello"]

Usage Examples

Custom Separators

// Use comma as separator
result, err := sp.Parse("apple,orange,banana", 
    sp.WithSeparators(','))
// Returns: ["apple", "orange", "banana"]

Allow Empty Elements

// Preserve empty elements between separators
result, err := sp.Parse("a   b", 
    sp.WithAllowEmpty(true))
// Returns: ["a", "", "", "b"]

// Skip empty elements (default behavior)
result, err := sp.Parse("a   b", 
    sp.WithAllowEmpty(false))
// Returns: ["a", "b"]

Windows Path Validation

// Valid Windows path
result, err := sp.Parse("C:\\Users\\Documents\\file.txt", 
    sp.WithWindowsPath())

// Invalid path (contains illegal characters < >)
result, err := sp.Parse("C:\\user\\invalid<>\\file.txt", 
    sp.WithWindowsPath())
// Returns error

Custom Processing Functions

// Trim whitespace from each element
result, err := sp.Parse("'fine' ' clean space' '  clean both spaces  '", 
    sp.WithProcessFunc(func(s string) (string, bool, error) {
        return strings.TrimSpace(s), false, nil
    }),
    sp.WithAllowEmpty(false))
// Returns: ["fine", "clean space", "clean both spaces"]

// Skip elements based on condition
result, err := sp.Parse("apple banana cherry", 
    sp.WithProcessFunc(func(s string) (string, bool, error) {
        if s == "banana" {
            return "", true, nil // skip this element
        }
        return s, false, nil
    }))
// Returns: ["apple", "cherry"]

// Convert to uppercase
result, err := sp.Parse("hello world", 
    sp.WithProcessFunc(func(s string) (string, bool, error) {
        return strings.ToUpper(s), false, nil
    }))
// Returns: ["HELLO", "WORLD"]

Custom Rune Processing

// Custom escape replacement
customEscapes := map[rune]rune{
    'n': '\n',
    't': '\t',
    's': ' ',  // \s becomes space
}

result, err := sp.Parse("hello\\sworld\\n", 
    sp.WithProcessRuneFuncs(
        sp.NewReplaceEscapedFunc(customEscapes),
    ))
// Returns: ["hello world\n"]

// Combine multiple rune processors
result, err := sp.Parse("path/to/file", 
    sp.AppendProcessRuneFuncs(
        sp.NewCharsetProcessRuneFunc(posixCharset),
    ))
// Default escape processing + charset validation

How It Works

The parser processes input strings character by character through a state machine:

1. Quote Detection - Recognizes single (') and double (") quotes to group elements
2. Escape Handling - Processes backslash escapes (\n, \t, \", etc.)
3. Separator Recognition - Splits elements at configured separator characters
4. Character Processing - Applies rune-level transformations via ProcessRuneFunc
5. Element Processing - Applies string-level transformations via ProcessFunc
6. Validation - Checks character sets if configured

Parsing Rules

• Quotes must appear at the start of an element (after a separator or at the beginning)
• Escaped characters are processed according to configured rules
• Separators inside quotes are treated as literal characters
• Empty elements between separators can be preserved or skipped
• The parser maintains strict validation of quote pairing and escape sequences

Configuration Options

Default Configuration

DefaultSeparators = []rune{' ', '\t', ','}

DefaultReplaceEscaped = map[rune]rune{
    'n': '\n',
    'r': '\r',
    't': '\t',
    '0': '\000',
}

DefaultProcessRuneFuncs = []ProcessRuneFunc{
    NewReplaceEscapedFunc(DefaultReplaceEscaped),
}

Available Options

• WithSeparators(...rune) - Set custom separator characters
• WithAllowEmpty(bool) - Control empty element preservation
• WithProcessFunc(ProcessFunc) - Apply custom transformation to each element
• WithProcessRuneFuncs(...ProcessRuneFunc) - Set rune-level processing functions (replaces defaults)
• AppendProcessRuneFuncs(...ProcessRuneFunc) - Add additional rune processors (keeps defaults)
• WithCharset(*Charset) - Validate against custom character set
• WithWindowsPath() - Use Windows path charset and validation

Function Signatures

// ProcessFunc transforms completed elements
// - processed: the transformed string
// - skip: if true, element is discarded
// - err: processing error
type ProcessFunc func(s string) (processed string, skip bool, err error)

// ProcessRuneFunc processes individual characters
// - idx: character position in original string
// - current: string built so far
// - escaped: whether this rune was escaped with backslash
// - r: the rune to process
type ProcessRuneFunc func(idx int, current string, escaped bool, r rune) (
    processed rune, skip bool, err error)

type Option func(*Options)

Error Handling

The parser provides detailed error messages with character positions:

result, err := sp.Parse("'unclosed quote")
if err != nil {
    // Error: unclosed quote
    log.Fatal(err)
}

result, err := sp.Parse("hello 'there")
if err != nil {
    // Error: unclosed quote
    log.Fatal(err)
}

result, err := sp.Parse("hello\\")
if err != nil {
    // Error: dangling escape
    log.Fatal(err)
}

result, err := sp.Parse("hel'lo")
if err != nil {
    // Error: invalid character ''' at index 3: unexpected unescaped quote inside token
    var invalidCharErr *sp.InvalidCharError
    if errors.As(err, &invalidCharErr) {
        fmt.Printf("Invalid char '%c' at position %d\n", 
            invalidCharErr.Char, invalidCharErr.Index)
    }
}

Error Types

• ErrUnclosedQuote - Quote opened but not closed
• ErrDanglingEscape - Backslash at end of input with nothing to escape
• ErrUnexpectedQuote - Quote appears in middle of unquoted token
• ErrNotInTheCharset - Character not allowed by configured charset
• InvalidCharError - Wraps errors with character position information

Common Use Cases

Parsing Command-Line Arguments

args, err := sp.Parse(`command --flag "value with spaces" --another-flag`)
// Returns: ["command", "--flag", "value with spaces", "--another-flag"]

Parsing CSV-Like Data

fields, err := sp.Parse("John,Doe,\"New York, NY\"", 
    sp.WithSeparators(','))
// Returns: ["John", "Doe", "New York, NY"]

Parsing File Paths

paths, err := sp.Parse("/home/user /opt/app '/path/with spaces'")
// Returns: ["/home/user", "/opt/app", "/path/with spaces"]

Building a Command Parser

func parseCommand(input string) ([]string, error) {
    return sp.Parse(input,
        sp.WithSeparators(' ', '\t'),
        sp.WithAllowEmpty(false),
        sp.WithProcessFunc(func(s string) (string, bool, error) {
            // Trim and skip empty
            trimmed := strings.TrimSpace(s)
            if trimmed == "" {
                return "", true, nil
            }
            return trimmed, false, nil
        }),
    )
}

parts, err := parseCommand(`git commit -m "Initial commit"`)
// Returns: ["git", "commit", "-m", "Initial commit"]

Advanced: Conditional Processing

// Parse and validate email addresses
result, err := sp.Parse("[email protected] [email protected]", 
    sp.WithProcessFunc(func(s string) (string, bool, error) {
        if !strings.Contains(s, "@") {
            return "", false, fmt.Errorf("invalid email: %s", s)
        }
        return strings.ToLower(s), false, nil
    }))