Yarnspinnex.Parser.Lexer (yarnspinnex v0.3.0)

Copy Markdown View Source

Tokenizes a Yarn expression source string. Pure source-to-tokens - no knowledge of expression grammar or the AST; that lives in Yarnspinnex.Parser.Expression.

Summary

Types

token()

Functions

tokenize(source)

Splits expression source into a list of tokens.

Types

token()

@type token() ::
  {:num, number()}
  | {:str, String.t()}
  | {:bool, boolean()}
  | {:var, String.t()}
  | {:ident, String.t()}
  | {:op, atom()}
  | :lparen
  | :rparen
  | :comma
  | :dot

Functions

tokenize(source)

@spec tokenize(String.t()) :: [token()]

Splits expression source into a list of tokens.