Lexer

Lexer

Lexical analyzer.

Analyze each lexeme from the given text in according to language alphabet/terminals/grammar.

Constructor

new Lexer(text) → {Lexer}

Source:

compiler/lexer.js, line 19

Lexer constructor.

Parameters:

Name	Type	Description
`text`	string	plain text as code source

Returns:

instance of Lexer analyzer

Type: Lexer

Methods

tokenize() → {Array.<Token>}

Source:

compiler/lexer.js, line 41

Tokenize source code.

Analyze source text/code in according to language grammar and produce a structure of tokens.

Returns:

array of Token instances

Type: Array.<Token>