Constructor and Description |
---|
Tokens(String expression)
Construct a
TokenProcessor based on the provided string expression. |
Modifier and Type | Method and Description |
---|---|
protected void |
checkpoint()
Indicate that the current token has been successfully processed.
|
protected int |
decrementPosition()
Decrement the current token position and return the new position.
|
protected Token |
eat(TokenKind expectedKind)
Consume the next token if it matches the indicated token kind;
otherwise throw
CheckPointedParseException . |
protected String |
getExpression()
Return the expression string converted to tokens.
|
List<Token> |
getTokenStream()
Return an immutable list of
tokens |
protected boolean |
hasNext()
Return
true if there are more tokens to process. |
protected boolean |
isNextAdjacent()
Return
true if the first character of the token at the current position
is the same as the last character of the token at the previous position. |
protected boolean |
lookAhead(int distance,
TokenKind desiredTokenKind)
Return
true if the token in the position indicated
by position + distance matches
the token indicated by desiredTokenKind . |
protected Token |
next()
Return the next
Token and advance the current token position. |
protected Token |
peek()
Return the token at the current position.
|
protected boolean |
peek(TokenKind desiredTokenKind)
Return
true if the indicated token matches the current token
position. |
protected int |
position()
Return the current token position.
|
protected void |
raiseException(int position,
DSLMessage message,
Object... inserts)
Throw a new
CheckPointedParseException based on the current and
last successfully processed token position. |
protected String getExpression()
protected int decrementPosition()
protected int position()
public List<Token> getTokenStream()
tokens
protected boolean lookAhead(int distance, TokenKind desiredTokenKind)
true
if the token in the position indicated
by position
+ distance
matches
the token indicated by desiredTokenKind
.distance
- number of token positions past the current positiondesiredTokenKind
- the token to check fordesiredTokenKind
protected boolean hasNext()
true
if there are more tokens to process.true
if there are more tokens to processprotected Token peek()
null
.null
if there are no more tokensprotected boolean peek(TokenKind desiredTokenKind)
true
if the indicated token matches the current token
position.desiredTokenKind
- token to matchprotected Token next()
Token
and advance the current token position.Token
protected Token eat(TokenKind expectedKind)
CheckPointedParseException
.expectedKind
- the expected token kindCheckPointedParseException
- if the next token does not match
the expected token kindprotected boolean isNextAdjacent()
true
if the first character of the token at the current position
is the same as the last character of the token at the previous position.protected void checkpoint()
lastGoodPosition
protected void raiseException(int position, DSLMessage message, Object... inserts)
CheckPointedParseException
based on the current and
last successfully processed token position.position
- position where parse error occurredmessage
- parse exception messageinserts
- variables that may be inserted in the error messageCopyright © 2016 Pivotal Software, Inc.. All rights reserved.