Next: Location accessors, Previous: Recognizer reference counting, Up: Recognizer methods [Contents][Index]
When successful, does the following:
MARPA_EVENT_EXHAUSTED
event.
See Exhaustion.
MARPA_EVENT_SYMBOL_NULLED
,
MARPA_EVENT_SYMBOL_PREDICTED
, or MARPA_EVENT_SYMBOL_EXPECTED
events.
See Events.
Return value: On success, a non-negative value, whose value is otherwise unspecified. On hard failure, -2.
The token_id argument must be the symbol ID of a terminal. The value argument is an integer that represents the “value” of the token, and which should not be zero. The length argument is the length of the token, which must be greater than zero.
On success, does the following, where current is the value of the current earleme before the call and furthest is the value of the furthest earleme before the call:
current+length
.
max(current+length,furthest)
.
After recoverable failure, the following are the case:
Libmarpa allows tokens to be ambiguous. Two tokens are ambiguous if they end at the same earleme location. If two tokens are ambiguous, Libmarpa will attempt to produce all the parses that include either of them.
Libmarpa allows tokens to overlap. Let the notation t@s-e indicate that token t starts at earleme s and ends at earleme e. Let t1@s1-e1 and t2@s2-e2 be two tokens such that s1<=s2. We say that t1 and t2 overlap iff e1>s2.
The value argument is not used inside Libmarpa — it is simply stored to be returned by the valuator as a convenience for the application. In applications where the token’s actual value is not an integer, it is expected that the application will use value as a “virtual” value, perhaps finding the actual value by using value to index an array. Some applications may prefer to track token values on their own, perhaps based on the earleme location and token_id, instead of using Libmarpa’s token values.
A value of 0 does not cause a failure, but it is reserved for unvalued symbols, a now-deprecated feature. See Valued and unvalued symbols.
Hard fails irrecoverably with MARPA_ERR_DUPLICATE_TOKEN
if the token added would be a duplicate.
Two tokens are duplicates iff all of the following are true:
marpa_r_alternative()
attempts
to read them while at the same current earleme.
If a token was not accepted
because of its token ID,
hard fails with the MARPA_ERR_UNEXPECTED_TOKEN_ID
.
This hard failure is fully recoverable
so that, for example,
the application may
retry this method with different token IDs
until it succeeds.
These retries are efficient,
and are quite useable as a parsing
technique —
so much so we have given the technique a name:
the Ruby Slippers.
The Ruby Slippers are used in several
applications.
Return value: On success, MARPA_ERR_NONE
.
On failure, an error code other than MARPA_ERR_NONE
.
The hard failure for MARPA_ERR_UNEXPECTED_TOKEN_ID
is fully recoverable.
For the purposes of this method description, we define the following:
marpa_r_earleme_complete
.
marpa_r_earleme_complete
.
marpa_r_terminal_is_expected()
determines if a terminal is “expected”
at the current earleme.
See marpa_r_terminals_expected().
marpa_r_alternative()
to end at an earleme after the current
earleme.
An anticipated terminal will have length greater than one.
“Anticipated” terminals only occur if the application is using
an advanced model of input.
See Advanced input models.
On success, does the final processing for the current earleme, including the following:
current+1
.
current+1
.
marpa_r_earleme_complete()
.
MARPA_EVENT_SYMBOL_COMPLETED
,
MARPA_EVENT_SYMBOL_NULLED
, MARPA_EVENT_SYMBOL_PREDICTED
,
or MARPA_EVENT_SYMBOL_EXPECTED
events.
See Events.
MARPA_EVENT_EARLEY_ITEM_THRESHOLD
event.
Often, the application will want to treat this event
as if it were a ancestry-recoverable
failure.
See marpa_r_earley_item_warning_threshold_set().
MARPA_EVENT_EXHAUSTED
event.
Exhaustion on success only occurs if no terminals
are expected at the current earleme after
the call to this method
(that is, at current+1
)
and no terminals are anticipated
after current+1
.
On hard failure
with the code MARPA_ERR_PARSE_EXHAUSTED
, does the following:
marpa_r_earleme_complete()
.
MARPA_EVENT_EXHAUSTED
event and no others.
MARPA_ERR_PARSE_EXHAUSTED
to be fully recoverable.
We note that exhaustion can occur when this method fails and when it succeeds. The distinction is that, on success, the call creates a new Earley set before becoming exhausted while, on failure, it becomes exhausted without creating a new Earley set.
This method is commonly called at the top of a loop.
Almost all applications will want to check the return value
and take special action in case of a value other than zero.
If the value is greater than zero, an event will have occurred
and almost all applications
should react to MARPA_EVENT_EARLEY_ITEM_THRESHOLD
events,
as described above,
and to unexpected events.
If the value is less than zero,
it may be due to an irrecoverable error,
and only in very unusual circumstances will an application wish
to ignore these.
How an application reacts to exhaustion will depend on the kind of parsing it is doing:
MARPA_EVENT_SYMBOL_COMPLETED
event.
Typically, these applications will treat exhaustion on method
failure and exhaustion before the end of input
as parse errors.
They may wish to ignore exhaustion on method success
at the end of input.
Return value: On success, the number of events generated.
On hard failure, -2.
Hard failure with the code
MARPA_ERR_PARSE_EXHAUSTED
is fully recoverable.
Next: Location accessors, Previous: Recognizer reference counting, Up: Recognizer methods [Contents][Index]