Package org.apache.manifoldcf.crawler.connectors.webcrawler

Class WebcrawlerConnector.EvaluatorTokenStream

java.lang.Object

org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector.EvaluatorTokenStream

Enclosing class:

WebcrawlerConnector

protected static class WebcrawlerConnector.EvaluatorTokenStream
extends java.lang.Object

Token stream.

Field Summary

Fields

Modifier and Type	Field	Description
protected int	pos
protected java.lang.String	text
protected WebcrawlerConnector.EvaluatorToken	token

Constructor Summary

Constructors

Constructor	Description
EvaluatorTokenStream(java.lang.String text)	Constructor.

Method Summary

Modifier and Type	Method	Description
void	advance()	Go on to next token.
protected WebcrawlerConnector.EvaluatorToken	nextToken()
WebcrawlerConnector.EvaluatorToken	peek()	Get current token.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

text

protected java.lang.String text

pos

protected int pos

token

protected WebcrawlerConnector.EvaluatorToken token

Constructor Detail

EvaluatorTokenStream

public EvaluatorTokenStream(java.lang.String text)

Constructor.

Method Detail

peek

public WebcrawlerConnector.EvaluatorToken peek()
                                        throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Get current token.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

advance

public void advance()

Go on to next token.

nextToken

protected WebcrawlerConnector.EvaluatorToken nextToken()
                                                throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException