Uses of Interface
cc.redpen.tokenizer.RedPenTokenizer

Packages that use RedPenTokenizer

Package	Description
cc.redpen.config	Configuration and SymbolTable are provided.
cc.redpen.model	Elements of Documents such as List, Sentence are provided.
cc.redpen.parser	Parser and the implementations are provided.
cc.redpen.tokenizer	Provides tokenizers for each languages.

Uses of RedPenTokenizer in cc.redpen.config

Methods in cc.redpen.config that return RedPenTokenizer

Modifier and Type	Method and Description
RedPenTokenizer	Configuration.getTokenizer() returns Tokenizer aasociated with this configuration

Uses of RedPenTokenizer in cc.redpen.model

Constructors in cc.redpen.model with parameters of type RedPenTokenizer

Constructor and Description
DocumentBuilder(RedPenTokenizer tokenizer) Constructor.

Uses of RedPenTokenizer in cc.redpen.parser

Methods in cc.redpen.parser with parameters of type RedPenTokenizer

Modifier and Type	Method and Description
Document	DocumentParser.parse(File file, SentenceExtractor sentenceExtractor, RedPenTokenizer tokenizer) Given input file name, return Document instance for the specified file.
Document	BaseDocumentParser.parse(File file, SentenceExtractor sentenceExtractor, RedPenTokenizer tokenizer)
Document	PlainTextParser.parse(InputStream is, Optional<String> fileName, SentenceExtractor sentenceExtractor, RedPenTokenizer tokenizer)
Document	DocumentParser.parse(InputStream io, Optional<String> fileName, SentenceExtractor sentenceExtractor, RedPenTokenizer tokenizer) Given input stream, return Document instance from a stream.
Document	AsciiDocParser.parse(InputStream io, Optional<String> fileName, SentenceExtractor sentenceExtractor, RedPenTokenizer tokenizer)
default Document	DocumentParser.parse(InputStream is, SentenceExtractor sentenceExtractor, RedPenTokenizer tokenizer) Given input stream, return Document instance from a stream.
Document	DocumentParser.parse(String content, SentenceExtractor sentenceExtractor, RedPenTokenizer tokenizer) Given content, return Document instance for the specified file.
Document	BaseDocumentParser.parse(String content, SentenceExtractor sentenceExtractor, RedPenTokenizer tokenizer)

Uses of RedPenTokenizer in cc.redpen.tokenizer

Classes in cc.redpen.tokenizer that implement RedPenTokenizer

Modifier and Type	Class and Description
class	JapaneseTokenizer
class	WhiteSpaceTokenizer