Package cc.redpen.tokenizer

Provides tokenizers for each languages.

See: Description

Interface Summary

Interface	Description
RedPenTokenizer

Class Summary

Class	Description
JapaneseTokenizer
TokenElement
WhiteSpaceTokenizer

Package cc.redpen.tokenizer Description

Provides tokenizers for each languages.