From: Seth Nielson (sethjn_at_[hidden])
Date: 2006-07-24 14:31:28
Hi
Two questions related to string tokenization.
1. Which is preferred? using "split" or the "tokenizer class"?
2. Both of these methods seem geared towards splitting on characters
rather than splitting on substrings. Is there yet another method that is
preferred for splitting a string on an exact substring? If I want to
split "I<mark>Am<mark>A<mark>Test" into I, Am, A, Test, what is the best
way? It seems that for split I'll have to write my own predicate, and
for tokenizer, I'll have to write my own tokenizerFunction.
-- Seth N.