Class Tokenizer

java.lang.Object
org.exolab.adaptx.util.Tokenizer

public class Tokenizer extends Object
Splits a String into tokens using a specified set of String delimiters. Delimeters are searched for in the that they appear in the delimiter list. If one delimiter is part of another delimiter, make sure you add them in the proper order for correct behavoir
Version:
$Revision: 3633 $ $Date: 2003-03-01 08:38:44 +0100 (Sat, 01 Mar 2003) $
Author:
Keith Visco
  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final String[]
    The set of delimiters to use, if none are specified
  • Constructor Summary

    Constructors
    Constructor
    Description
    Tokenizer(String pattern)
    Creates a new Tokenizer using the given String and the default set of delimiters.
    Tokenizer(String pattern, String[] delimiters)
    Creates a new Tokenizer using the given String and the default set of delimiters
  • Method Summary

    Modifier and Type
    Method
    Description
    void
    advance(int offset)
    Advances the index of this tokenizer ahead by the given offset
    int
    Counts the number of times nextToken can be called without returning null
    int
     
    boolean
    Determines if there are any tokens available
    boolean
    Determines if the specified token is contained in the token list of this Tokenizer
    boolean
    Determines if the specified token is a delimiter for this Tokenizer
    lookAhead(int offset)
    Allows looking ahead for tokens without affecting the token sequence as called by nextToken or previousToken
    Retrieves the next available token
    Retrieves the previous token
    void
    Resets the position of the token pointer to the beginning
    void
    setPosition(int position)
    Sets the current position of this tokenizer.
     

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, wait, wait, wait
  • Field Details

    • DEFAULT_DELIMITERS

      public static final String[] DEFAULT_DELIMITERS
      The set of delimiters to use, if none are specified
  • Constructor Details

    • Tokenizer

      public Tokenizer(String pattern)
      Creates a new Tokenizer using the given String and the default set of delimiters. The default delimiters are: " ", "\n", "\r", and "\t";
    • Tokenizer

      public Tokenizer(String pattern, String[] delimiters)
      Creates a new Tokenizer using the given String and the default set of delimiters
  • Method Details

    • advance

      public void advance(int offset)
      Advances the index of this tokenizer ahead by the given offset
    • countTokens

      public int countTokens()
      Counts the number of times nextToken can be called without returning null
    • getPosition

      public int getPosition()
    • hasMoreTokens

      public boolean hasMoreTokens()
      Determines if there are any tokens available
      Returns:
      true if there are tokens available, otherwise false
    • hasToken

      public boolean hasToken(String token)
      Determines if the specified token is contained in the token list of this Tokenizer
      Parameters:
      token - the String to look for in the token list
      Returns:
      true if the String argument is contained in this Tokenizer's token list, otherwise false
    • isDelimiter

      public boolean isDelimiter(String token)
      Determines if the specified token is a delimiter for this Tokenizer
      Parameters:
      token - the String to compare to the delimiters
      Returns:
      true if the String argument is a delimiter
    • lookAhead

      public String lookAhead(int offset)
      Allows looking ahead for tokens without affecting the token sequence as called by nextToken or previousToken
    • nextToken

      public String nextToken()
      Retrieves the next available token
      Returns:
      the next available token or null if there are none
    • previousToken

      public String previousToken()
      Retrieves the previous token
      Returns:
      the previous token or null if a previous token is not available
    • resetPosition

      public void resetPosition()
      Resets the position of the token pointer to the beginning
    • setPosition

      public void setPosition(int position)
      Sets the current position of this tokenizer.
      Parameters:
      position - the index value to set the current position to. if position is greater than the number of tokens, the tokenizer is advanced to the end. If the given position is less than 0, the tokenizer is positioned at the beginning
    • toString

      public String toString()
      Overrides:
      toString in class Object