Package org.languagetool.rules.ngrams
Class ConfusionProbabilityRule
java.lang.Object
org.languagetool.rules.Rule
org.languagetool.rules.ngrams.ConfusionProbabilityRule
LanguageTool's homophone confusion check that uses ngram lookups
to decide which word in a confusion set (from
confusion_sets.txt
) suits best.
Also see http://wiki.languagetool.org/finding-errors-using-n-gram-data.- Since:
- 2.7
-
Field Summary
FieldsModifier and TypeFieldDescriptionprivate static final com.google.common.cache.LoadingCache<String,
Map<String, List<ConfusionPair>>> private static final boolean
private final int
private final Language
private final LanguageModel
static final float
private static final double
static final String
private final Map<String,
List<ConfusionPair>> -
Constructor Summary
ConstructorsConstructorDescriptionConfusionProbabilityRule
(ResourceBundle messages, LanguageModel languageModel, Language language) ConfusionProbabilityRule
(ResourceBundle messages, LanguageModel languageModel, Language language, int grams) ConfusionProbabilityRule
(ResourceBundle messages, LanguageModel languageModel, Language language, int grams, List<String> exceptions) -
Method Summary
Modifier and TypeMethodDescriptionprivate boolean
covers
(int exceptionStartPos, int exceptionEndPos, int startPos, int endPos) private void
int
A number that estimates how many words there must be after a match before we can be (relatively) sure the match is valid.private ConfusionString
getAlternativeTerm
(List<ConfusionString> confusionSet, GoogleToken token) private @Nullable ConfusionString
getBetterAlternativeOrNull
(GoogleToken token, List<GoogleToken> tokens, List<ConfusionString> confusionSet, long factor) private ConfusionString
getBetterAlternativeOrNull
(GoogleToken token, List<GoogleToken> tokens, ConfusionString otherWord, long factor) private ConfusionString
getConfusionString
(List<ConfusionString> confusionSet, GoogleToken token) getContext
(GoogleToken token, List<GoogleToken> tokens, String newToken, int toLeft, int toRight) A short description of the error this rule can detect, usually in the language of the text that is checked.getId()
A string used to identify the rule in e.g.protected String
getMessage
(ConfusionString textString, ConfusionString suggestion) int
Returns the ngram level used, typically 3.getSuggestions
(String message) protected boolean
isException
(String sentenceText) Return true to prevent a match.private boolean
isLocalException
(AnalyzedSentence sentence, GoogleToken googleToken) match
(AnalyzedSentence sentence) Check whether the given sentence matches this error rule, i.e.void
Deprecated.used only for testsMethods inherited from class org.languagetool.rules.Rule
addExamplePair, getAntiPatterns, getCategory, getConfigureText, getCorrectExamples, getDefaultValue, getErrorTriggeringExamples, getIncorrectExamples, getLocQualityIssueType, getMaxConfigurableValue, getMinConfigurableValue, getSentenceWithImmunization, getUrl, hasConfigurableValue, isDefaultOff, isDefaultTempOff, isDictionaryBasedSpellingRule, isOfficeDefaultOff, isOfficeDefaultOn, makeAntiPatterns, setCategory, setCorrectExamples, setDefaultOff, setDefaultOn, setDefaultTempOff, setErrorTriggeringExamples, setIncorrectExamples, setLocQualityIssueType, setOfficeDefaultOff, setOfficeDefaultOn, setUrl, supportsLanguage, toRuleMatchArray, useInOffice
-
Field Details
-
RULE_ID
- Since:
- 3.1
- See Also:
-
MIN_COVERAGE
public static final float MIN_COVERAGE- See Also:
-
MIN_PROB
private static final double MIN_PROB- See Also:
-
DEBUG
private static final boolean DEBUG- See Also:
-
confSetCache
private static final com.google.common.cache.LoadingCache<String,Map<String, confSetCacheList<ConfusionPair>>> -
wordToPairs
-
lm
-
grams
private final int grams -
language
-
exceptions
-
-
Constructor Details
-
ConfusionProbabilityRule
public ConfusionProbabilityRule(ResourceBundle messages, LanguageModel languageModel, Language language) -
ConfusionProbabilityRule
public ConfusionProbabilityRule(ResourceBundle messages, LanguageModel languageModel, Language language, int grams) -
ConfusionProbabilityRule
public ConfusionProbabilityRule(ResourceBundle messages, LanguageModel languageModel, Language language, int grams, List<String> exceptions) - Since:
- 4.7
-
-
Method Details
-
getFilenames
-
getId
Description copied from class:Rule
A string used to identify the rule in e.g. configuration files. This string is supposed to be unique and to stay the same in all upcoming versions of LanguageTool. It's supposed to contain only the charactersA-Z
and the underscore. -
estimateContextForSureMatch
public int estimateContextForSureMatch()Description copied from class:Rule
A number that estimates how many words there must be after a match before we can be (relatively) sure the match is valid. This is useful for check-as-you-type, where a match might occur and the word that gets typed next makes the match disappear (something one would obviously like to avoid). Note: this may over-estimate the real context size. Returns-1
when the sentence needs to end to be sure there's a match.- Overrides:
estimateContextForSureMatch
in classRule
-
match
Description copied from class:Rule
Check whether the given sentence matches this error rule, i.e. whether it contains the error detected by this rule. Note that the order in which this method is called is not always guaranteed, i.e. the sentence order in the text may be different than the order in which you get the sentences (this may be the case when LanguageTool is used as a LibreOffice/OpenOffice add-on, for example). -
isLocalException
-
covers
private boolean covers(int exceptionStartPos, int exceptionEndPos, int startPos, int endPos) -
getSuggestions
-
isException
Return true to prevent a match. -
getDescription
Description copied from class:Rule
A short description of the error this rule can detect, usually in the language of the text that is checked.- Specified by:
getDescription
in classRule
-
getMessage
-
setConfusionPair
Deprecated.used only for tests -
getNGrams
public int getNGrams()Returns the ngram level used, typically 3.- Since:
- 3.1
-
getBetterAlternativeOrNull
@Nullable private @Nullable ConfusionString getBetterAlternativeOrNull(GoogleToken token, List<GoogleToken> tokens, List<ConfusionString> confusionSet, long factor) -
getAlternativeTerm
-
getConfusionString
-
getBetterAlternativeOrNull
private ConfusionString getBetterAlternativeOrNull(GoogleToken token, List<GoogleToken> tokens, ConfusionString otherWord, long factor) -
getContext
List<String> getContext(GoogleToken token, List<GoogleToken> tokens, String newToken, int toLeft, int toRight) -
debug
-