Tokenizer is provided for CLARIN VLO!

The Tokenizer is provided for the CLARIN VLO.  This service is intended to locate tokens in the text. The text that requires tokenization is sent to the service input. After processing the text on the output, the user receives a list of the extended tokens. Figures and punctuation marks are processed too. The token is a wordform, e. g. English come, came, coming, or cat, cats, or cases of Belarusian ‘hand’ рука, руку, рукі, руцэ, etc. The service handles Belarusian, Russian and English. The  service is available via the API. Details are presented  here. The direct link is here.