Documentation

Overview

The Text Truncator with File Upload application is designed to provide users with tools to manipulate text data efficiently. This document provides detailed information on each feature and guides on how to use them.

Features

• Token Limit: Allows users to set a limit on the number of words (tokens) to include in the processed text.
• File Upload: Users can upload text files from which the text is extracted and displayed in the input area for processing.
• Cleaning Options: Multiple options are provided to strip unnecessary or unwanted text components, such as special characters, numbers, punctuation, or emojis.

Getting Started

File Upload

To upload a text file:

1. Click the "Choose File" button to browse and select a file from your local system.
2. Once a file is selected, its contents will automatically be loaded into the input area.

Setting Token Limit

Define the maximum number of tokens for the truncation process in the "Max tokens" field. If left blank, the default value of 7000 tokens is used.

Text Cleaning Options

The application offers various cleaning options that can be selected before processing the text:

• Convert to Lowercase: Changes all uppercase characters to lowercase.
• Remove Special Characters: Deletes characters like @, #, $, and others that are not alphanumeric.
• Remove Numbers: Strips away all digits from the text.
• Remove Punctuation: Removes punctuation marks such as periods, commas, and question marks.
• Remove Emojis: Clears all emojis from the text.

Processing the Text

Truncate Text

After setting your preferences, click the "Truncate Text" button to apply the token limit. The processed text will appear in the "Truncated Text" area.

Clean Text

Select the desired cleaning options and click the "Clean Text" button to apply those filters to the text in the input area.