LEA (Language Engineering & Analysis) is a natural language understanding engine developed based on machine learning/deep learning in Saltlux AI Suite. It can handle text analysis functions such as morpheme analysis, object name recognition, sentence structure analysis, and emotion analysis for non-structured data processing. In addition, our natural language processing is also used to provide higher-level analysis results, such as understanding hidden intentions in sentences or identifying the types of questions. This function allows the system to understand the purposes of conversation processing and the meaning of questions for deep Q/A analysis. LEA is the basic engine needed for AI Suite’s engines to operate.
The high-precision language analyzers that make up the natural language engine can be used to optimize the quality of each domain through large-scale language resources (large-capacity learning data, dictionaries, and rules). Morpheme analyzer provides quality analysis of more than 98% and syntax analysis, while object name extractors offer the world’s best performance through parallel/distributed processing. LEA engine enables multilingual responses in Korean, English, Japanese, and others. It is a natural language processing engine that could realize semantic analysis, Q&A, and dialogue systems by connecting with knowledge graphs.
< Natural Language Understanding Engine – LEA block diagram >
Main Features
High-quality natural language processing based on machine learning and deep learning
As mentioned, the language analyzers are applied with machine learning and deep learning technologies. Morpheme analysis and object name recognition were based on the latest machine learning model Structural-SVM. The positive/negative emotional analysis was based on Latent Structural-SVM, while the dependent sentence structure analysis was based on Transition-based (Arc-Eager) Dependency Parsing. Together they provide faster and higher performance than existing algorithms. In addition, Word Embedding makes it possible to apply deep learning to natural language processing.
Ease of domain application
Unlike other natural language engines where language processing involves common words (terms), the LEA engine can optimize the quality of each domain by using large language resources. It supports features of separately building large-capacity learning data to allow learning. In addition to the common dictionaries, dictionaries and rules specific to each domain are utilized. Thus, analysis can be tailored to the linguistic characteristics unique to such areas as healthcare, finance, and law.
Meaning identification through the connection with the knowledge graph
The natural language understanding engine identifies not only natural language processing like morpheme analysis and sentence structure analysis, but also the objects and meanings of the analyzed words. Such meaning identification is made possible through connection with the knowledge base. The knowledge graph is used to determine which objects are referred to in the sentence and tag them in a form that the machine can understand. In addition, the semantic information of the knowledge graph is used as core information for AI knowledge processing. For example, it can grasp the intention of spoken sentences in dialogue processing or identify the question’s type and meaning in Q&A processing.
Main functions and specifications
Natural language processing features
The natural language understanding engine provides basic natural language processing features for non-structured text entered through morpheme analyzers, object name recognizers, and sentence structure analyzers.
– Morpheme analysis feature
– Sentence structure analysis feature
– Object name extraction feature
– Reputation(emotion) extraction feature
Intention analysis feature
This feature analyzes not only the lexical meaning expressed in a sentence, but also the meaning of the This feature analyzes the linguistic meaning expressed in a sentence and its intention while also proposing meaning-based classification results. Unlike natural language processing using simple dictionaries or rules that are sensitive to spacing or typos, this feature recursively reconstructs and re-analyzes sentences until an optimal analysis is deducted. In addition to correcting errors in an input sentence, this process could provide a robust analysis of errors in the sentences that users input in dialogue processing or Q&A.
Question understanding features
LEA provides not only simple natural language processing results but also semantic analysis results that understand user questions in conversation processing as well as Q&A processing.
The semantic object is determined by linking the NLP’s analysis results with the Knowledge Graph information. This function also determines whether the sentence is a declarative sentence or an interrogative sentence. In case it is a question, the type of the question will be classified. As such, by analyzing semantic and syntactic knowledge information included in a sentence and deriving their results, it can understand the content and intent of the question. Question understanding is the core function of AI services’ cognitive/understanding process. It combines high-level language recognition technology with deep learning-based language/intention/knowledge learning technology with intention analysis.
Dictionary management features
It provides a dictionary management function in which major language dictionaries used by the engine are integrated and managed. The web-based integrated language dictionary management feature makes it easy to add and reflect language resources such as essential terms in a specific domain, or words and synonyms that need to be excluded during language processing. This feature provides analysis results that are customized based on users or domains, along with improving the quality of language processing through periodic and continuous management.
Main engine screen
